nano-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Nano-devel] reducing system calls for file io


From: Jay Carlson
Subject: Re: [Nano-devel] reducing system calls for file io
Date: Sun, 21 Apr 2002 14:40:52 -0400

On Sunday, April 21, 2002, at 12:10 PM, Adam Rogoyski wrote:

On Sat, 20 Apr 2002, Jay Carlson wrote:

Because these kinds of problems affect many applications, there are
libraries that can reduce this overhead.  One of them is called libc,
and the subsystem implementing this is available through #include

The correct thing would be to buffer the writes the same way it buffers
the reads, which your patch removes.  If you are really interested in
reducing the number of system calls, you want to use a larger buffer that
what libc does.  I think when I last looked at this libc tended to use
only half of BUFSIZ.

Why write more code?

On Linux, glibc looks up the preferred block size of the file/filesystem (via stat.st_blksize) and uses that if available, rather than BUFSIZ. See libio/filedoalloc.c for the code. For read/write files, this is reasonable behavior. So usually stdio ends up with 4k buffers.

For stuff we're just reading, a larger buffer size might be appropriate. Fortunately, stdio allows tuning of this behavior. After call to fdopen in files.c:open_file(), add:

  setvbuf(f, my_buffer, _IOFBF, sizeof(my_buffer));

Passing in an 8k buffer restores the previous system call count on Linux, and this can be set arbitrarily high there. However, some systems do have limits on setvbuf size (win32's is 32k) and on others, some amount of accounting data gets skimmed off the top. My feeling is that monkeying with this is just not worth it. My quick survey of non-buffer-gap editors shows most of them making 4k reads on Linux. Hell, /bin/cat uses 4k blocks.

If you *really* wanna cut system calls above all else, just mmap() the damn file. I forget if any libc implementations out there implement readonly streams that way, but it's always a possibility.

But libc usually has reasonable defaults for this kind of thing. I guess my point isn't so much that arbitrarily cutting the number of system calls is an unqualified good; it's that there are already mechanisms that other people are maintaining that will solve the problem well. If files.c used them, it would have done decimal orders of magnitude fewer write() calls.

   Since reading files is done much more that writing files, and writing
to devices it typically much slower than reading, no one probably noticed it being slow and no one bothered to change it. Only when reading a file are you actively waiting to do something. When writing a file, typically
your task is done and an extra second doesn't usually bother people.

I agree that you should optimize for common tasks and for those that are sensitive to the user. But stdio does reasonably well at these kinds of IO tasks, it's been written already, it's maintained for you, and because so many apps use stdio its performance will be scrutinized by many people. Getting rid of code in your apps is a good thing.

Jay




reply via email to

[Prev in Thread] Current Thread [Next in Thread]