[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#32228: sed -i does not buffer, causing excessive writes
From: |
Vidar Holen |
Subject: |
bug#32228: sed -i does not buffer, causing excessive writes |
Date: |
Fri, 20 Jul 2018 13:23:35 -0700 |
Hi,
I'm using noticing that `sed -i` does not do any kind of output
buffering. While behavior is correct, this causes an excessive number of
small writes:
$ strace -e write ./sed -i 's/foo/bar/g' file.txt
write(4, " GNU GENERAL "..., 47) = 47
write(4, " Version 3"..., 47) = 47
write(4, "\n", 1) = 1
Compare this to redirection:
$ strace -e write ./sed 's/foo/bar/g' file.txt > tmpfile
write(1, " GNU GENERAL "..., 4096) = 4096
write(1, "om or adapt all or part of the w"..., 4096) = 4096
write(1, ".\n\n You may make, run and propa"..., 4096) = 4096
On my system, this makes `sed -i` take 7x longer than redirection+mv:
3.7s vs 0.5s for 100MB, for 675 vs 10 writes on the same input.
I'm seeing this on Debian with the latest sed commit (c52a676) and libc
2.27-2. The root cause appears to be `ck_mkstemp` using `fdopen`, which
unlike `fopen` does not buffer by default.
Enabling buffering should be simple and effective, resulting in a nice
speedup.
Regards,
Vidar Holen
- bug#32228: sed -i does not buffer, causing excessive writes,
Vidar Holen <=