[debbugs-tracker] bug#22768: closed (Crash safety)

[GNU bug Tracking System]

Your message dated Mon, 22 Feb 2016 23:28:07 -0800
with message-id <address@hidden>
and subject line Re: bug#22768: Crash safety
has caused the debbugs.gnu.org bug report #22768,
regarding Crash safety
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden)


-- 
22768: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22768
GNU Bug Tracking System
Contact address@hidden with problems

--- Begin Message --- Subject: Crash safety Date: Mon, 22 Feb 2016 03:57:47 -0500

Hi gzip developers,

Gzip version: 1.6

I am developing a tool to validate crash safety of application software. I have just found that the file deletion has a potential safety venerability: if only a prefix of I/O operations are flushed to disk, after reboot, the file-system would only contain a 0-byte file (the data is not reached to disk yet).

A paper FYI: http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f.r43.cf2.rackcdn.com/17780-osdi14-paper-pillai.pdf (Table 1 on Page 440). Data append can be (virtually) reordered with any operation at default ext3 and ext4 settings. I recommend to use fsync() to persist the .gz file before deletion.

— strace log —

 36 open("a", O_RDONLY|O_NOCTTY|O_NONBLOCK|O_LARGEFILE|O_NOFOLLOW) = 3

 37 fstat64(3, {st_mode=S_IFREG|0664, st_size=19730, ...}) = 0

 38 rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE TERM XCPU XFSZ], [], 8) = 0

 39 open("a.gz", O_WRONLY|O_CREAT|O_EXCL|O_LARGEFILE, 0600) = 4

 40 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0

 41 read(3, "10017649652034232324895361757801"..., 65536) = 19730

 42 read(3, "", 45806)                      = 0

 43 write(4, "\37\213\10\10\24\312\312V\0\3a\0-\334m\226\234\274\22\3\340\377Y\r    \30l\354\375o,z\324"..., 9954) = 9954

 44 close(3)                                = 0

 45 utimensat(4, NULL, {{1456130580, 76955623}, {1456130580, 128955620}}, 0) = 0

 46 fchown32(4, 1000, 1000)                 = 0

 47 fchmod(4, 0664)                         = 0

 48 close(4)                                = 0

 49 rt_sigprocmask(SIG_BLOCK, [HUP INT PIPE TERM XCPU XFSZ], [], 8) = 0

 50 unlink("a")                             = 0

Thank you for your attention!

Regards,

Yanyan Jiang 蒋炎岩

Institute of Computer Software,

Dept. of Computer Science, Nanjing University

--- End Message ---

--- Begin Message --- Subject: Re: bug#22768: Crash safety Date: Mon, 22 Feb 2016 23:28:07 -0800 User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 Thanks for reporting the problem. It's annoying that gzip must invoke fsync, as that's way overkill compared to the write-ordering that is needed and fsync will slow gzip down, but I don't see any safe and reasonably portable alternative so I installed the attached patch on Savannah, here:

http://git.savannah.gnu.org/cgit/gzip.git/commit/?id=22aac8f8a616a72dbbe0e4119db8ddda0f076c04

0001-fsync-output-file-before-closing.txt
Description: Text document

--- End Message ---