[coreutils] Re: Bug#609049: du: process_file: Assertion `level == prev_level - 1' failed

[Jim Meyering]

Jonathan Nieder wrote:
> Package: coreutils
> Version: 8.5-1
>
> Happy new year.  Today I ran
>
>  $ du -sk ~/*
>
> to see where all the space is going.  The traversal almost certainly
> was in ~/src, and being bored, I suspended it to examine that
> directory:
>
>  ^Z
>  [1]+  Stopped                 du -sk ~/*
>
> In another terminal I ran "cd ~/src && du -sk *".  ~/src/netbsd was
> very big, so I moved it (using chmod +w first) to ~/.cache on the same
> filesystem for future reference.  And now time to resume the toplevel
> search:
>
>  $ fg
>  du -sk ~/*
>  du: fts_read failed: No such file or directory
>  du: du.c:583: process_file: Assertion `level == prev_level - 1' failed.
>  Aborted (core dumped)
>
> Lacking debugging symbols, the core dump does not seem so useful, but
> I still have it.  libc6 is 2.11.2-7, filesystem is ext4, and the
> kernel is 2.6.37-rc7-686.  If there is additional information I could
> provide, just ask.

Thank you for the fine bug report.
Here's a fix:

While I do have a reproducer that will become a test suite addition
(coming soon), it relies on python (a first) and the python-inotify
package, so I'll have to be careful to skip the test when those
prerequisites aren't installed.  Also, there's an inherent race condition,
so I'll have to find the right compromise between absolute test-robustness
(too expensive in time and inodes) and reasonableness.

>From 92bbec0f697575a8bcbbe3fe0cb54b468c1022a4 Mon Sep 17 00:00:00 2001
From: Jim Meyering <address@hidden>
Date: Sat, 8 Jan 2011 17:44:55 +0100
Subject: [PATCH] du: don't abort when a subdir is renamed out from under us

* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Also improve a diagnostic.
Reported by Johathan Nieder in http://bugs.debian.org/609049
---
 NEWS     |    7 +++++++
 src/du.c |   15 +++++++++++----
 2 files changed, 18 insertions(+), 4 deletions(-)

diff --git a/NEWS b/NEWS
index 2a71ca6..793a8e6 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,13 @@ GNU coreutils NEWS                                    -*- 
outline -*-

 * Noteworthy changes in release ?.? (????-??-??) [?]

+** Bug fixes
+
+  du could abort with a failed assertion when part of the hierarchy being
+  traversed was moved to a higher level in the directory tree, and when
+  there was at least one more command line directory argument following
+  the one containing the moved sub-tree.
+

 * Noteworthy changes in release 8.9 (2011-01-04) [stable]

diff --git a/src/du.c b/src/du.c
index 77deb0c..4dc5456 100644
--- a/src/du.c
+++ b/src/du.c
@@ -63,8 +63,11 @@ extern bool fts_debug;
 /* A set of dev/ino pairs.  */
 static struct di_set *di_set;

-/* Define a class for collecting directory information. */
+/* Keep track of the preceding "level" (depth in hierarchy)
+   from one call of process_file to the next.  */
+static size_t prev_level;

+/* Define a class for collecting directory information. */
 struct duinfo
 {
   /* Size of files in directory.  */
@@ -399,7 +402,6 @@ process_file (FTS *fts, FTSENT *ent)
   struct duinfo dui;
   struct duinfo dui_to_print;
   size_t level;
-  static size_t prev_level;
   static size_t n_alloc;
   /* First element of the structure contains:
      The sum of the st_size values of all entries in the single directory
@@ -582,10 +584,15 @@ du_files (char **files, int bit_flags)
             {
               if (errno != 0)
                 {
-                  /* FIXME: try to give a better message  */
-                  error (0, errno, _("fts_read failed"));
+                  error (0, errno, _("%s: fts_read failed"),
+                         quotearg_colon (fts->fts_path));
                   ok = false;
                 }
+
+              /* When exiting this loop early, be careful to reset the
+                 global, prev_level, used in process_file.  Otherwise, its
+                 (level == prev_level - 1) assertion could fail.  */
+              prev_level = 0;
               break;
             }
           FTS_CROSS_CHECK (fts);
--
1.7.3.5