Re: [Bug-tar] add --ignore-missing to ignore missing input files

[Nathan Stratton Treadway]

On Tue, Apr 24, 2012 at 22:10:54 -0700, Paul Eggert wrote:
> The general idea here, as I understand it, is ignoring problems that
> can occur merely because tar is backing up a live file system, while
> not ignoring I/O errors that happen for other reasons.  Tar already
> attempts to do that, by means of its file_removed_diag function, and

Ah!  That sheds some light on things....

> any fix along these lines should be using that function rather than
> causing all 'stat' failures with errno==ENOENT to not be an error
> (which is clearly the wrong thing to do).
> 
> In other words, there shouldn't be a need for a new option at all;
> the functionality is already there, it's just that it's apparently
> buggy in some cases (cases that haven't been described well).


Okay, as far as describing such a case, here's a simple example:

==========================================================
$ tar --version | head -1
tar (GNU tar) 1.26
$ lsb_release -d
Description:    Ubuntu 12.04 LTS

$ dd if=/dev/zero of=big.test
  [hit ^C after about 5 seconds... the point is just to get a file 
  that takes longer than a second to process on the test machine]
293754+0 records in
293754+0 records out
150402048 bytes (150 MB) copied, 6.84027 s, 22.0 MB/s

$ echo "blah" > file2.test
  [just to have a second file that makes it into the archive]


$ touch file1.test; (sleep 1; rm file1.test)&  tar cf t1.tar *.test; echo 
"Exit: $?"
[1] 2074
tar: file1.test: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
[1]+  Done                    ( sleep 1; rm file1.test )
Exit: 2
$ tar tf t1.tar
big.test
file2.test


$ touch file1.test; (sleep 1; rm file1.test)&  tar cf t2.tar 
--ignore-failed-read *.test; echo "Exit: $?"
[1] 2080
tar: file1.test: Warning: Cannot stat: No such file or directory
[1]+  Done                    ( sleep 1; rm file1.test )
Exit: 0
$ tar tf t2.tar
big.test
file2.test

$ diff -s t?.tar
Files t1.tar and t2.tar are identical

$ chmod u-rwx file2.test
$ tar cf t3.tar *.test; echo "Exit: $?"
tar: file2.test: Cannot open: Permission denied
tar: Exiting with failure status due to previous errors
Exit: 2
$ tar cf t4.tar --ignore-failed-read *.test; echo "Exit: $?"
tar: file2.test: Warning: Cannot open: Permission denied
Exit: 0
$ diff -s t3.tar t4.tar
Files t3.tar and t4.tar are identical
==========================================================

So, the point is that in situations where you aren't surprised that some
of your files disappear as tar is running, you might want to avoid
having an error exit code in that case -- while still ensuring that you
get an error exit status when a "real" error occurs....  (At the same
time, the actual contents of the tar file generated aren't changed by
the options under discussion.)


With that example in hand (i.e. the "t1.tar" line above), I followed a
tar invocation using gdb and confirmed that misc.c:file_removed_diag()
was indeed called when file1.test wasn't found -- but at that point
top_level was true, so the WARNOPT code was not executed, and instead
diagfn() (which was set to "stat_diag") was called (as we'd expect,
given that's the function that Stefan's patch was changing).

(I assume that top_level would also be true for files passed in using
-T, so the same explanation would apply to Stefan's find/tar combo,
too.)


So... what about leaving stat_diag unchanged, but instead adding a way
to control what file_removed_diag() does when ENOENT occurs on a
top-level file?

I'm not sure off hand if that should be done using an option along the
lines of --ignore-missing, a new --warning= keyword, or both....

Here's what the "both" approach might look like:

  if (errno == ENOENT)
    {
      if (!top_level)
        {
          WARNOPT (WARN_FILE_REMOVED,
                   (0, 0, _("%s: File removed before we read it"),
                    quotearg_colon (name)));
          set_exit_status (TAREXIT_DIFFERS);
        }
      else 
        if (ignore_missing)
          {
            WARNOPT (WARN_FILE_MISSING,
                     (0, 0, _("%s: Listed file not found"),
                      quotearg_colon (name)));
          }
        else
          diagfn (name);
    }
  else
    diagfn (name);


Note that in this case "--ignore-missing" is really short for "ignore
missing explicitely-listed files"; perhaps there is a better name for
the option....

Anyway, the theory is that the --ignore-missing option would specify
that these missing files would be considered a warning instead of an
error... and in the treat-as-warning case one could then control whether
the warning message text is actually printed using the
--warning=no-file-missing option.


(Also note that to keep the behavior of Stefan's patch, the above code
specifically does _not_ set the exit status to DIFFERS in the
--ignore-missing situation.  

For what it's worth, that matches the behavor of the
--ignore-failed-read option in the "File shrank" case [in
create.c:dump_regular_file()] -- though it looks like the "file changed
as we read it" case [in dump_file0()] sets the exit status to DIFFERS
without regard to the --ignore-failed-read setting....  )




                                                Nathan



----------------------------------------------------------------------------
Nathan Stratton Treadway  -  address@hidden  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239