parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

parallel interprets rsync log file format options as files to transfer t


From: Jason Paul Joines
Subject: parallel interprets rsync log file format options as files to transfer that do not exist
Date: Thu, 11 May 2017 11:39:48 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0

GNU Parallel Users,


        I am using GNU Parallel version 20170422 on Scientific Linux
release 6.6.  When using rsync log file format options with parallel,
the logs are filled with errors because each log file format option gets
interpreted as a file to transfer that does not exist.

paralleltest/sdir$ parallel --version
GNU parallel 20170422
Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015,2016,2017
Ole Tange and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.

Web site: http://www.gnu.org/software/parallel

When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --citation'.
paralleltest/sdir$


paralleltest/sdir$ ls
tfile1  tfile2  tfile3  tfile4  tfile5  tfile6  tfile7  tfile8
paralleltest/sdir$


        This works fine:

paralleltest/sdir$ ls -1 | parallel rsync -aHAX --delete --log-file
../testlog {} ../ddir/
paralleltest/sdir$


        This also works fine, but with more detailed logging:

paralleltest/sdir$ for J in *; do rsync -aHAX --delete --log-file
../testlog --log-file-format="%o %f %L %l %b %i %t" "$J" ../ddir/; done
paralleltest/sdir$


        This breaks because the rsync log file format options are
interpreted as files to transfer when passed through parallel:

paralleltest/sdir$ ls -1 | parallel rsync -aHAX --delete --log-file
../testlog --log-file-format="%o %f %L %l %b %i %t" {} ../ddir/
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%f" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%L" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%l" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%b" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%i" failed: No
such file or directory (2)
rsync: link_stat "/local_scratch/joines/paralleltest/sdir/%t" failed: No
such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors)
(code 23) at main.c(1039) [sender=3.0.6]

        The seven lines above are repeated for each of the eight files
in the transfer list.


        I get the same behavior if I enclose the options in single
quotes instead of double quotes, put the log-file-format option before
the log-file option, or pass the the transfer list into a BASH script
that contains:  parallel -j 8 rsync -aHAX --delete --log-file-format='%o
%f %L %l %b %i %t' --log-file ../testlog {} ../ddir/ ::: $@


        Any ideas how to get parallel and rsync log-file-format options
to cooperate?


Thanks,

Jason
===========



reply via email to

[Prev in Thread] Current Thread [Next in Thread]