bug-parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

GNU Parallel Bug Reports parallel --pipe no longer honours --max-lines s


From: Shawn Halpenny
Subject: GNU Parallel Bug Reports parallel --pipe no longer honours --max-lines setting
Date: Mon, 30 Sep 2013 16:23:18 -0700

I recently upgraded from an older version (2012-10-22) of parallel to 2013-09-22 and discovered that --max-lines no longer has any effect if --pipe is used.  I downloaded and untarred each parallel release from 2012-01 to 2013-09 and ran this:

for i in parallel-*
do
(
cd $i
echo $i
time sh -c 'seq 1 100000 | ./src/parallel --tmpdir . --pipe --max-lines 1000 -- cat \| wc -l | wc -l'
echo
)

i.e. 100K lines of input, pass 1000 lines at a time to "cat | wc -l", and count how many lines it outputs.  I expected each loop to show that 100 lines were output, since that's how many times "cat | wc -l" should be invoked on that input data set.  Each release I tried up to and including 20130122 was fine, although some reported more than 100 lines, which was odd.  Every release including and after 20130222 all showed only one line (i.e. all input was being consumed by one invocation of "cat | wc -l" instead of spread between them all.

Test output:
parallel-20120122
102
3.038 real  3.480 user  5.816 system   305%  sh -c 

parallel-20120222
100
3.055 real  3.916 user  5.212 system   298%  sh -c 

parallel-20120322
101
2.850 real  3.656 user  5.420 system   318%  sh -c 

parallel-20120422
100
2.879 real  3.608 user  5.380 system   312%  sh -c 

parallel-20120522
103
2.949 real  3.840 user  5.476 system   315%  sh -c 

parallel-20120622
100
18.735 real  19.845 user  5.800 system   136%  sh -c 

parallel-20120722
100
18.614 real  19.849 user  5.936 system   138%  sh -c 

parallel-20120822
100
19.890 real  20.801 user  5.812 system   133%  sh -c 

parallel-20121022
100
18.742 real  20.193 user  5.724 system   138%  sh -c 

parallel-20121122
100
18.808 real  19.933 user  5.736 system   136%  sh -c 

parallel-20121222
102
3.144 real  3.976 user  5.852 system   312%  sh -c 

parallel-20130122
100
3.642 real  4.136 user  6.620 system   295%  sh -c 

parallel-20130222
1
0.453 real  0.284 user  0.340 system   137%  sh -c 

parallel-20130422
1
0.454 real  0.320 user  0.300 system   136%  sh -c 

parallel-20130522
1
0.459 real  0.348 user  0.300 system   141%  sh -c 

parallel-20130622
1
0.461 real  0.352 user  0.288 system   138%  sh -c 

parallel-20130722
1
0.443 real  0.264 user  0.344 system   137%  sh -c 

parallel-20130822
1
0.436 real  0.328 user  0.316 system   147%  sh -c 

parallel-20130922
1
0.453 real  0.296 user  0.340 system   140%  sh -c 

reply via email to

[Prev in Thread] Current Thread [Next in Thread]