Re: incorrect output when using cat | parallel

[Ole Tange]

On Tue, Apr 29, 2014 at 2:44 PM, james pettengill <fixtgear@gmail.com> wrote:

> I am using GNU parallel 20140222 and have noticed a problem when piping to
> parallel from cat.
>
> It appears that the output from the commands within the file I am cat'ing
> from is incorrect where a few lines are missing or lines are written out in
> an incorrect format that causes downstream processing to fail.
>
> An example of the commands within the file (flashParallel.txt) I am cat'ing
> is:
>
> flash kc148_S1_L001_R1_001.fastq kc148_S1_L001_R2_001.fastq  -o  kc148
> flash kc1_S6_L001_R1_001.fastq kc1_S6_L001_R2_001.fastq  -o  kc1
>
> flash is a program in my path used to merge DNA sequence data that requires
> two files and -o specifies the output prefix.
>
> I have been running parallel with the following command:
>
> cat flashParallel.txt | parallel
>
> The files are rather large so not sure how best to provide data to reproduce
> the problem.

First see if you can reproduce it using something else that flash. Can
you reproduce it using 'cat bigfile' or 'echo foo'?

Can you reproduce the error on other systems?

>From your error description it sound like you may be using -u.

> Any suggestions/solutions would be greatly appreciated and thanks for
> developing the software.

The section REPORTING BUGS in the man page will also give you some suggestions.


/Ole