bug-parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

GNU Parallel Bug Reports java flag being interpreted by bash, and 2+ ext


From: Cameron Prybol
Subject: GNU Parallel Bug Reports java flag being interpreted by bash, and 2+ extension chomping
Date: Fri, 21 Aug 2015 10:55:19 -0700

Hi,

```
$ parallel --version
GNU parallel 20150722
Copyright (C) 2007,2008,2009,2010,2011,2012,2013,2014,2015 Ole Tange
and Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
GNU parallel comes with no warranty.

Web site: http://www.gnu.org/software/parallel

When using programs that use GNU Parallel to process data for publication
please cite as described in 'parallel --bibtex'.
```

I'm having an issue with parallel interpreting a java flag as a file redirect.

running my script

```
$ ./s13.variant_calibration.sh
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
/bin/bash: 2.0: No such file or directory
```

is breaking because of the --filterExpression "QD < 2.0" being interpreted as a < file input in this command

```
parallel java -jar $GATK -T VariantFiltration -R "$REF_DIR/human_g1k_v37.fasta" -V {} -window 35 -cluster 3 --filterName QD --filterExpression "QD < 2.0" --filterName FS --filterExpression "FS > 30.0" -o "$VCF_OUT/"'{= s:.*/::; s:\.[^.]+$::; s:\.[^.]+$::; =}'.filtered.vcf ::: "$(ls "$VCF_IN"/*.vcf)"
```

I know this because if I change the 
```
--filterExpression "QD < {different value}"
```
to anything besides 2, the error will return
```
/bin/bash: {different value}: No such file or directory
```
(additionally, I have tried several combinations of quotations, \escapes, and changing the flag names of the java program to no avail)



If I drop parallel and just run the java code with hardcoded files (or shell variables), the program runs without error

```
java -jar $GATK -T VariantFiltration -R "$REF_DIR/human_g1k_v37.fasta" -V /users/cjprybol/Parkinsons/VCF/SRR922052.realligned.vcf -window 35 -cluster 3 --filterName QD --filterExpression "QD < 2.0" --filterName FS --filterExpression "FS > 30.0" -o "$VCF_OUT/"SRR922052.filtered.vcf
```

~~~~~

an additional problem I have is with the tandem extension dropping

for example
```
parallel java -jar \
    $GATK \
    -T HaplotypeCaller \
    -R "$REF_DIR/human_g1k_v37.fasta" \
    -I {} \
    -dontUseSoftClippedBases \
    --dbsnp "$REF_DIR/dbsnp_138.b37.vcf" \
    -stand_call_conf 20 \
    -stand_emit_conf 20 \
    -o "$VCF_OUT/"'{= s:.*/::; s:\.[^.]+$::; s:\.[^.]+$::; =}'.vcf \
    ::: "$(ls "$BAM_DIR"/*.score_recalibrated.bam)"
```

runs fine with the perl regular expressions in tandem, but anytime I want to drop two or more extensions i.e. {/..}, {/...} it fails

```
parallel java -jar \
    $GATK \
    -T HaplotypeCaller \
    -R "$REF_DIR/human_g1k_v37.fasta" \
    -I {} \
    -dontUseSoftClippedBases \
    --dbsnp "$REF_DIR/dbsnp_138.b37.vcf" \
    -stand_call_conf 20 \
    -stand_emit_conf 20 \
    -o "$VCF_OUT/"'{/..}'.vcf \
    ::: "$(ls "$BAM_DIR"/*.score_recalibrated.bam)"
```

outputs

```
##### ERROR MESSAGE: Couldn't write file /users/cjprybol/Parkinsons/VCF/{/..}.vcf because either the containing directory doesn't exist or it isn't writable
```

however single extension works fine {/.}

parallel has worked fine with 8 other calls I make to the same jar (and another, called Picard), none of which use a < or >, so it seems specific to that IO redirect symbol.

Thanks for making this great tool available!

Best,
Cameron

reply via email to

[Prev in Thread] Current Thread [Next in Thread]