[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parallel a script
From: |
Ole Tange |
Subject: |
Re: Parallel a script |
Date: |
Sat, 25 Jun 2011 20:44:14 +0200 |
On Sat, Jun 25, 2011 at 8:34 PM, yacob sen <yacob_123@yahoo.com> wrote:
>
> Dear Ole,
> Thank you so much. I am learning more and more from your responses.
Most of this is actually covered in the intro videos. Did you watch those?
http://www.youtube.com/watch?v=OpaiGYxkSuQ
http://www.youtube.com/watch?v=P40akGWJ_gY
http://www.youtube.com/watch?v=1ntxT-47VPA
http://www.youtube.com/watch?v=fOX1EyHkQwc
> I am not sure why "-N" is not correct to use.
It was never designed for the situation you are in. If it works, it is
shear luck.
> The way I want the program should take the entries from files "name_list" and
> "date_list" is as follows :
> say "name_list" has entries in raw form
> a00101.11
> a00102.11
> while "date_list" has entries in raw form
> 2011-01-01
> 2011-01-02
>
> If I use
> parallel prog -i {1} -type {2} :::: name_list date_list
> the program will run 4 times like the for loop:
> parallel prog -i a00101.11 -type 2011-01-01
> parallel prog -i a00101.11 -type 2011-01-02
> parallel prog -i a00102.11 -type 2011-01-01
> parallel prog -i a00102.11 -type 2011-01-02
>
> But I want the program to run only twice,
> parallel prog -i a00101.11 -type 2011-01-01
> parallel prog -i a00102.11 -type 2011-01-02
Yep.
> So the -N option some how helps me to pick the above options. I will be happy
> if there is another way to get the same result, i,e,
> parallel prog -i a00101.11 -type 2011-01-01
> parallel prog -i a00102.11 -type 2011-01-02
The supported way to do that is:
parallel --xapply prog -i {1} -type {2} :::: name_list date_list
> The "prog" program does output to the same file (several files for that
> matter) . But when I run it as you have suggested
> parallel 'mkdir {1}_{2}.dir; cd {1}_{2}.dir; prog -i ../{1} -type {2};
> mv file.out ../{1}_{2}.out; cd ..; rmdir {1}_{2}.dir'
> I got an error:
> parallel: Input is tty. Press CTRL-D to exit.
>
> Any idea why ?
You need to append the argument files. So the full line will be:
parallel --xapply 'mkdir {1}_{2}.dir; cd {1}_{2}.dir; prog -i ../{1}
-type {2}; mv file.out ../{1}_{2}.out; cd ..; rmdir {1}_{2}.dir' ::::
name_list date_list
Also you need to change file.out to the output file that prog creates.
If you want more help, sign up for the mailing list
http://lists.gnu.org/mailman/listinfo/parallel and explain in more
details what a single job looks like.
/Ole