On Jul 17, 2011 8:09 AM, "Dan Kokron" <
daniel.kokron@nasa.gov> wrote:
> All,
>
> I have a bunch (~200) small (1K to 100K) binary files that I want to
> 'cat' into a larger file. I usually use "cat pe* > diag", but this
> takes considerable time on the Lustre file system we are using. I am
> exploring using GNU parallel for this task but have run into some
> difficulties. Basically the resulting diag file only contains one of
> the input files.
>
> I've tried the following variations.
>
> parallel "cat {} >diag_amsua_n18_03.2011041700" ::: pe*
> parallel cat {} ">"diag_amsua_n18_03.2011041700 ::: pe*
> ls pe* | parallel cat {} ">"diag_amsua_n18_03.2011041700
> ls pe* | parallel -j4 -k cat {} ">"diag_amsua_n18_03.2011041700
> ls pe* | parallel -k cat {} ">"diag_amsua_n18_03.2011041700
> parallel -j4 -k "cat {} >diag_amsua_n18_03.2011041700" ::: pe*
>
> Should this work? Any ideas what I'm doing wrong?
> Thanks
>
> parallel --version
> GNU parallel 20110622
>
> on
> SLESS11
>
> perl --version
> This is perl, v5.10.0 built for x86_64-linux-thread-multi
> --
> Dan Kokron
> Global Modeling and Assimilation Office
> NASA Goddard Space Flight Center
> Greenbelt, MD 20771
>
Daniel.S.Kokron@nasa.gov> Phone: (301) 614-5192
> Fax: (301) 614-5304
>
>