Re: csv with multiline records

[Ryan Brothers]

Ole,

On Wed, Dec 14, 2016 at 2:36 PM, Ole Tange <ole@tange.dk> wrote:
> But if you can somehow replace the record separator, then you can use 
> --recend.
>
> Given your input this might work:
>
>     parallel --pipe --recend '"\n"
>
> assuming a good part of the records have a last column with newlines.

Thank you for your help.  I can't assume the last column will always
have newlines, but your suggestion with --recend gave me an idea to do
something like:

cat file.csv | php reformat.php | parallel --pipe --recend '@@@'
--remove-rec-sep wc

reformat.php is a PHP script that reads the csv and writes it out to
stdout with @@@ in-between each record.

That seems to work great except I don't believe I can use --pipepart
with this method because the csv with @@@ is generated on the fly.  I
would have to save the reformatted csv file to disk.  Do you have any
thoughts to get around that?  If not, generating a new csv file in
this format would also be ok for my use case.

Thanks again.

Ryan