Re: GNU Parallel on Amazon Linux

[Osborne, Darryl]

Darryl S. Osborne
Storage Specialist | AWS Solutions Architecture
o: 469.283.5878
c: 206.817.2906

On 12/15/17, 5:00 PM, "ole.tange@gmail.com on behalf of Ole Tange" 
<ole.tange@gmail.com on behalf of ole@tange.dk> wrote:

    On Fri, Dec 15, 2017 at 3:10 PM, Osborne, Darryl <darrylo@amazon.com> wrote:
    
    > First, thanks for developing and maintaining GNU Parallel. It’s an 
essential
    > tool and I recommend it in just about every customer call, talk, or
    > presentation I give.
    
    Happy to hear that.
    
    I believe I linked to one of your presentations in the release notes
    for 20170122:
    
    * AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202) 
slide 45
      
http://www.slideshare.net/AmazonWebServices/aws-reinvent-2016-deep-dive-on-amazon-elastic-file-system-stg202
      and https://youtu.be/PlTuJx4VnGw?t=30m16s
    
        >> Great! Thanks for including that.

    > Second, I don’t know if you’re aware but parallel is now included in the
    > Amazon repository so AWS customers running Amazon Linux can easily install
    > it with a simple “sudo yum install parallel –y” command.
    
    I was not aware of that.

       >> This may be a good to mention during the next issues of Parallel 
Digest
    
    > One of my sample tutorial results sums it up well. In this example
    > (attached) when I combined GNU parallel and standard Linux copy commands
    > like cp and cpio, I achieved a throughput increase of 690% and transferred
    > files 7.9 times faster when compared to using the copy commands alone.
    
    Good to see. Do you know what the theoretical max is?

        >>The maximum throughput you can drive an EFS file system per EC2 
instance is 250 MB/s. 3 GB/s+ throughput is achieved scaling across multiple 
instances (like a data center).
    
    > Have you had a chance to play around with Amazon EFS? I’d be curious to 
hear
    > your feedback.
    
    I have yet to have a job where I needed EFS.
    
    
    /Ole