[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Limiting memory used by parallel?
|
From: |
hubert depesz lubaczewski |
|
Subject: |
Re: Limiting memory used by parallel? |
|
Date: |
Thu, 25 Jan 2018 17:27:13 +0100 |
|
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On Thu, Jan 25, 2018 at 11:26:24AM -0500, Joe Sapp wrote:
> Hi depesz,
>
> On Thu, Jan 25, 2018 at 10:33 AM, hubert depesz lubaczewski
> <depesz@depesz.com> wrote:
> [snip]
> > But it looks that parallel itself is consumming HUGE amount of memory
> > - comparable with size of /some/directory itself.
> >
> > Server that I run it on has 64GB of ram, and the script gets killed
> > after ~ 3 minutes with "Out of memory!" error.
>
> Have you tried using --memfree? That will limit parallel from
> spawning new processes
No, I haven't. Thanks for the hint, will test it, and we'll see :)
Best regards,
depesz