[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: sort: Parallel merging
From: |
Shaun Jackman |
Subject: |
Re: sort: Parallel merging |
Date: |
Wed, 17 Feb 2010 14:52:01 -0800 |
On Wed, 2010-02-17 at 13:27 -0800, Chen Guo wrote:
> Hi Shaun,
> Last year someone named Glen Lenker did something like that, and I
> think the patch was rejected because the maintainers didn't see enough
> speed up as the number of CPUs got higher.
Thanks. I'll definitely look into this.
> As for buffer size, I highly doubt using 8 mb, even if we're magically
> guaranteed to get 100% of the cpu cache, would work better than a larger
> buffer.
>
> The main reason would be for larger files, you'd have to repeatedly write
> temporary files out to disk, then merge those temporary files. Whatever
> time you save talking to cache is more than lost to the extra time talking
> to disk.
What if the temporary files were stored in RAM (i.e. tmpfs) rather than
on magnetic disk?
Cheers,
Shaun
- sort: Parallel merging, Shaun Jackman, 2010/02/17
- Re: sort: Parallel merging, Chen Guo, 2010/02/17
- Re: sort: Parallel merging,
Shaun Jackman <=
- Re: sort: Parallel merging, Chen Guo, 2010/02/17
- Message not available
- Re: sort: Parallel merging, Shaun Jackman, 2010/02/17
- Message not available
- Re: sort: Parallel merging, Shaun Jackman, 2010/02/17
- Re: sort: Parallel merging, Chen Guo, 2010/02/17
- Re: sort: Parallel merging, Chen Guo, 2010/02/17