Re: [PATCH] sort: Add --threads option, which parallelizes internal sort.

[Ralf Wildenhues]

* Pádraig Brady wrote on Mon, Apr 20, 2009 at 01:57:59AM CEST:
> Ralf Wildenhues wrote:
> > 
> > This comparison isn't entirely fair, as the splicing was done as a
> > precomputation.  However, the difference is so pronounced that even
> > taking the splicing into account, the non-thread version would be
> > quite a bit faster.  So to me, it would seem that some approach going
> > in that direction could be beneficial.
> 
> Absolutely. That's one of the things Glen was going to work on this summer.
> I.E. enhance split and perhaps have a helper script so as to achieve the
> above more easily.

Great!

> It would be useful to know where you stored the 8 sets of split data.
> Were they files on different spindles or cached in RAM, or on SSD etc.

/dev/shm, so tmpfs, as before.

> > Other than that, have you thought of implementing something like
> > described in <http://www.it.uom.gr/teaching/dbpp/text/node127.html>?
> 
> That link is timing out for me?

Try the google cache on it.  I don't know if this link needs a cookie,
but it's one of the first few entries when searching for "parallel merge
sort":
<http://74.125.77.132/search?q=cache:ovtsrdT72b4J:www.it.uom.gr/teaching/dbpp/text/node127.html+parallel+merge+sort&cd=5&ct=clnk>

Cheers,
Ralf