bug#6600: [PATCH] sort: add --threads option to parallelize internal sort.

[Pádraig Brady]

On 21/07/10 09:13, Chen Guo wrote:
> Hi all,
>     So the gcc compile farm just got a 24 core donation
> from AMD (2x12 at 1.5 GHz), so I couldn't help but run
> some results.
>     As a way of isolating sort time versus I/O time, I installed
> a timer output, such that sort prints to stderr the elapsed
> time during sort()'s call to sortlines().
>     There are other people using the machine, so the
> timings wont be the most accurate, but I figure you guys
> may still be interested. On 24 threads, the pure sort runs
> in about 15% of the single threaded time, while with I/O
> that figure's 17.5%.
> 
> $ for i in 1 2 4 8 12 16 24; do echo "T=$i: ";  /usr/bin/time -f "%e"
> sort --parallel=$i 1M > /dev/null; echo ""; done

Cool! I had intended to do this at the weekend on the
gcc niagra (32 processor) machine but my DSL died.
I also intended to plot the cumulative CPU used,
and also with using mutexes rather than spinlocks.

Reformatting your results as:

1 11.074463 11.38
2 5.672202 5.96
4 3.453562 3.71
8 2.351003 2.65
12 2.100730 2.39
16 1.821079 2.09
24 1.674106 1.99

And using this gnuplot script:

set term pngcairo font 'Sans,10' size 640,480
set output 'sort-amd-24way.png'
set title "Multicore sort time on 12x2 AMD"
set ylabel "time (seconds)"
set xlabel "processors"
set xtics (2,4,6,8,12,16,24,32)
set xrange [0:32]
plot "amd" using 1:2 with lines title "no I/O", \
     "amd" using 1:3 with lines title "with I/O"

Gives the attached plot.

cheers,
Pádraig.

sort-amd-24way.png
Description: PNG image