Re: How do parallel builds scale?

[Ralf Wildenhues]

* Ludovic Courtès wrote on Mon, Mar 07, 2011 at 06:16:17PM CET:
> Ralf Wildenhues writes:
> > * Ludovic Courtès wrote on Fri, Mar 04, 2011 at 06:59:45PM CET:
> >> Ralf Wildenhues writes:
> >> > * Ludovic Courtès wrote on Thu, Mar 03, 2011 at 04:42:52PM CET:
> >> >>   http://hubble.gforge.inria.fr/parallel-builds.html
> >> >
> >> > Thank you!  Would you be so kind and also describe what we see in the
> >> > graphs?  I'm sorry but I fail to understand what they are showing, what
> >> > the axes really mean, and how to interpret the results.
> >> 
> >> Y is the number of packages with a speedup <= X.  Does it help?
> >
> > Well, it helps in that it allows me to understand the graphs, but it
> > doesn't allow me to interpret them.  How about a histogram with x the
> > speedup (in some number of intervals) and y the number of packages
> > exhibiting that speedup?  That would IMVHO be easier to read.
> 
> I’ve added that.

Thank you, those are much more informative to me.

> FWIW I think the cumulative plots make sense when trying to answer the
> question “how many packages have a speedup <= X”.

Well, if I'd ask a cumulative question I'd probably rather ask how many
packages have a speedup >= X.  Small but significant difference.

> >> > I am fairly surprised GCC build times scaled so little.  IIRC I've seen
> >> > way higher numbers.  Is you I/O hardware adequate?
> >> 
> >> I think so.  :-)

Can you please make your hardware specs available?  I've had two 16-way
systems, and one could barely get more than a speedup of 2, while the
other cucked away nicely at 10 or so.

> > OK, so I guess I'd like some details as to how you build it.  Is
> > bootstrapping enabled (three-stage build),
> 
> Yes—that’s the default according to the manual (info "(gccinstall)
> Configuration").
> 
> > which languages do you build?
> 
>  - C and C++ for ‘gcc-4.5’;
> 
>  - Fortran only for ‘gfortran-4.5’, though I suspect
>    --enable-langauges=fortran implies --enable-languages=c.
> 
> Unfortunately there’s no data for GNAT and GCJ.

It would be awesome to have a graph of the load (y axis) over the time
(x axis) to see what's going on, for one of these builds.

> >> > Did you use only -j or also -l for the per-package times (I would
> >> > recommend to not use -l).
> >> 
> >> I actually used ‘-jX -lX’.  What makes you think -l shouldn’t be used?
> >
> > Typically, -lX leads to waves in the load, due to the latency between
> > measure and action, and of course the retardation from the measure
> > interval.  There are long periods in which processes are already done
> > but the load is still listed as too high.
> 
> Right, I see.
> 
> I don’t think it hindered scalability though, since as the measurements
> show that few packages scale beyond 2, even with ‘-j32 -l32’.

I don't believe this argument.  I think -l is fundamentally flawed, and
I don't see where the slowdown it causes should be bounded from below,
except for the number of total compiles (which is higher than 32 in the
case of GCC).

> > Then, you still have the waves as above.  It might be simpler to just
> > use -j3 for each of them.
> >
> > Do you build GCC and Glibc in separate virtual containers?
> 
> If you mean Linux containers, no.

So there's a GSoC project proposal for GNU make, below.

Paul, are you reading this?

> > If not, we could think of providing a high-level job server that just
> > serves out job cookies to the makes of both projects.  It might require
> > a bit of adjustment to GNU make to do this in all sorts of interesting
> > distro build setups.  But it would allow for (more) effective load
> > usage.
> 
> Yeah.

Thanks,
Ralf