RE: Looking for help improving performance (advice or potential contract position)

[Rakesh Sharma]

>From your description what seems to be ailing your build systemis the 
>architecture of your makefile itself. 
Without any clues as to what ails your build (since you didnt provide the 
makefile), you need to make your makefile non-recursive in order to reap the 
speed dividends. The changes that you talk about (simply-expanded variables, no 
shell commands, etc.) go only so far in gaining speed.
For more help you can look up the Miller paper on non-recursive makefiles which 
explains the reasons for slow/inaccurate builds & also how to overcome them.
HTH
-Rakesh

> Date: Wed, 1 Oct 2014 13:44:02 -0400
> Subject: Looking for help improving performance (advice or potential contract 
> position)
> From: address@hidden
> To: address@hidden
> 
> Greetings all,
> We've been making an effort to optimize our builds and we seem to be
> hitting the limit of what we can accomplish with makefile cleanup.  I've
> turned around some significant improvements by using techniques like
> simplifying makefiles where possible, converting recursively-expanded
> variables to simple variables, eliminating shell commands, etc, but we're
> still seeing slow performance from Make itself.
> 
> Our build is extremely parallelizable and we run with -j (unlimited) to run
> as many things as possible in parallel.  We use a home-grown job
> distribution tool to send individual tasks to clients on other machines for
> execution, so there's very little load on the machine running Make itself.
> What we see is that if we have more than 30 or so clients connected to the
> build Make is often unable to generate tasks fast enough to keep them busy.
> 
> We brought in Electric Cloud to do an evaluation of their make-compatible
> accelerator tool and found that it was able to instantly saturate as many
> workers as we could throw at it and keep them busy for the duration of the
> build, cutting the overall time nearly in half.  Unfortunately, their
> approach introduces a number of changes that would require significant
> reworking of our build system, and more importantly they want quite a lot
> of money for it (enough to hire additional senior developers for our team).
> 
> Our management doesn't want us taking time to learn the internals of make
> and see if we can improve it ourselves, but they are willing to throw money
> at the problem, either by offering some sort of donation/bounty for
> performance improvements or directly hiring an expert on a contract basis;
> any improvements achieved would of course be submitted back to the
> community as patches.
> 
> Is there an established place to post this sort of offer? fossfactory.org
> seems like what I'm looking for, but it doesn't look like it's terribly
> active.
> _______________________________________________
> Help-make mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/help-make