Re: Looking for help improving performance (advice or potential contract position)

[Dan Kegel]

I know this isn't an answer to your question, but do you have
a simplified, sanitized test case yet that demonstrates the problem
and could be posted publicly?
- Dan



On Wed, Oct 1, 2014 at 10:44 AM, Phil Grabsky <address@hidden> wrote:
> Greetings all,
> We've been making an effort to optimize our builds and we seem to be
> hitting the limit of what we can accomplish with makefile cleanup.  I've
> turned around some significant improvements by using techniques like
> simplifying makefiles where possible, converting recursively-expanded
> variables to simple variables, eliminating shell commands, etc, but we're
> still seeing slow performance from Make itself.
>
> Our build is extremely parallelizable and we run with -j (unlimited) to run
> as many things as possible in parallel.  We use a home-grown job
> distribution tool to send individual tasks to clients on other machines for
> execution, so there's very little load on the machine running Make itself.
> What we see is that if we have more than 30 or so clients connected to the
> build Make is often unable to generate tasks fast enough to keep them busy.
>
> We brought in Electric Cloud to do an evaluation of their make-compatible
> accelerator tool and found that it was able to instantly saturate as many
> workers as we could throw at it and keep them busy for the duration of the
> build, cutting the overall time nearly in half.  Unfortunately, their
> approach introduces a number of changes that would require significant
> reworking of our build system, and more importantly they want quite a lot
> of money for it (enough to hire additional senior developers for our team).
>
> Our management doesn't want us taking time to learn the internals of make
> and see if we can improve it ourselves, but they are willing to throw money
> at the problem, either by offering some sort of donation/bounty for
> performance improvements or directly hiring an expert on a contract basis;
> any improvements achieved would of course be submitted back to the
> community as patches.
>
> Is there an established place to post this sort of offer? fossfactory.org
> seems like what I'm looking for, but it doesn't look like it's terribly
> active.
> _______________________________________________
> Help-make mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/help-make