Patch #5108: Selective serialization in parallel builds

[Cesar Crusius]

Hi all,

I just posted a patch (#5108) that allows one to select targets that can't be 
built in parallel. Here's the description:

==============
This patch introduces a .SERIAL special target which causes all its 
dependencies to be built as if make was called with '-j1'. This solves the 
"ar" problem described in the section "Dangers When Using Archives" section 
of the manual.
 
 This is also very useful for linking libraries and executables, as those 
tasks are I/O bound and can lock the machine if done in parallel.
 
 I think SCO make has a similar thing using a .MUTEX special target. I don't 
care about either name, and would use .NOTPARALLEL but that can break 
existing makefiles where .NOTPARALLEL has bogus dependencies.
===============

The reaon that prompted me to write this was this:

In our company, the makefile system is built so that there are no recursive 
makes, and you can build from any subdirectory. The infrastructure is 
cross-platform, compiling in Windows, Solaris, Linux, Mac, and FreeBSD.

Recently we started using 'icecream' (the successor of distcc) to speed up 
compilation times. While that works, there are problems with paralellization. 
Only compilation jobs are distributed, linking remains local. Thus, if we 
type (as we do)

make -j

we have very fast compilation but the local machine ends up heavily overloaded 
when, for example, make decides that 10 libraries and executables are to be 
linked in parallel.

Right now we have a kludge in the makefile that serializes those things, but 
that is not ideal for many reasons. One of them is that in calling make 
recursively we waste parsing time. Another one is that once the recursive 
call is made, targets that would otherwise be run in parallel (while in that 
call) are not. We tried other solutions but all have some pretty serious 
drawbacks too.

My solution, implemented in the patch, is this: introduce a 
special ".SERIAL" target, which makes all its dependencies be serialized, 
that is, they can only be built when nothing else is being built. If no 
dependencies are specified, then it is equivalent to ".NOTPARALLEL". The 
reason I didn't use the existent ".NOTPARALLEL" is that it would break 
compatibility with people who used the target with bogus dependencies. As far 
as I know there is something similar in SCO make, under the ".MUTEX" special 
target.

Code review (that was my first look at make code) and suggestions welcome.

- Cesar