Re: About .NOTPARALLEL

[Peng Yu]

On Sun, Apr 17, 2011 at 7:06 PM, Paul Smith <address@hidden> wrote:
> On Sun, 2011-04-17 at 16:44 -0500, Peng Yu wrote:
>> However, I think the syntax of .NOTPARALLEL can be improved a little.
>> For example, I'd like the a.txt and b.txt are not generated in
>> parallel with anything other .txt files, but c.txt and d.txt be
>> generated in parallel, when '-j' option is used. Currently the
>> following Makefile is essentially the same as the above one. May I
>> suggest to add this syntax to make?
>> .PHONY: all clean
>>
>> all: a.txt b.txt c.txt d.txt
>>
>> .NOTPARALLEL: a.txt b.txt
>>
>> %.txt:
>>       sleep 2; touch $@
>>
>> clean:
>>       $(RM) *.txt
>
> There are two issues here.  First, deciding on the syntax is one thing;
> implementing it inside of GNU make is decidedly another thing entirely.
> Currently make has a concept of parallelism, or no parallelism.  There's
> no concept of "per-target parallelism".  Getting make's internal
> algorithms to understand that some targets should not be built in
> parallel, skipped, and come back to later when there's nothing else
> running is not a trivial change.

You get my point.

> Second, your syntax is not very clear.  Does it mean that a.txt and
> b.txt should never be built in parallel with anything else at all?  So
> before we build a.txt we must wait for all other running jobs to
> complete and not start any new ones, then we run a.txt, then the same
> for b.txt after a.txt completes?  Or does it mean that a.txt and b.txt
> cannot be run in parallel with each other but they could be run at the
> same time as any other targets?  Or...?

My syntax above means that a.txt can not be run in parallel with any
other targets (i.e., b.txt, c.txt, d.txt). b.txt can not be run in
parallel with a.txt, c.txt, and d.txt. But c.txt and d.txt can run
parallel. Basically, any dependencies to .NOTPARALLEL can not run in
parallel with any targets in the Makefile. It is not trivial to
implement per target parallelism in a general sense. But I think that
it may not be that difficult to implement for the scenario that I'm
proposing.

The scenario is common in practice. Suppose that the generation of
a.txt and b.txt can use all the cores in a machine. Therefore, you
don't want them be run in parallel with any other processes. But
generation of c.txt and d.txt use just a single core, then you want
them to run in parallel. I hope that I make my point clear.

-- 
Regards,
Peng