Re: Gnu make option to fix parallel multiple ar on a shared archive

[Philip Guenther]

On Sun, Aug 11, 2024 at 11:31 AM Kaz Kylheku <kaz@kylheku.com> wrote:
>
> On 2024-08-10 22:37, Philip Guenther wrote:
> > I think the correctness concern is that this provides mutual exclusion
> > for ar only.  Other programs may be reading the .a, including make
> > itself.  If your system's ar does atomic updates (write new temp file,
> > rename over existing .a) then the readers should have a
> > self-consistent view and this should work Just Fine.  If not, then
> > depending on exactly how it does update then make may get confused
> > about the .a contents.
> >
> >
> > The performance concern is two-fold:
> >
> > 1) ar's normal operation includes rebuilding the symbol table after
> > each change.  Adding objects to a .a with separate ar invocations
> > results in O(N^2) symbol operations.
> >
> > 2) make's parallel operation isn't aware of the "these operations will
> > be serialized" behavior, so make may schedule multiple 'invoke rule
> > using $(AR)' recipes concurrently despite them being serialized,
> > resulting in reduced parallelism vs the "build all of the out-of-date
> > objects then insert them all at once" rule setup.
>
> Idea: fix the code generator so that for each rule that updates
> an archive, it uses a unique name for the archive specific to that
> rule. Perhaps by using numbers:   libfoo.001.a, libfoo.002.a, ...
>
> Then, the generator must also emit a new rule like this
>
>   libfoo.a: libfoo.001.a libfoo.002.a ...
>      ...
>
> whose recipe lines combines the smaller archives into the one big one
> that is then used elsewhere as a dependency.

Hmm, can you expound on the value the libfoo.###.a files provide?  If
each of those contains just one file that will end up as a member of
the final target libfoo.a, then I'm missing what they provide over
just having the libfoo.a rule rebuild the target from all the .o files
with neither libfoo.a(bar.o) nor libfoo.###.a as prerequisites.  There
must be more to these libfoo.###.a files that I'm not seeing.


Philip  Guenther