[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
make behavior WRT background processes
From: |
Mervyn Keene |
Subject: |
make behavior WRT background processes |
Date: |
Mon, 19 Feb 2018 08:36:13 -0800 |
I'm looking for insight in the area of how make handles background
processes started from recipes.
Let me
start
with
a
summary
of
my
real-word scenario: we have a Linux makefile which is large, complex and
proprietary but in the end all it does is compile a bunch of objects and
link them together. After
that
final binary is linked there are quite a few post-actions performed on it
involving objdump, objcopy, and other ELF-wrangling tools. Because there
are so many of these post-tasks I've wrapped them up in a Python script. So
the basic workflow
, from make's POV,
looks like:
-
compile many objects
-
link them together
into blah.exe
-
run a python script
which operates on blah.exe
One of the sub-tasks invoked from the Python script is (a)
quite
slow and (b)
rarely
important (it's a disassembly which is occasionally useful in debugging
but is never used within the build per se). I'd like to add a "fast build"
mode in which the disassembly is spun off in the background, the build
returns without waiting for it, and
that file
shows
up
later (or not
, if it fails
). Of course there would be a production mode
which builds it
synchronously too.
I've implemented this but make
still
appears to wait for the background process anyway. In other words
the disassembler
definitely goes into the background, subsequent tasks run, everything
is finished
...
but
then make waits for the background process
so all the benefit is lost
. I've put together a simple test case
(below)
but the problem isn't reproduced
by it
and w
ithout being able to reproduce I'm a little stuck. AFAICT
this
test case faithfully reproduces the situation from a process POV but it
behaves "correctly"
by not waiting:
% cat Makefile
.PHONY: all
all: fast1 slow1 fast2
@echo "MAKE DONE"
fast1:
@echo "Done with $@"
fast2:
@sleep 1
@echo "Done with $@"
slow1:
python -c "import os; os.system('(sleep 5; echo PYTHON JOB DONE) &')"
@echo "Done with $@"
% make -j
python -c "import os; os.system('(sleep 5; echo PYTHON JOB DONE) &')"
Done with fast1
Done with slow1
Done with fast2
MAKE DONE
[time passes ...]
% PYTHON JOB DONE
Note that make says "MAKE DONE"
and exits back to the shell,
then
5 seconds later
the "PYTHON JOB DONE"
arrives
. This is exactly
the behavior
desired but in the real makefile the "PYTHON JOB DONE" shows up before
"MAKE DONE"
meaning that make has waited for it.
Does anyone have a
theory
about
how this could happen
?
- make behavior WRT background processes,
Mervyn Keene <=