[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: how to make dirs without races
From: |
Tim Murphy |
Subject: |
Re: how to make dirs without races |
Date: |
Tue, 5 Mar 2013 10:51:28 +0000 |
Hi,
Using rules to make directories is a huge pain in the neck for many
reasons. Directories behave more like files than one really wants
them to - any change to a directory by adding a file means it's time
is updated even if this is totally irrelevant, it will trigger a
rebuild of another file that has that directory as a prerequisite.
Order only prereqs don't help all that much - and of course I can't
remember why because that's GNUMake for you - it's so twisted and
complicated that I can't remember every last horrible windy road that
I have had to travel down.
I found it much simpler to create directories before the build. There
are probably loads of ways to do this well and the one I came up with
was overcomplicated but it worked quickly on exceedingly large builds.
It used $(shell) but very sparingly so that it wasn't too slow.
https://bitbucket.org/tnmurphy/raptor/src/fbb2e624d320e5eabc0689105e2f2b80d131ca03/lib/flm/metaflm.mk?at=default#cl-116
The general plan was:
1. Paths were absolute
2. Use mkdir -p
3. Feed up to 30 directories into 1 $(shell mkdir -p ..... ) call -
buffer until 30 are available.
4. Accept the fact that at every run of make you're going to attempt
to make directories - it's hardly noticeable.
for each target one might call:
$(call makepathfor,$(BITMAPHEADER))
Or for a logical collection of targets like all the targets that end
up in a library it's more efficient to append all required directories
to a variable like CREATABLEPATHS and then do it in one call:
$(call makepath,$(CREATABLEPATHS))
The buffering mechanism requires that you flush it at the end of your makefile:
$(call makepathfinalize)
Regards,
Tim
# Make the destination directory if necessary. For some
# make engines we must do this outside the rule or they
# get confused by the apparent way in which different rules
# can create the same particular directory and they infer some kind
# of dependency.
# Buffering with repeat prevention, makes directories after every 30
calls. Any more might overload
# the createprocess limit on arguments.
GNUMKDIR:=/usr/bin/mkdir
makepathLIST:=
# add path to buffer. If the buffer has reached 30 paths then make
them and clear it.
define makepath_single
$(if $(findstring $1,$(makepathLIST)),,$(eval makepathLIST:=$(makepathLIST) $1))
$(if $(subst 30,,$(words $(makepathLIST))),,$(shell $(GNUMKDIR) -p
$(makepathLIST))$(eval makepathLIST:=))
endef
# Attempting to make things not already made is slow. It's something
to do with $(eval)
# or to do with creating huge numbers of TARGET_ variables?
# define makepath
# $(info makepath_start)$(foreach DIR,$1,$(if $(TARGET_$(1)),,$(call
makepath_single,$(DIR))$(eval TARGET_$(1):=1)))$(info makepath_end)
# endef
define makepath
$(strip $(foreach DIR,$(sort $1),$(call makepath_single,$(DIR))))
endef
define makepathfor
$(call makepath,$(dir $1))
endef
# Make any remaining paths in the path buffer
define makepathfinalise
$(strip $(if $(makepathLIST),$(shell $(GNUMKDIR) -p
$(makepathLIST))$(eval makepathLIST:=),))
endef
--
You could help some brave and decent people to have access to
uncensored news by making a donation at:
http://www.thezimbabwean.co.uk/friends/