[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnuastro-commits] master bfed8f15: Book: more practical example for Mak
From: |
Mohammad Akhlaghi |
Subject: |
[gnuastro-commits] master bfed8f15: Book: more practical example for Make extension ast-text-prev-in-list |
Date: |
Thu, 15 Feb 2024 14:09:23 -0500 (EST) |
branch: master
commit bfed8f15a921d9ad04df47289f62eb65192c9866
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Book: more practical example for Make extension ast-text-prev-in-list
Until now, the example of this Make extension was too basic to show its
usefulness!
With this commit, a more practical scenario has been shown in the
documentation to help readers better appreciate it.
---
doc/gnuastro.texi | 50 ++++++++++++++++++++++++++++++++++++++++----------
1 file changed, 40 insertions(+), 10 deletions(-)
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index a2e0e216..45307026 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -35918,25 +35918,55 @@ all:
Returns the word in @code{LIST} that is previous to @code{TARGET}.
If @code{TARGET} is the first word of the list, or is not within it at all,
this function will return an empty string (nothing).
-One scenario when this function can be useful is when you want a list of
targets to always be executed in sequence (even when Make is run in parallel).
-The @code{sleep} command is used in the recipe to show the serialization more
clearly.
-To see the effect of this function put the example below in a @file{Makefile}
and run @code{make -j4} (to simultaneously exectute 4 jobs); then
comment/remove this function (so there is no prerequisite) and re-run
@code{make -j4}.
+One scenario when this function can be useful is when you want a list of
higher-level targets to always be executed in sequence (even when Make is run
in parallel).
+But you want their lower-level prerequisites to be executed in parallel.
+
+The fully working example below shows this in practice: the ``final'' target
depends on the sub-components @file{a.fits}, @file{b.fits}, @file{c.fits} and
@file{d.fits}.
+But each one of these has seven dependencies (for example @file{a.fits}
depends on the sub-sub-components @file{a-1.fits}, @file{a-2.fits},
@file{a-3.fits}, ...).
+Without this function, Make will first build all the sub-sub-components first,
then the sub-components and ultimately the final target.
+
+When the files are small and there aren't too many of them, this is not a
problem.
+But when you hundreds/thousands of sub-sub-components, your computer may not
have the capacity to hold them all in storage or RAM (during processing).
+In such cases, you want to build the sub-components to built in series, but
the sub-sub-components of each sub-component to be built in parallel.
+This function allows just this in an easy manner as below: the
sub-sub-components of each sub-component depend on the previous sub-component.
+
+To see the effect of this function put the example below in a @file{Makefile}
and run @code{make -j12} (to simultaneously exectute 12 jobs); then
comment/remove this function (so there is no prerequisite in @code{$(subsubs)})
and re-run @code{make -j12}.
@example
+# Basic settings
all: final
.SECONDEXPANSION:
load /usr/local/lib/libgnuastro_make.so
-list=$(foreach i,a b c d,$(i).txt)
-$(list): %.txt: $$(ast-text-prev-in-list $$@@, $(list))
- @@sleep 0.3; echo "$@@: $^"
+# 4 sub-components (alphabetic), each with 7
+# sub-sub-components (numeric).
+subids = a b c d
+subsubids = 1 2 3 4 5 6 7
+subs=$(foreach s, $(subids), $(s).fits)
+subsubs=$(foreach s, $(subids), \
+ $(foreach ss, $(subsubids), \
+ $(s)-$(ss).fits))
-final: $(list)
+# Build the sub-components:
+$(subsubs): %.fits: $$(ast-text-prev-in-list \
+ $$(word 1, $$(subst -, ,%)).fits, \
+ $(subs))
+ @@echo "$@@: $^"
+
+# Build the final components
+$(subs): %.fits: $$(foreach s, $(subsubids), %-$$(s).fits)
+ @@echo "$@@: $^"
+
+# Final
+final: $(subs)
+ @@echo "$@@: $^"
@end example
-As you see, when this function is in the prerequisite list, the recipes are
run one by one; but without it, all four are printed after an initial 0.3
seconds of delay.
-Unfortunately the @code{.NOTPARALLEL} target of GNU Make, will also serialize
the prerequisites of the targets given to it.
-But through this function, the prerequisites of the serialized targets will
still be executed in parallel.
+As you see, when this function is present, the sub-sub-components of each
sub-component are executed in parallel, while at each moment, only a single
sub-component's prerequisites are being made.
+Without this function, make first builds all the sub-sub-components, then goes
to the sub-components.
+There can be any level of components between these, allowing this operation to
be as complex as necessary in your data analysis pipeline.
+Unfortunately the @code{.NOTPARALLEL} target of GNU Make doesn't allow this
level of customization.
+
@item $(ast-text-to-upper STRING)
Returns the input string but with all characters in UPPER-CASE.
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [gnuastro-commits] master bfed8f15: Book: more practical example for Make extension ast-text-prev-in-list,
Mohammad Akhlaghi <=