make: disable parallel execution of some targets

I have a compile job where linking is taking a lot of IO work. We have around a dozen of cores开发者_如何学运维 so we run make -j13, but when it comes to linking the 6 targets, I'd like those to be done in a round robin way. I thought about making one depend on the next but I think this would break the individual targets. Any ideas how to solve this small issue?

---

make itself doesn't provide a mechanism to request "N of these, but no more than M of those at a time".

You might try using the sem command from the GNU parallel package in the recipe of your linker rules. Its documentation has an example of ensuring only one instance of a tool runs at once. In your example, you would allow make to start up to 13 sems at a time, but only one of those at a time will run the linker, while the others block.

The downside is that you could get into a situation where 5 of your make's 13 job slots are tied up with instances of sem that are all waiting for a linker process to finish. Depending on the structure of your build, that might mean some wasted CPU time. Still beats 6 linkers thrashing the disk at once, though :-)

---

You should specify that your six targets cannot be built in parallel. Add a line like this to your makefile:

.NOTPARALLEL: target1 target2 target3 target4 target5 target6

For more information look here https://www.gnu.org/software/make/manual/html_node/Parallel-Disable.html.

---

I've stumbled upon a hacky solution:

For each recipe it runs, Make does two things: it expands variables/functions in the recipe, and then runs the shell commands.

Since the first step can read/write the global variables, it seems to be done synchronously.

So if you run all your shell commands during the first step (using $(shell )), no other recipe will be able to start while they're running.

E.g. consider this makefile:

all: a b

a:
        sleep 1

b:
        sleep 1

time make -j2 reports 1 second.

But if you rewrite it to this:

# A string of all single-letter Make flags, without spaces.
override single_letter_makeflags = $(filter-out -%,$(firstword $(MAKEFLAGS)))

ifneq ($(findstring n,$(single_letter_makeflags)),)
# See below.
override safe_shell = $(info Would run shell command: $1)
else ifeq ($(filter --trace,$(MAKEFLAGS)),)
# Same as `$(shell ...)`, but triggers a error on failure.
override safe_shell = $(shell $1)$(if $(filter-out 0,$(.SHELLSTATUS)),$(error Unable to execute `$1`, exit code $(.SHELLSTATUS)))
else
# Same functions but with logging.
override safe_shell = $(info Shell command: $1)$(shell $1)$(if $(filter-out 0,$(.SHELLSTATUS)),$(error Unable to execute `$1`, exit code $(>
endif

# Same as `safe_shell`, but discards the output and expands to nothing.
override safe_shell_exec = $(call,$(call safe_shell,$1))


all: a b

a:
        $(call safe_shell_exec,sleep 1)
        @true

b:
        $(call safe_shell_exec,sleep 1)
        @true

time make -j2 now reports 2 seconds.

Here, @true does nothing, and suppresses Nothing to be done for ?? output.

There are some problems with this approach though. One is that all output is discarded unless redirected to file or stderr...

---

It won't break individual targets. You can create any number of (:) rules for a target, as long as only one of them has an actual recipe for building it. This appears to be a good use case for that.