make -j 在有和没有 sub make 的情况下表现不同
make -j behaves differently with and without sub makes
有人可以解释为什么 make 和 sub-make 有不同的并行化行为吗?
下面有两个例子。在一个中,所有目标都在一个 Makefile 中,在第二个示例中每个目标都有自己的 folder/makefile 并且它们使用 $(MAKE)
.
相互调用
运行 它们与 -j
产生的输出截然不同。在第一种情况下,所有对顶级依赖项(下面的a.out
)的访问都是同步的。在 运行 任何依赖它的目标之前等待目标构建。
在第二种情况下,顶级目标(下面的a.out
)受到并发访问。这会给我们带来严重的问题,除非我们使用自己的同步技术来克服它。
示例 1:
jesaremi@u16-3:~/maketest$ cat Makefile
.ONESHELL:
all: d.out b.out c.out a.out
a.out: CALLER ?= self
a.out:
@
echo entering $@ "(called by $(CALLER))"
sleep 10
echo exiting $@
b.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
c.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
d.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
jesaremi@u16-3:~/maketest$ make
entering a.out (called by self)
exiting a.out
entering d.out
exiting d.out
entering b.out
exiting b.out
entering c.out
exiting c.out
jesaremi@u16-3:~/maketest$ make -j
entering a.out (called by self)
exiting a.out
entering d.out
entering c.out
entering b.out
exiting c.out
exiting d.out
exiting b.out
示例 2(使用 sub-make):
jesaremi@js-u16-1:~/maketest$ cat */Makefile
---------- a/Makefile -------------
.ONESHELL:
a.out: CALLER ?= self
a.out:
@
echo entering $@ "(called by $(CALLER))"
sleep 10
echo exiting $@
---------- b/Makefile -------------
.ONESHELL:
export CALLER:=b
a.out:
$(MAKE) -C ../a
b.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
---------- c/Makefile -------------
.ONESHELL:
export CALLER:=c
a.out:
$(MAKE) -C ../a
c.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
---------- d/Makefile -------------
.ONESHELL:
export CALLER:=d
a.out:
$(MAKE) -C ../a
d.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
jesaremi@js-u16-1:~/maketest$ cat Makefile
all: d.out c.out b.out a.out
%.out:
$(MAKE) -C $*
.PHONY:all
jesaremi@js-u16-1:~/maketest$ make
make -C d
make[1]: Entering directory '/home/jesaremi/maketest/d'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by d)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/d'
make -C c
make[1]: Entering directory '/home/jesaremi/maketest/c'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by c)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/c'
make -C b
make[1]: Entering directory '/home/jesaremi/maketest/b'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by b)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/b'
make -C a
make[1]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by self)
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/a'
jesaremi@js-u16-1:~/maketest$ make -j
make -C d
make -C c
make -C b
make -C a
make[1]: Entering directory '/home/jesaremi/maketest/d'
make -C ../a
make[1]: Entering directory '/home/jesaremi/maketest/c'
make -C ../a
make[1]: Entering directory '/home/jesaremi/maketest/b'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
make[1]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by self)
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by c)
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by d)
entering a.out (called by b)
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/a'
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/c'
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/d'
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/b'
也许您认为 make 的递归调用相互通信 他们正在构建的目标 并且以某种方式互锁,这样就没有两个子 make 试图构建相同的同时瞄准。
这与 make 可以做的事情相去甚远。 make 所能做的就是通知其他子 makes 他们正在构建多少 个目标,以便确保您的目标总数不超过 N(对于-jN
) 被调用。
如果您想确保两个不同的 makefile 不会尝试构建相同的目标,这取决于您如何组织您的 makefile; make 不能为你做。
在第二个示例中,您将所有先决条件列为一个目标:
all: d.out c.out b.out a.out
这个食谱说,"before all
can be built the targets d.out
, c.out
, b.out
, and a.out
must be finished"。这表示 nothing 先决条件(d.out
、c.out
、b.out
或 a.out
)本身之间的相对关系,因此当您运行 和 -j
将同时调用所有这些子品牌。如果它们中的每一个都试图构建相同的共享目标,那么它们将相互干扰。
如果您想确保不会发生这种情况,您必须在您的 makefile 中声明先决条件关系,以便 make 了解它。例如你可以这样做:
all: d.out c.out b.out a.out
d.out c.out b.out: a.out
%.out:
$(MAKE) -C $*
.PHONY:all
现在,构建 d.out
、c.out
和 b.out
的配方将在构建 a.out
的配方完成后才会开始。
有人可以解释为什么 make 和 sub-make 有不同的并行化行为吗?
下面有两个例子。在一个中,所有目标都在一个 Makefile 中,在第二个示例中每个目标都有自己的 folder/makefile 并且它们使用 $(MAKE)
.
运行 它们与 -j
产生的输出截然不同。在第一种情况下,所有对顶级依赖项(下面的a.out
)的访问都是同步的。在 运行 任何依赖它的目标之前等待目标构建。
在第二种情况下,顶级目标(下面的a.out
)受到并发访问。这会给我们带来严重的问题,除非我们使用自己的同步技术来克服它。
示例 1:
jesaremi@u16-3:~/maketest$ cat Makefile
.ONESHELL:
all: d.out b.out c.out a.out
a.out: CALLER ?= self
a.out:
@
echo entering $@ "(called by $(CALLER))"
sleep 10
echo exiting $@
b.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
c.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
d.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
jesaremi@u16-3:~/maketest$ make
entering a.out (called by self)
exiting a.out
entering d.out
exiting d.out
entering b.out
exiting b.out
entering c.out
exiting c.out
jesaremi@u16-3:~/maketest$ make -j
entering a.out (called by self)
exiting a.out
entering d.out
entering c.out
entering b.out
exiting c.out
exiting d.out
exiting b.out
示例 2(使用 sub-make):
jesaremi@js-u16-1:~/maketest$ cat */Makefile
---------- a/Makefile -------------
.ONESHELL:
a.out: CALLER ?= self
a.out:
@
echo entering $@ "(called by $(CALLER))"
sleep 10
echo exiting $@
---------- b/Makefile -------------
.ONESHELL:
export CALLER:=b
a.out:
$(MAKE) -C ../a
b.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
---------- c/Makefile -------------
.ONESHELL:
export CALLER:=c
a.out:
$(MAKE) -C ../a
c.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
---------- d/Makefile -------------
.ONESHELL:
export CALLER:=d
a.out:
$(MAKE) -C ../a
d.out: a.out
@
echo entering $@
sleep 1
echo exiting $@
jesaremi@js-u16-1:~/maketest$ cat Makefile
all: d.out c.out b.out a.out
%.out:
$(MAKE) -C $*
.PHONY:all
jesaremi@js-u16-1:~/maketest$ make
make -C d
make[1]: Entering directory '/home/jesaremi/maketest/d'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by d)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/d'
make -C c
make[1]: Entering directory '/home/jesaremi/maketest/c'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by c)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/c'
make -C b
make[1]: Entering directory '/home/jesaremi/maketest/b'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by b)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/b'
make -C a
make[1]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by self)
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/a'
jesaremi@js-u16-1:~/maketest$ make -j
make -C d
make -C c
make -C b
make -C a
make[1]: Entering directory '/home/jesaremi/maketest/d'
make -C ../a
make[1]: Entering directory '/home/jesaremi/maketest/c'
make -C ../a
make[1]: Entering directory '/home/jesaremi/maketest/b'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
make[1]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by self)
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by c)
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by d)
entering a.out (called by b)
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/a'
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/c'
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/d'
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/b'
也许您认为 make 的递归调用相互通信 他们正在构建的目标 并且以某种方式互锁,这样就没有两个子 make 试图构建相同的同时瞄准。
这与 make 可以做的事情相去甚远。 make 所能做的就是通知其他子 makes 他们正在构建多少 个目标,以便确保您的目标总数不超过 N(对于-jN
) 被调用。
如果您想确保两个不同的 makefile 不会尝试构建相同的目标,这取决于您如何组织您的 makefile; make 不能为你做。
在第二个示例中,您将所有先决条件列为一个目标:
all: d.out c.out b.out a.out
这个食谱说,"before all
can be built the targets d.out
, c.out
, b.out
, and a.out
must be finished"。这表示 nothing 先决条件(d.out
、c.out
、b.out
或 a.out
)本身之间的相对关系,因此当您运行 和 -j
将同时调用所有这些子品牌。如果它们中的每一个都试图构建相同的共享目标,那么它们将相互干扰。
如果您想确保不会发生这种情况,您必须在您的 makefile 中声明先决条件关系,以便 make 了解它。例如你可以这样做:
all: d.out c.out b.out a.out
d.out c.out b.out: a.out
%.out:
$(MAKE) -C $*
.PHONY:all
现在,构建 d.out
、c.out
和 b.out
的配方将在构建 a.out
的配方完成后才会开始。