[CMake] Building all tests in parallel (whole subtrees)

[Wojciech Knapik]

On Thu, Jan 24, 2013 at 11:19:11AM -0800, Alan W. Irwin wrote:

> Therefore, I advice using unique target names (just like I do).  If
> you happen to like a particular target name such as "ut", then append
> a suffix to it (such as subdirectory name) to make that target easy to
> identify and globally unique.

That was my thought initially, but I couldn't find a suffix, that would
be unique and obvious (the last directory in the current path would not
give me that - not unique) and it all felt like a hack, so I gave up on
that route.

> I don't understand _why_ you are running into race conditions with
> parallel builds. Sure that is the fundamental issue with parallel
> builds, but that is addressed by implementing proper dependencies. 

I don't know why, but I can tell you how it looks like. It looks exactly
like the issue with recursive make, where common dependencies outside
the current subtree are built by two make processes, that have no idea
about each other.

Is this the case with CMake-generated makefiles ? I don't know. But it
seems only the "all" target is guaranteed to build >1 dependency in
parallel with no issues. I thought this was well known and understood...

> You used a pure Makefile approach previously.  Didn't you have the
> same problem there until you fixed all dependencies?  Assuming your
> pure Makefile system did have proper dependencies, all you have to do
> is also implement those same dependencies for your CMake-based system.

The original makefiles did not have almost any dependencies. I had to
figure them out myself. It's not rocket science, but it was not a
pleasant task.

Anyway - yeah, the dependencies are there now and I don't think I'm
missing any. If I create a target foo, that depends on bar and qux and
both bar and qux do target_link_libraries(target fred), then when I try
to build foo in parallel, I get a race when building fred as a
dependency for bar and qux.

Like I said above though, this is probably about the location of the
targets foo, bar, baz, qux and fred in the source tree relative to each
other.

I don't feel like researching this further, I've spent enough time on
it, so I've just moved on to other options.

> Note, CMake has both file dependencies and target dependencies, and
> both have to be set up correctly in order for parallel builds to work
> properly.  CMake often does the right thing here, but there are also
> cases where you have to explicitly tell it about dependencies. For
> more detail on the two kinds of dependencies look extremely carefully
> at the documentation of add_custom_command, add_custom_target, and
> add_dependencies.  I tend to carefully re-read that documentation
> every time I implement a new test target.

Ok, so let's say I have a custom command that generates code from
.idl file for instance. If I give all the targets that require that
generated code a dependency on a target that depends on the generated
files - is that enough ?

> I believe I have responded to all your questions above and also
> questions implied by your posts before and after the above one.
> However, often the example of an implemented build system for a
> real-world project is a quicker method than question and answer to
> learn about CMake. Therefore, I suggest you take a look at the
> CMake-based build systems of the recently released ephcom-3.0.0 and
> te_gen-2.0.0 projects where I implemented a fair number of test
> targets with proper dependencies between them.  See timephem.sf.net
> for access to the release announcements which should give you further
> links to download or browse those projects.  By the way,
> timephem.sf.net is outdated by those new releases, and I plan to
> update it appropriately soon. But until I do that, trust the release
> announcements more than timeephem.sf.net.  If you have further
> questions about the ephcom and/or te_gen build systems, don't hesitate
> to contact me again on or off list as you think appropriate.

Thanks for your answers. I'll take a look at those projects. This is the
first time I'm using CMake and, although it's easy to start using, I
could probably use some examples of how things should be done.

For now I handled the issue of tests by creating a "Test" build type. In
"Test", only tests and their dependencies are build, and in "Debug" and
"Release" only application code is built. 
The idea came from one of the developers and while I was a bit sceptical
in the beginning, it seems to work rather nicely. Wherever in the build
tree I go, I can just type "make -j" and get all the tests in that
subtree built.
Has anyone else used this approach ?

As for running - ctest does the job nicely.

Thanks again,
WK