<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
I'm been using Boost Test, and I've written a module that lets you
add the executable and the tests (where possible, individual named
tests) with a single command - see my github repositories for
projects that include this module. <a class="moz-txt-link-freetext" href="http://github.com/rpavlik">http://github.com/rpavlik</a><br>
<br>
On 7/9/10 9:04 PM, Alok Govil wrote:
<blockquote cite="mid:COL101-W38F42325111839F59C2744BDB60@phx.gbl"
type="cite">
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style>
While I was suspecting something like this about CTest, your
explanation helps a lot.<br>
<br>
I have integrated UnitTest++ into my flow, and it is working quite
well. It does also return a 0 if all tests pass, so one could
integrate with CTest/CDash if desirable.<br>
<br>
Thanks<br>
<hr id="stopSpelling">Date: Fri, 9 Jul 2010 14:54:46 -0700<br>
From: <a class="moz-txt-link-abbreviated" href="mailto:chillery-cmake@lambda.nu">chillery-cmake@lambda.nu</a><br>
To: <a class="moz-txt-link-abbreviated" href="mailto:bo@askmonty.org">bo@askmonty.org</a><br>
CC: <a class="moz-txt-link-abbreviated" href="mailto:cmake@cmake.org">cmake@cmake.org</a><br>
Subject: Re: [CMake] CTest examples<br>
<br>
For CTest, running a test means running an executable and seeing
whether the result of that process is 0 (success) or not 0
(failure). It is also possible to scan the output of the
executable for certain regular expressions to determine success or
failure.<br>
<br>
It's a very simple and somewhat limited mechanism. It does depend
on you writing your own test driver, which makes sense because
CMake can't possibly know how to test your code.<br>
<br>
As an example, Zorba is an XQuery (xml query language) processor.
The majority of our tests are actual XQueries stored in text files
with the extension .xq, along with corresponding expected results
in .xml files. We have a testdriver program which executes the
query, checks the results, and returns 0 or 1.<br>
<br>
Our CMakeLists looks vaguely like this:<br>
<br>
add_executable(testdriver ${TESTDRIVER_SRCS})<br>
file(GLOB_RECURSE TESTFILES "${CMAKE_CURRENT_SOURCE_DIR}" "*.xq")<br>
foreach(TESTFILE ${TESTFILES})<br>
add_test(${TESTFILE} testdriver ${TESTFILE})<br>
endforeach()<br>
<br>
This creates one test for each .xq file. The name of the test is
the name of the .xq file, and that same name is passed as an
argument to testdriver.<br>
<br>
This works well, and lets us use ctest's functionality like -R to
run certain tests by name. It also results in nice easy-to-read
listings on CDash because there's one line for each test, making
it easy to examine failures. (Although given that we have over
20,000 tests, some pages on CDash take a looooong time to load...)
The main downside is speed, since every single test has to start
up a whole Zorba process.<br>
<br>
When you get to unit testing, however, IMHO things don't work as
well. CMake does offer the create_test_sourcelist() command, and
you should look into it. We use that in a couple places to create
a single test driver from several separate small unit test
drivers. However, you still need to write a loop in CMakeLists to
add multiple tests using that test driver.<br>
<br>
In another project I work on, I've been using CxxTest for a unit
testing framework. This is a really nice setup as it makes it very
trivial to write many unit tests. However, the integration with
ctest is poor. There's a reasonably good FindCxxTest script
shipped with CMake that makes it easy to get CxxTest suites
running; in a nutshell:<br>
<br>
find_package(CxxTest)<br>
cxx_add_test(my_unit_test my_unit_test.cpp
"${CMAKE_CURRENT_SOURCE_DIR}/my_unit_test.h")<br>
<br>
Unit tests in CxxTest are written as classes in a .h file;
cxx_add_test() runs CxxTest to generate the named .cpp file from
that, then builds it into the named test driver and adds a CTest
test for it. So far so good. However, the main benefit of CxxTest
is grouping similar test cases into a single .h file, probably
with similar setup and so forth. Unfortunately cxx_add_test() only
adds one test to CTest, and that one test runs all the test cases
in the .h file. So you lose the ability to run certain tests with
ctest -R, and the reports on CDash will just show whether ALL the
tests in my_unit_test succeeded or not.<br>
<br>
The fundamental problem is that ctest requires a test case to be a
program execution, and has no facilities to allow a program to
represent multiple tests. So either you pay the cost in speed, or
you lose some of the flexibility ctest and cdash offer. (To be
fair, even if this were fixed in ctest, CxxTest would also need
some modifications as it currently doesn't let you run subsets of
tests either.)<br>
<br>
Ceej<br>
aka Chris Hillery<br>
<br>
<div class="ecxgmail_quote">On Fri, Jul 9, 2010 at 4:37 AM, Bo
Thorsen <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:bo@askmonty.org">bo@askmonty.org</a>></span>
wrote:<br>
<blockquote class="ecxgmail_quote" style="padding-left: 1ex;">
Hi people,<br>
<br>
I have converted a set of applications to cmake and cpack, and
now have my eyes set on ctest.<br>
<br>
I'd like to hear if someone here has some good advice, or
links to good advice, on how to structure tests. I'm searching
for help on how to put different tests into what executables.
On how to handle multiple tests on each classes, on how to
best structure the test of the static libraries (all of those
are part of the source tree) that are linked in to the
application. And on how to test classes from the main
application itself.<br>
<br>
I have read the ctest FAQ, documentation etc. and still don't
know anything that help me write the actual test code.<br>
<br>
From the looks of it, ctest only provides the framework to run
a test, no help is given to write the code of the tests
themselves, is this right? I have previously been using
cppunit, and it looks like this will still be useful.<br>
<br>
To sum it up, I'm looking for real life advice on what you
guys have done with ctest. This information seem almost
completely missing on the net, where all searches on ctest
leads to useless presentation on ctest features.<br>
<br>
Cheers,<br>
<br>
Bo Thorsen.<br>
Monty Program AB.<br>
<font color="#888888">
<br>
-- <br>
<br>
MariaDB: MySQL replacement<br>
Community developed. Feature enhanced. Backward compatible.</font>
<div>
<div class="h5"><br>
_______________________________________________<br>
Powered by <a moz-do-not-send="true"
href="http://www.kitware.com" target="_blank">www.kitware.com</a><br>
<br>
Visit other Kitware open-source projects at <a
moz-do-not-send="true"
href="http://www.kitware.com/opensource/opensource.html"
target="_blank">http://www.kitware.com/opensource/opensource.html</a><br>
<br>
Please keep messages on-topic and check the CMake FAQ at:
<a moz-do-not-send="true"
href="http://www.cmake.org/Wiki/CMake_FAQ"
target="_blank">http://www.cmake.org/Wiki/CMake_FAQ</a><br>
<br>
Follow this link to subscribe/unsubscribe:<br>
<a moz-do-not-send="true"
href="http://www.cmake.org/mailman/listinfo/cmake"
target="_blank">http://www.cmake.org/mailman/listinfo/cmake</a><br>
</div>
</div>
</blockquote>
</div>
<br>
<br>
_______________________________________________
Powered by <a class="moz-txt-link-abbreviated" href="http://www.kitware.com">www.kitware.com</a>
Visit other Kitware open-source projects at
<a class="moz-txt-link-freetext" href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a>
Please keep messages on-topic and check the CMake FAQ at:
<a class="moz-txt-link-freetext" href="http://www.cmake.org/Wiki/CMake_FAQ">http://www.cmake.org/Wiki/CMake_FAQ</a>
Follow this link to subscribe/unsubscribe:
<a class="moz-txt-link-freetext" href="http://www.cmake.org/mailman/listinfo/cmake">http://www.cmake.org/mailman/listinfo/cmake</a>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
Powered by <a class="moz-txt-link-abbreviated" href="http://www.kitware.com">www.kitware.com</a>
Visit other Kitware open-source projects at <a class="moz-txt-link-freetext" href="http://www.kitware.com/opensource/opensource.html">http://www.kitware.com/opensource/opensource.html</a>
Please keep messages on-topic and check the CMake FAQ at: <a class="moz-txt-link-freetext" href="http://www.cmake.org/Wiki/CMake_FAQ">http://www.cmake.org/Wiki/CMake_FAQ</a>
Follow this link to subscribe/unsubscribe:
<a class="moz-txt-link-freetext" href="http://www.cmake.org/mailman/listinfo/cmake">http://www.cmake.org/mailman/listinfo/cmake</a></pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Ryan Pavlik
Human-Computer Interaction Graduate Student
Virtual Reality Applications Center
Iowa State University
<a class="moz-txt-link-abbreviated" href="mailto:rpavlik@iastate.edu">rpavlik@iastate.edu</a>
<a class="moz-txt-link-freetext" href="http://academic.cleardefinition.com/">http://academic.cleardefinition.com/</a></pre>
</body>
</html>