View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0002336CMakeCTestpublic2005-10-06 14:122010-12-14 18:50
ReporterMiguel Figueroa 
Assigned ToDavid Cole 
PriorityhighSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Version 
Target VersionFixed in Version 
Summary0002336: ctest -C option doesn't build proper configuration
DescriptionI think I found a bug with ctest and below I report what I have found so far and my hack at it in the mean time. I will appreciate clarifications from the more experienced CMake users, and I would gladly help reproduce/debug/fix this if it is in fact a bug. If it is the intended behavior, it is a non-intuitive one and at least should be documented properly in the usage and elsewhere.


Goal
----
I am interested in using ctest script to build/test/submit a nightly build of VXL (and CMake as well, now that I'm at it), but I want to have the debug and release builds in the same binary tree (although the source tree is separated).


System
------
- OS: WinXP Pro with SP2
- Compiler: Visual Studio 7 .NET 2003
- Machine: AMD Athlon XP 2400+ 2GHz 1GB DDR RAM
- cmake and vxl updated daily from cvs


Problem/Bug
-----------
After several hours (well hours throughout several days!! ;) ), I can't get the debug configuration built with the following (typed in the command line):

cmake /path/to/source/CMake
ctest -D NightlyStart
ctest -D NightlyConfigure
ctest -C Debug -D NightlyBuild

By inspecting the output files, the build command is:

.../devenv.com CMake.sln /build Release /project ALL_BUILD

ctest -C Debug -D NightlyTest -- does choose the correct tests to build, only they all fail because they weren't built. Something like this:

""
CMake Error: config type specified on the command line, but test executable not found. D:/builds/CMake/bin/Debug/ctest 50/ 50 Testing VSExternalInclude

The following tests FAILED:
          1 - kwsys.testSystemTools (Not Run)
          2 - kwsys.testProcess-1 (Not Run)
                      ...
         50 - VSExternalInclude (Not Run)
""


Diagnostics
-----------
After going into the ctest source code and using the debugger, I see that it is using the command read from the DartConfiguration.tcl file and it is not overiding the "/build" value with the "-C" flag. That is, it doesn't modify the makecommand in none of:

m_CTestConfiguration["MakeCommand"]
m_CTestConfigurationOverwrites["MakeCommand"]

from cmCTest class in Source/cmCTest.cxx

I guess that when the "-C" flag is encountered, it should set something somewhere and at some point generate the make command according to this value (maybe in one of the two map variables above). However, I don't understand the CMake/CTest architecture enough to know where (or if) it should be setting it.


Work-Around
-----------
The only way I have been able to get around this is by supplying the MAKECOMMAND variable in the cmake command as in:

cmake -DMAKECOMMAND:STRING=".../devenv.com CMake.sln /build Debug /project ALL_BUILD" /path/to/source/CMake
ctest -D NightlyStart
ctest -D NightlyConfigure
ctest -C Debug -D NightlyBuild // I think I don't need the -C Debug here
ctest -C Debug -D NightlyTest // I do need it here

I don't know if this is the intended behavior, but it is unintuitive to me that if I can supply the "-C Debug" in the ctest command line it only works for selecting the correct test, but not building them... Sorry for the long blabering, if for other reasons it is designed this way. But then, I think some proper documentation is needed to make clear the workings of the "-C" option.


Finally
-------
My solution is being tested and submitted to dashboard under:

site: miguelf.home_desktop

and you can see the script file used in the Notes.xml file.


Thanks in advance for any help you can provide me,
--Miguel
TagsNo tags attached.
Attached Filespatch file icon build_config.patch [^] (1,094 bytes) 1969-12-31 19:00 [Show Content]

 Relationships
related to 0011026closedDavid Cole CMake 2.8.1 breaks parallel builds during CTest. MAKECOMMAND not respected. 

  Notes
(0004561)
Mike Talbot (reporter)
2006-07-31 06:43

 I have a similar problem where I am trying to do the same thing using the script mode, i.e. run this command

ctest -C Debug -S script.cmake

where script.cmake contains this:

CMAKE_MINIMUM_REQUIRED(VERSION 2.4)

SITE_NAME(CTEST_SITE)

SET(CTEST_DASHBOARD_ROOT "${CTEST_SCRIPT_DIRECTORY}/../..")
SET(CTEST_SOURCE_DIRECTORY "${CTEST_DASHBOARD_ROOT}/src")
SET(CTEST_BINARY_DIRECTORY "${CTEST_DASHBOARD_ROOT}/test_build")

SET(CTEST_PROJECT_NAME "Ecl")
SET(CTEST_CMAKE_GENERATOR "Visual Studio 7 .NET 2003")

CTEST_START(Nightly)
CTEST_CONFIGURE(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_BUILD(BUILD "${CTEST_BINARY_DIRECTORY}")
CTEST_TEST(BUILD "${CTEST_BINARY_DIRECTORY}")

When I run this script, ctest will build Release but try to test Debug and fails. I can add this:

SET(CTEST_BUILD_CONFIGURATION Debug)

In which case it works correctly, however I have to make sure that this variable is set to the same value as the -C argument. I have attached a patch which does this.

Unfortunately this does not solve Miguel's issue since -D Nightly just uses MAKECOMMAND directly (via MakeCommand in DartConfiguration.tcl) which is hard coded to Release when it is initialized in cmBuildCommand, via Modules/CTest.cmake.
(0014093)
David Cole (manager)
2008-11-12 15:00

 Turns out that there is a workaround for this long-standing, low-priority issue:

If you set an environment variable named CMAKE_CONFIG_TYPE on the very first CMake configure step, it will produce the correct MAKECOMMAND in the cache.

Otherwise, the cached value of MAKECOMMAND is what ends up getting used in most of the "build via ctest" scenarios. I have this issue on my queue now and will ponder whether or not a non-breaking fix may be made... So far, I have not come up with one, but it is on one of my mental back burners for now...
(0018695)
David Cole (manager)
2009-12-04 17:07

 This commit fixes this issue in CVS HEAD of CMake today.

$ cvs commit -m "Fix issue 0002336 - honor the -C arg to ctest. Honor it for all stages of running -D dashboards from the command line and running ctest_configure, ctest_build and ctest_test commands in -S scripts. Also, allow a script to change it by setting the CTEST_CONFIGURATION_TYPE variable: allows for multiple configuration build/test cycles within one script. Add a new signature for the cmake command build_command that accepts CONFIGURATION as one argument. The original build_command signature is still there, but now marked as deprecated in the documentation. Of course... also add CTestConfig tests to verify that -C is honored for -D dashboards and -S scripts."

/cvsroot/CMake/CMake/Modules/CTest.cmake,v <-- Modules/CTest.cmake
new revision: 1.21; previous revision: 1.20
/cvsroot/CMake/CMake/Modules/DartConfiguration.tcl.in,v <-- Modules/DartConfiguration.tcl.in
new revision: 1.13; previous revision: 1.12
/cvsroot/CMake/CMake/Source/cmBuildCommand.cxx,v <-- Source/cmBuildCommand.cxx
new revision: 1.26; previous revision: 1.25
/cvsroot/CMake/CMake/Source/cmBuildCommand.h,v <-- Source/cmBuildCommand.h
new revision: 1.15; previous revision: 1.14
/cvsroot/CMake/CMake/Source/cmGlobalGenerator.h,v <-- Source/cmGlobalGenerator.h
new revision: 1.129; previous revision: 1.128
/cvsroot/CMake/CMake/Source/cmGlobalVisualStudioGenerator.h,v <-- Source/cmGlobalVisualStudioGenerator.h
new revision: 1.16; previous revision: 1.15
/cvsroot/CMake/CMake/Source/cmGlobalXCodeGenerator.cxx,v <-- Source/cmGlobalXCodeGenerator.cxx
new revision: 1.241; previous revision: 1.240
/cvsroot/CMake/CMake/Source/cmGlobalXCodeGenerator.h,v <-- Source/cmGlobalXCodeGenerator.h
new revision: 1.66; previous revision: 1.65
/cvsroot/CMake/CMake/Source/CTest/cmCTestBuildCommand.cxx,v <-- Source/CTest/cmCTestBuildCommand.cxx
new revision: 1.23; previous revision: 1.22
/cvsroot/CMake/CMake/Source/CTest/cmCTestBuildHandler.cxx,v <-- Source/CTest/cmCTestBuildHandler.cxx
new revision: 1.78; previous revision: 1.77
/cvsroot/CMake/CMake/Source/CTest/cmCTestBuildHandler.h,v <-- Source/CTest/cmCTestBuildHandler.h
new revision: 1.18; previous revision: 1.17
/cvsroot/CMake/CMake/Source/CTest/cmCTestConfigureCommand.cxx,v <-- Source/CTest/cmCTestConfigureCommand.cxx
new revision: 1.14; previous revision: 1.13
/cvsroot/CMake/CMake/Source/CTest/cmCTestHandlerCommand.cxx,v <-- Source/CTest/cmCTestHandlerCommand.cxx
new revision: 1.18; previous revision: 1.17
/cvsroot/CMake/CMake/Source/CTest/cmCTestTestCommand.cxx,v <-- Source/CTest/cmCTestTestCommand.cxx
new revision: 1.16; previous revision: 1.15
/cvsroot/CMake/CMake/Tests/CMakeLists.txt,v <-- Tests/CMakeLists.txt
new revision: 1.161; previous revision: 1.160
/cvsroot/CMake/CMake/Tests/CMakeCommands/build_command/CMakeLists.txt,v <-- Tests/CMakeCommands/build_command/CMakeLists.txt
initial revision: 1.1
/cvsroot/CMake/CMake/Tests/CMakeCommands/build_command/RunCMake.cmake,v <-- Tests/CMakeCommands/build_command/RunCMake.cmake
initial revision: 1.1
/cvsroot/CMake/CMake/Tests/CTestConfig/CMakeLists.txt,v <-- Tests/CTestConfig/CMakeLists.txt
initial revision: 1.1
/cvsroot/CMake/CMake/Tests/CTestConfig/CTestConfig.cxx,v <-- Tests/CTestConfig/CTestConfig.cxx
initial revision: 1.1
/cvsroot/CMake/CMake/Tests/CTestConfig/dashboard.cmake.in,v <-- Tests/CTestConfig/dashboard.cmake.in
initial revision: 1.1
/cvsroot/CMake/CMake/Tests/CTestConfig/script.cmake.in,v <-- Tests/CTestConfig/script.cmake.in
initial revision: 1.1
(0019506)
David Cole (manager)
2010-02-10 14:48

 Fix for the fix... in CVS HEAD of CMake:

/cvsroot/CMake/CMake/Source/CTest/cmCTestConfigureCommand.cxx,v <-- Source/CTest/cmCTestConfigureCommand.cxx
new revision: 1.16; previous revision: 1.15
(0024114)
David Cole (manager)
2010-12-14 18:50

 Closing bugs that have been resolved for more than 3 months without any further updates.

 Issue History
Date Modified Username Field Change
2007-08-31 11:28 Alex Neundorf Assigned To System Admin => David Cole
2008-11-12 15:00 David Cole Note Added: 0014093
2009-12-01 14:37 David Cole Priority low => high
2009-12-04 17:07 David Cole Note Added: 0018695
2009-12-04 17:07 David Cole Status assigned => resolved
2009-12-04 17:07 David Cole Resolution open => fixed
2010-02-10 14:47 David Cole Status resolved => feedback
2010-02-10 14:47 David Cole Resolution fixed => reopened
2010-02-10 14:48 David Cole Note Added: 0019506
2010-02-10 14:48 David Cole Status feedback => resolved
2010-02-10 14:48 David Cole Resolution reopened => fixed
2010-09-21 09:11 David Cole Relationship added related to 0011026
2010-12-14 18:50 David Cole Note Added: 0024114
2010-12-14 18:50 David Cole Status resolved => closed

Copyright © 2000 - 2018 MantisBT Team