View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0009450CMakeCTestpublic2009-08-25 16:092009-11-24 09:00
ReporterBrad King 
Assigned ToBrad King 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOSOS Version
Product VersionCMake-2-6 
Target VersionFixed in Version 
Summary0009450: CTest delayed-initialization mode is broken
DescriptionWhen a ctest dashboard script sets CTEST_CHECKOUT_COMMAND the goal is to start without a source directory. The ctest_start() command cannot load the CTestConfig.cmake file (which has the submission site) because it doesn't exist yet. Instead the intention is that ctest_start delays initialization until ctest_update(). After ctest_update() performs the initial checkout it performs initialization. However, this does not seem to work, and the submission site is empty.
Additional InformationThe cmCTest::InitializeFromCommand method is central to this problem. I think when ctest_update() calls it the code

  if ( !first && !this->CurrentTag.empty() )
    {
    return true;
    }

skips the main body of the function because a tag is already available.

This is broken in CMake 2.6 and 2.7.20090825 from CVS. The work-around is to duplicate the CTestConfig.cmake information in the dashboard script, but this really should work.

A fix may be to remove the code mentioned above, but CTest has so many initialization paths I'm not sure what effects that would cause. Someone needs to trace all paths and clean/combine them to improve robustness.
TagsNo tags attached.
Attached Files

 Relationships

  Notes
(0018447)
Maik Beckmann (reporter)
2009-11-16 13:58

 I've hit this problem with our testing setup for falcon.

$ svn co svn://falconpl.org/falcon/tools/trunk/testing [^]
A testing/ctest_driver.cmake.in
A testing/ctest_driver_details.cmake
A testing/README
Ausgecheckt, Revision 178.
$ cd testing/
$ cp ctest_driver.cmake.in ctest_driver.cmake
$ gvim ctest_driver.cmake # remove the first colum of comment symbols
$ ctest -VV -S ctest_driver.cmake
results in (complete log http://pastebin.ca/1673794 [^])
{{{
Cannot instantiate test handler ctest_build
CMake Error at ctest_driver_details.cmake:56 (ctest_build):
  ctest_build CTEST_BUILD_COMMAND or CTEST_CMAKE_GENERATOR not specified.
  Please specify the CTEST_CMAKE_GENERATOR and CTEST_PROJECT_NAME if this is
  a CMake project, or specify the CTEST_BUILD_COMMAND for cmake or any other
  project.
}}}

CTEST_PROJECT_NAME along with the CTEST_DROP_LOCATION etc. are provided by a CTestConfig.cmake in each of the tested source trees. That the projects are set up fine and just fails on the initial checkout is proven by running
  $ ctest -VV -S ctest_driver.cmake
again, since everything works fine now.
(0018448)
Brad King (manager)
2009-11-16 14:01

 Further analysis reveals that CurrentTag gets set on ctest_start() when the test mode is Experimental or Continuous (not Nightly). The reason is that those two modes do not need the nightly start time to compute a tag. The start time is stored in CTestConfig.cmake so the Nightly tag needs it.
(0018449)
Maik Beckmann (reporter)
2009-11-16 14:06

 CMake versions I tested
  - 2.6-4
  - 2.8-0
same result for both.
(0018450)
Brad King (manager)
2009-11-16 14:17

 Maik, your driver sets

  set(FALCON_CTEST_TYPE "Experimental")

As I mentioned in my previous comment, the initial-checkout/delayed-initialization currently works only for Nightly mode. This is a bug in CTest.
(0018452)
Maik Beckmann (reporter)
2009-11-16 14:31
edited on: 2009-11-16 14:38

 I got this, nice analysis!

Until its resolved, I'd stay with the workaround you find in ctest_driver_details.cmake, which sends an error message to the user after the initial run of ctest, which ask to just rerun it.

For "Nightly" I might skip this, since it works as expected.
(0018553)
Brad King (manager)
2009-11-23 17:12

 I realized that much of the complication in CTest's initialization code is needed to support delaying full init until ctest_update(). However, the code path in ctest_update() to do the initial checkout of the source tree is totally separate from the main update step.

A great simplification here is to move the initial checkout into ctest_start(). That way the source tree always exists during the first initialization pass, and much of the complication can be removed. I prototyped this change locally and it works well. I'll clean it up and commit when I get a chance.
(0018563)
Brad King (manager)
2009-11-24 08:59

 CTest: Simplify Initialize method signature
/cvsroot/CMake/CMake/Source/cmCTest.cxx,v <-- Source/cmCTest.cxx
new revision: 1.378; previous revision: 1.377
/cvsroot/CMake/CMake/Source/cmCTest.h,v <-- Source/cmCTest.h
new revision: 1.124; previous revision: 1.123

CTest: Move initial checkout to ctest_start()
/cvsroot/CMake/CMake/Source/CTest/cmCTestStartCommand.cxx,v <-- Source/CTest/cmCTestStartCommand.cxx
new revision: 1.18; previous revision: 1.17
/cvsroot/CMake/CMake/Source/CTest/cmCTestStartCommand.h,v <-- Source/CTest/cmCTestStartCommand.h
new revision: 1.8; previous revision: 1.7
/cvsroot/CMake/CMake/Source/CTest/cmCTestUpdateCommand.cxx,v <-- Source/CTest/cmCTestUpdateCommand.cxx
new revision: 1.20; previous revision: 1.19
/cvsroot/CMake/CMake/Source/CTest/cmCTestUpdateHandler.cxx,v <-- Source/CTest/cmCTestUpdateHandler.cxx
new revision: 1.69; previous revision: 1.68
/cvsroot/CMake/CMake/Source/CTest/cmCTestUpdateHandler.h,v <-- Source/CTest/cmCTestUpdateHandler.h
new revision: 1.16; previous revision: 1.15
/cvsroot/CMake/CMake/Source/cmCTest.cxx,v <-- Source/cmCTest.cxx
new revision: 1.379; previous revision: 1.378
/cvsroot/CMake/CMake/Source/cmCTest.h,v <-- Source/cmCTest.h
new revision: 1.125; previous revision: 1.124

 Issue History
Date Modified Username Field Change
2009-08-25 16:09 Brad King New Issue
2009-08-27 09:18 Brad King Status new => assigned
2009-08-27 09:18 Brad King Assigned To => Zach Mullen
2009-11-16 13:58 Maik Beckmann Note Added: 0018447
2009-11-16 14:01 Brad King Note Added: 0018448
2009-11-16 14:06 Maik Beckmann Note Added: 0018449
2009-11-16 14:17 Brad King Note Added: 0018450
2009-11-16 14:18 Brad King Assigned To Zach Mullen => Brad King
2009-11-16 14:31 Maik Beckmann Note Added: 0018452
2009-11-16 14:38 Maik Beckmann Note Edited: 0018452
2009-11-23 17:12 Brad King Note Added: 0018553
2009-11-24 08:59 Brad King Note Added: 0018563
2009-11-24 09:00 Brad King Status assigned => closed
2009-11-24 09:00 Brad King Resolution open => fixed

Copyright © 2000 - 2018 MantisBT Team