<br><br><div class="gmail_quote">On Tue, Sep 13, 2011 at 1:58 PM, David Cole <span dir="ltr"><<a href="mailto:david.cole@kitware.com">david.cole@kitware.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div><div></div><div class="h5">On Tue, Sep 13, 2011 at 1:39 PM, Alexander Neundorf<br>
<<a href="mailto:a.neundorf-work@gmx.net">a.neundorf-work@gmx.net</a>> wrote:<br>
> On Tuesday, September 13, 2011 05:07:00 AM Clifford Yapp wrote:<br>
>> I am trying to compare two large lists of file paths (about 14,000 lines<br>
>> each) to identify which entries in each list are missing from the other,<br>
>> and while I can get CMake to do it I must be doing it the wrong way<br>
>> because the results are hideously slow.<br>
>><br>
>> I currently generate two files with the paths and then read them in as<br>
>> lists, using LIST() commands to peform STREQUAL tests. I was hoping to<br>
><br>
> How do you do that ?<br>
> Do you iterate over one list using foreach() and then list(FIND) to check<br>
> whether it exists in the other list ?<br></div></div></blockquote><div><br>I tried a couple of ways, most of them variations on that theme (BUILD_FILES and SVN_FILES are two manifest lists, and I need items from each list that are not in the other list)<br>
<br>FOREACH(ITEM ${BUILD_FILES})<br> LIST(FIND ${ITEM} SVN_FILES POS)<br> IF(NOT POS STREQUAL "-1")<br> LIST(REMOVE_ITEM SVN_FILES ${ITEM})<br> LIST(REMOVE_ITEM BUILD_FILES ${ITEM})<br> ENDIF()<br>
ENDFOREACH()<br><br>In essence, the idea is BUILD_FILES will end up holding items unique to BUILD_FILES and SVN_FILES will end up holding items unique to SVN_FILES, which are the two pieces of information I'm after.<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div><div class="h5">
> Internally, every cmake variable is stored as a plain std::string.<br>
> When using a list() command, this string is converted to a<br>
> std::vector<std::string>, and then cmake operates on this vector.<br>
> So if you do this 14000 times, each time a 14000 std::strings are created,<br>
> which makes this O(n^2) I think.<br>
><br>
> Maybe something like this works ?<br>
><br>
> set(uniqueItems ${list1} ${list2})<br>
> list(REMOVE_DUPLICATES uniqueItems )<br></div></div></blockquote><div><br>I thought about REMOVE_DUPLICATES, but if I understand correctly wouldn't that give me the union of the two lists? I need everything not present in both lists. <br>
</div><div> <br></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
I realize this same technique will not be very useful with files<br>
14,000 lines long... but thought I'd mention it so you could look at<br>
it and perhaps draw inspiration from it.<br></blockquote><div><br>Basically I'm currently trying a variation on that, except I'm trying to get a cross-platform comparing tool working that I can guarantee will be there - I'm trying comm at the moment but I'm not sure comm's notions of what consistute a sorted file and CMake's LIST(SORT are compatible when it comes to things like .file and ~file.<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
If you could pass along the code that you're using, we might be able<br>
to suggest a better way to achieve the same thing within the CMake<br>
language if that's absolutely necessary.<br></blockquote></div><br>Currently (using comm) I'm doing this (replacing /. with /tmpdot in an attempt to let comm handle the sorted output):<br><br>STRING(REGEX REPLACE "/\\." "/tmpdot" BUILD_FILES "${BUILD_FILES}")<br>
STRING(REGEX REPLACE "/\\." "/tmpdot" SVN_FILES "${SVN_FILES}")<br>LIST(SORT BUILD_FILES)<br>LIST(SORT SVN_FILES)<br>STRING(REGEX REPLACE ";" "\n" BUILD_FILES "${BUILD_FILES}")<br>
STRING(REGEX REPLACE ";" "\n" SVN_FILES "${SVN_FILES}")<br>FILE(WRITE @CMAKE_BINARY_DIR@/build_files_list.txt ${BUILD_FILES})<br>FILE(WRITE @CMAKE_BINARY_DIR@/svn_files_list.txt ${SVN_FILES})<br>
STRING(REGEX REPLACE "\n" ";" BUILD_FILES "${BUILD_FILES}")<br>STRING(REGEX REPLACE "\n" ";" SVN_FILES "${SVN_FILES}")<br><br>EXECUTE_PROCESS(COMMAND @CMAKE_BINARY_DIR@/@BIN_DIR@/comm -3 @CMAKE_BINARY_DIR@/build_files_list.txt @CMAKE_BINARY_DIR@/svn_files_list.txt OUTPUT_VARIABLE COMM_RAWOUT)<br>
STRING(REGEX REPLACE "\n" ";" COMM_OUT "${COMM_RAWOUT}")<br>STRING(REGEX REPLACE "/tmpdot" "/\\\\." COMM_OUT "${COMM_OUT}")<br>STRING(REGEX REPLACE "\\\\" "" COMM_OUT "${COMM_OUT}")<br>
<br>FOREACH(ITEM ${COMM_OUT})<br> LIST(FIND BUILD_FILES ${ITEM} INBUILD)<br> LIST(FIND SVN_FILES ${ITEM} INSVN)<br> IF(INBUILD STREQUAL "-1" AND NOT INSVN STREQUAL "-1")<br> LIST(APPEND SVN_FILES_NOT_IN_BUILD ${ITEM})<br>
LIST(REMOVE_ITEM COMM_OUT ${ITEM})<br> ENDIF(INBUILD STREQUAL "-1" AND NOT INSVN STREQUAL "-1")<br> IF(INSVN STREQUAL "-1" AND NOT INBUILD STREQUAL "-1")<br> LIST(APPEND BUILD_FILES_NOT_IN_SVN ${ITEM})<br>
LIST(REMOVE_ITEM COMM_OUT ${ITEM})<br> ENDIF(INSVN STREQUAL "-1" AND NOT INBUILD STREQUAL "-1")<br>ENDFOREACH(ITEM ${COMM_OUT})<br><br>I was hoping that CMake's diff ability might indicate lurking in there was the ability to get actual diff style output from a cmake -E command that could be parsed (if CMake had sucked in openbsd's diff to implement its diff abilities, for example) but if not it looks like performance considerations will require ensuring some sort of cross-platform tool is available. Would it perhaps make sense to have a cmake -E diff the same way there is a cmake -E tar?<br>
<br>The broader context is implementing a "make distcheck" rule for BRL-CAD along the lines of the one previously implemented in autotools - the idea is to have subversion tell us what files are in the repository (svn info), have the build system report what files it knows about (via some custom CMake function/macro logic that we have working to record that) and crank that information back into user reporting and cpack (for example, a file known to subversion but not addressed in the build logic is a FATAL_ERROR, and a file unknown to both subversion and the build logic will be ignored when running CPack). This ability has proven very useful over the years with our autotools build for making tarball distributions (there are more steps to distcheck but this looks to be the last really tricky one to do in CMake).<br>
<br>Cheers, and thanks everybody for the help!<br><br>CY<br>