Hi,<br><br>I think I have found the problem. It's in the structure of the Makefile generated by CMake. For each target, make is called on a sub Makefile and this is problematic for OCaml. I have reproduced the problem with an handwritten Makefile (see attachment).<br>
<br>Try make -j all2 and make -j all1. The first command will always work whereas the second, with recursive calls to make, will fail sometimes.<br><br>Recursive Makefiles with OCaml seem to be incompatible with parallel make. In OCaml, each object depends on others. If two targets depend on io.cmx and if make is launched in parallel on these two targets, the file io.cmx will be built twice. And if the first make command builds the object whereas the second tries to link the executable at the same time, the link command will fail because the object is being corrupted.<br>
<br>I think it's only a problem for OCaml. In C or C++, objects only depend on sources, and not on objects previously compiled.<br><br>Do you agree with this explanation ?<br><br>I see one solution : add a macro to create an only target with all OCaml libraries and executables, but it's not perfect.<br>
<br>Bye,<br><br>J. Bedouet<br><br><br><br><div class="gmail_quote">2010/1/22 Keyan <span dir="ltr"><<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
hi,<br>
<br>
i didnt have much time lately to look deeply into the scripts. at first i thought that one has to add some add_dependencies, which was my first strategy, before i wrote useocaml.cmake. but then i saw, that i removed them, because i got the DEPENDS right. so my guess was wrong.<br>
<br>
what you write seems plausible. i got the messages with your script when i compiled with make -j 2 while my computer was busy. for some reason i didnt get them with my scripts yet, which could just be by chance.<br>
<br>
i do not know how to proceed from here on.<br>
<br>
anyone got an idea?<br>
<br>
cheers,<br>
<font color="#888888">keyan<br>
</font><div><div></div><div class="h5"><br>
<br>
<br>
On 22 Jan 2010, at 02:16, Judicaël Bedouet wrote:<br>
<br>
> Hello,<br>
><br>
> I try to compare Makefiles generated by your scripts and mine. I don't see any big differences, except the copy rules.<br>
><br>
> While trying your scripts, I also reproduced a similar bug.<br>
><br>
> Generating file_node.cmx<br>
> Generating message_node.cmi<br>
> Generating file_node.cmi<br>
> File "/Users/judi/src/ocaml/cmake/<br>
> code-analysis/build/<a href="http://file_node.ml" target="_blank">file_node.ml</a>", line 1, characters 0-1:<br>
> Error: Corrupted compiled interface<br>
> file_node.cmi<br>
> make[2]: *** [file_node.cmx] Error 2<br>
><br>
> Try with<br>
> while true; do make clean; make -j; done<br>
> and you will reproduce it.<br>
><br>
> I think dependencies in our scripts are good (or almost good). The reason may be elsewhere.<br>
><br>
><br>
> I don't see a solution yet but I have found a promising lead. When two targets depends on the same object, make tries to build the object twice.<br>
><br>
> If make builds the same object twice at the same time, the command 'cmake -E copy_if_different' may be called at the same time and one of the cmake command fails because it can't open the file. I suppose it's the reason of<br>
><br>
> > Error copying file (if different) from "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake.tmp"<br>
> > to "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake".<br>
><br>
> In your code, the loc and messages executables depends both on io.cmo. Sometimes, it tries to link loc and build io.cmo (probably for messages) at the same time. I suppose it's the reason of<br>
><br>
><br>
> > Building OCaml object io.cmo<br>
> > Building OCaml object files.cmo<br>
> > [ 0%] Linking OCaml executable loc<br>
> > File "_none_", line 1, characters 0-1:<br>
> > Error: The file /Users/judi/src/ocaml/cmake/code-analysis/build/io.cmo is not a bytecode object file<br>
><br>
><br>
> I add a third target which depends on io.cmo (create a common library and make loc and messages depend on this library). It appears that make tries to build the object io.cmo three times and build fails are almost systematic.<br>
><br>
><br>
> If you keep only one target, there is never no problems.<br>
><br>
><br>
> An idea ? Too late in the night for me :-(<br>
><br>
><br>
> Bye,<br>
><br>
> J. Bedouet<br>
><br>
><br>
><br>
><br>
> 2010/1/20 Keyan <<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>><br>
> hi,<br>
><br>
> i started to work on your scripts. very nice work!<br>
><br>
> unfortunately, i couldnt find the problem, and i cant reproduce it with my scripts. there must be a difference somewhere.<br>
><br>
> cheers,<br>
> keyan<br>
><br>
><br>
><br>
> On 19 Jan 2010, at 00:30, Judicaël Bedouet wrote:<br>
><br>
> > Hi,<br>
> ><br>
> > Effectively, I have reproduced your two bugs by using make -j. It's not systematic (once in ten).<br>
> ><br>
> > >> Error: The file /Users/zahedi/Downloads/code-analysis-build/io.cmo is not a bytecode object file<br>
> > When this error occurs, it also appears that the object is being built. And if I rerun make, linking is correct, without rebuilding io.cmo.<br>
> ><br>
> > Building OCaml object io.cmo<br>
> > Building OCaml object files.cmo<br>
> > [ 0%] Linking OCaml executable loc<br>
> > File "_none_", line 1, characters 0-1:<br>
> > Error: The file /Users/judi/src/ocaml/cmake/code-analysis/build/io.cmo is not a bytecode object file<br>
> ><br>
> > Dependencies in the generated Makefile seem to be correct :<br>
> > loc: file_node.cmo<br>
> > loc: files.cmo<br>
> > loc: io.cmo<br>
> > loc: file_statistics.cmo<br>
> > loc: message_node.cmo<br>
> > loc: loc.cmo<br>
> ><br>
> > Is OCaml parallel make safe ? I'm not sure... I readd CMake address : maybe someone has an idea...<br>
> ><br>
> > To speed up compilation, I could look for ocamlc.opt and if not found ocamlc. Bytecode compilations should be faster.<br>
> ><br>
> > J. Bedouet<br>
> ><br>
> ><br>
> ><br>
> > 2010/1/18 Keyan <<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>><br>
> > hi,<br>
> ><br>
> > i am just trying to reproduce the message i posted. i cant, but i got something else:<br>
> ><br>
> > Error: The file /Users/zahedi/Downloads/code-analysis-build/io.cmo is not a bytecode object file<br>
> ><br>
> > i use make -j 2 (at least 2), and my guess is that it has to do with parallel compilation. could that be the case?<br>
> ><br>
> > cheers,<br>
> > keyan<br>
> ><br>
> > On 18 Jan 2010, at 18:01, Judicaël Bedouet wrote:<br>
> ><br>
> > > Hi,<br>
> > ><br>
> > > >> first of all, wow, good job! very nice scripts, much more sophisticated then mine :)<br>
> > > Thanks :-)<br>
> > ><br>
> > > >> I only had one error when first trying: Error copying file (if different) from "[...]/code-analysis-build/Dependencies<br>
> > > >> /message_node.ml.dep.cmake.tmp" to "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake".<br>
> > > Maybe the Dependencies directory is not created. It is done at line 32 of OCamlDep, though. Let me know if you find something.<br>
> > ><br>
> > > I have tested my scripts with OCaml 3.10 and I have found that ocamlfind is officially distributed with OCaml since 3.11. Maybe, I should put ocamlfind stuff in a FindOCamlFind script...<br>
> > ><br>
> > > J. Bedouet<br>
> > ><br>
> > ><br>
> > ><br>
> > > 2010/1/18 Keyan <<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>><br>
> > > Hi,<br>
> > ><br>
> > > i am sorry that i could not reply any sooner.<br>
> > ><br>
> > > first of all, wow, good job! very nice scripts, much more sophisticated then mine :)<br>
> > ><br>
> > > i only had one error when first trying:<br>
> > ><br>
> > > Error copying file (if different) from "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake.tmp" to "[...]/code-analysis-build/Dependencies/message_node.ml.dep.cmake".<br>
> > ><br>
> > > i will try to solve that myself. this way i know if i understand your scripts.<br>
> > ><br>
> > > cheers,<br>
> > > keyan<br>
> > ><br>
> > > On 16 Jan 2010, at 14:51, Judicaël Bedouet wrote:<br>
> > ><br>
> > > > Hi,<br>
> > > ><br>
> > > > Here are my OCaml scripts, with your OCaml code (it helped me to find a bug about a source file included in two targets).<br>
> > > ><br>
> > > > I have kept your idea about an only macro to compile and link, reused your macro ocaml_parse_arguments (I should use for my other macros :-) ) and added a macro find_ocaml_package to find ocaml packages with ocamlfind.<br>
> > > ><br>
> > > > Tell me what you think,<br>
> > > ><br>
> > > > J. Bedouet<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > > 2010/1/15 Keyan <<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>><br>
> > > ><br>
> > > > hi,<br>
> > > ><br>
> > > > > Indeed, it is much more sophisticated, so it may be more bugged ;-)<br>
> > > ><br>
> > > > i can help debugging :)<br>
> > > ><br>
> > > > > >> shell i send you my scripts to take a look?<br>
> > > > > It's not necessary. I have downloaded the refactoring branch of yars. Thanks.<br>
> > > ><br>
> > > > oh, ok, then you should have the newest version (last yars revision is 513).<br>
> > > ><br>
> > > > > I comment my OCaml scripts and I send you, so that you could also have a look.<br>
> > > ><br>
> > > > great.<br>
> > > ><br>
> > > > > I have took a look at your UseOCaml script. I can't see why you copy ML files into the current binary dir before compiling them. Is there a reason ?<br>
> > > ><br>
> > > > no. i copied most of the stuff from the UseLATEX.cmake. it was done so there, and at the time i wrote the script, i thought its a good idea. don't ask me why :)<br>
> > > ><br>
> > > > > Keep in touch,<br>
> > > ><br>
> > > > let me know when i can jump in and help. looking forward.<br>
> > > ><br>
> > > > cheers,<br>
> > > > keyan<br>
> > > ><br>
> > > > > J. Bedouet<br>
> > > > ><br>
> > > > ><br>
> > > > > On Fri, Jan 15, 2010 at 8:26 AM, Keyan <<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>> wrote:<br>
> > > > > I totally agree.<br>
> > > > ><br>
> > > > > Judicaël: how should we proceed? my first impression is that your usecaml.cmake is much more sophisticated then mine :) same for the findocaml.cmake scripts, as mine up to now does nothing more than to call a bunch of find_program commands.<br>
> > > > ><br>
> > > > > shell i send you my scripts to take a look?<br>
> > > > ><br>
> > > > > i have no problem with merging my stuff into your scripts, keeping you as main author, if there is anything useful for you.<br>
> > > > ><br>
> > > > > cheers,<br>
> > > > > keyan<br>
> > > > ><br>
> > > > ><br>
> > > > > On 15 Jan 2010, at 02:39, Philip Lowman wrote:<br>
> > > > ><br>
> > > > > > On Thu, Jan 14, 2010 at 5:11 AM, Keyan <<a href="mailto:ml@pulsschlag.net">ml@pulsschlag.net</a>> wrote:<br>
> > > > > >> hi,<br>
> > > > > >><br>
> > > > > >> ups ... i thought i replied to the list. thanks for the link. i will adapt my find-script to the requirements and maintain the package.<br>
> > > > > ><br>
> > > > > > You should work with Judicaël to combine the best of both of your<br>
> > > > > > works together into one FindOCaml and UseOCaml script. There is no<br>
> > > > > > sense having two of them out there.<br>
> > > > > ><br>
> > > > > > --<br>
> > > > > > Philip Lowman<br>
> > > > ><br>
> > > > ><br>
> > > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > ><br>
> > ><br>
> > ><br>
> ><br>
> ><br>
><br>
><br>
><br>
<br>
<br>
</div></div></blockquote></div><br>