1 2 Previous Next 19 Replies Latest reply on Jun 16, 2010 11:27 AM by 807575 Go to original post
      • 15. Re: ir2hf out of memory
        807575
        Interestingly I have found in my buildlogs more occurrences of this:
        ir2hf: error: Ran out of memory
        Linking CXX shared library ../../lib/Slicer3/Plugins/libRigidRegistrationLib.so
        ld: fatal: file CMakeFiles/RigidRegistrationLib.dir/RigidRegistration.cxx.o: open failed: No such file or directory
        and
        [ 23%] Building CXX object Applications/CLI/CMakeFiles/TestGridTransformRegistrationLib.dir/TestGridTransformRegistration.cxx.o
        ir2hf: error: Ran out of memory
        gmake[6]: *** [Applications/CLI/CMakeFiles/RigidRegistrationLib.dir/RigidRegistration.cxx.o] Error 154
        but I could not reproduce it.

        This might have been caused by the fact that - due to the introduction of Qt into Slicer, and Qt is supported with stdcxx4 - I had to transition the builds from stlport4 to stdcxx4. So now with stdcxx4, this very same file compiles fine with Studio12. And yes, the problems arise with S12.1, and 1006 express.
        (I have even found this beauty, two such errors right after each other.)
        If needed, I can put together a test case with stlport4 & SS12, that should be no problem.
        Yes, with Studio Express I get the libsunir error with rigidregistration.cxx.
        This ir2hf error seems to be dependent on the used STL - might it be that different implementations of STLs eat up different amount of memory, and this is why ir2hf breaks with different files? Really just guessing! :)

        And yes, Studio Express generates bogus object files too.

        To be honest I gave up with Studio 12 after a while, and transitioned to SS12.1. So I have more detailed "case studies" with that compiler. For example: when I met the ir2hf error with S12, I always gave up the whole compilation. After I began to use SS12.1, I just discovered that I can continue the compilation by removing the -xOn flags, so I met more of these cases. So I had only one of these with SS12 & stlport4, while there are at least 6-7 with SS12.1 & stdcxx4.

        Thanks for the instructions, I think I can post the results here (links) today!

        And thanks all for looking into this!
        Attila
        • 16. Re: ir2hf out of memory
          807575
          Just a quick note (will add some info a bit later)

          Done, but enormous amount of data has been generated:
          # du -hs ./*                        
           1,9G     ./corrupted_ss12_1
           759M     ./corrupted_ss12_stdcxx
           1,1G     ./corrupted_ss12_stlport
           1,9G     ./corrupted_ssx
           1,6G     ./ir2hf_ss12_1
           1,0G     ./ir2hf_ss12_stdcxx
           1,4G     ./ir2hf_ss12_stlport
           649M     ./ir2hf_ssx
          
          # du -hs ./                         
            10G     .
          That is:
          (subdirectories:)
          1,0K jún. 10 15:48 corrupted_ss12_stdcxx
          1,0K jún. 10 15:50 corrupted_ss12_stlport
          193K jún. 10 15:51 corrupted_ssx
           512 jún. 10 16:24 ir2hf_ss12_1
          1,0K jún. 10 15:39 ir2hf_ss12_stdcxx
           512 jún. 10 16:28 ir2hf_ss12_stlport
          193K jún. 10 15:55 ir2hf_ssx
             0 jún. 10 18:02 ls.txt
          
          ./corrupted_ss12_1:
          total 3919122
             0 jún. 10 15:36 ccfe.04853.0.err
          457M jún. 10 16:05 ccfe.05028.0.ir
          9,3M jún. 10 16:07 ccfe.05028.1.s
             0 jún. 10 16:03 ccfe.05028.2.err
          766M jún. 10 16:07 ir2hf.05028.5.o
          418M jún. 10 16:07 iropt.05028.3.ir
             0 jún. 10 16:05 iropt.05028.4.err
           37M jún. 10 16:18 RegisterImagesSlicer_ss12_1_stdcxx_corrupted.cxx.o
           24K jún. 10 15:36 ss12_1_corrupted_stdcxx-preprocess.log
           25K jún. 10 16:17 ss12_1_corrupted_stdcxx.log
          7,9K jún. 10 15:37 ss12_1-regimagesslicer-stdcxx-keepfiles-preprocess.txt
          8,0K jún. 10 13:17 ss12_1-regimagesslicer-stdcxx-keepfiles.txt
          226M jún. 10 16:17 ube.05028.6.s
          
          ./corrupted_ss12_stdcxx:
          total 1554946
          270M jún. 10 12:53 ccfe.04618.0.ir
          7,3M jún. 10 12:53 ccfe.04618.1.s
             0 jún. 10 12:52 ccfe.04618.2.err
             0 jún. 10 15:48 ccfe.04947.0.err
          192M jún. 10 12:54 ir2hf.04618.5.o
          262M jún. 10 12:54 iropt.04618.3.ir
             0 jún. 10 12:53 iropt.04618.4.err
           25M jún. 10 13:00 RegisterImagesSlicer_ss12_stdcxx_corrupted.cxx.o
          2,5M jún. 10 15:48 RegisterImagesSlicer.i
           23K jún. 10 15:48 ss12_corrupted_stdcxx-preprocessed.log
           25K jún. 10 13:00 ss12_corrupted_stdcxx.log
          7,9K jún. 10 15:44 ss12-regimagesslicer-stdcxx-keepfiles-preprocess.txt
          8,0K jún. 10 12:36 ss12-regimagesslicer-stdcxx-keepfiles.txt
          
          ./corrupted_ss12_stlport:
          total 2263874
          384M jún. 10 12:42 ccfe.04605.0.ir
          7,3M jún. 10 12:42 ccfe.04605.1.s
             0 jún. 10 12:40 ccfe.04605.2.err
             0 jún. 10 15:49 ccfe.04957.0.err
          295M jún. 10 12:43 ir2hf.04605.5.o
          384M jún. 10 12:43 iropt.04605.3.ir
             0 jún. 10 12:42 iropt.04605.4.err
           32M jún. 10 12:52 RegisterImagesSlicer_ss12_stlport_corrupted.cxx.o
          2,4M jún. 10 15:49 RegisterImagesSlicer.i
           23K jún. 10 15:49 ss12_corrupted_stlport-preprocessed.log
           25K jún. 10 12:52 ss12_corrupted_stlport.log
          7,8K jún. 10 15:49 ss12-regimagesslicer-stlport-keepfiles-preprocessed.txt
          7,9K jún. 10 12:36 ss12-regimagesslicer-stlport-keepfiles.txt
          
          ./corrupted_ssx:
          total 3921286
          468M jún. 10 13:17 ccfe.1276168515.4639.01.ir
          9,3M jún. 10 13:19 ccfe.1276168515.4639.02.sd
             0 jún. 10 13:15 ccfe.1276168515.4639.03.err
             0 jún. 10 15:50 ccfe.1276177856.4967.01.err
          728M jún. 10 13:20 ir2hf.1276168515.4639.07.hf
          421M jún. 10 13:19 iropt.1276168515.4639.05.ir
             0 jún. 10 13:17 iropt.1276168515.4639.06.err
           37M jún. 10 13:30 RegisterImagesSlicer_ssx_stdcxx_corrupted.cxx.o
          2,5M jún. 10 15:50 RegisterImagesSlicer.i
           24K jún. 10 15:50 ssx_corrupted_stdcxx-preprocessed.log
           26K jún. 10 13:30 ssx_corrupted_stdcxx.log
          7,9K jún. 10 15:50 ssx-regimagesslicer-stdcxx-keepfiles-preprocessed.txt
          8,0K jún. 10 12:36 ssx-regimagesslicer-stdcxx-keepfiles.txt
          9,0K jún. 10 13:30 ube.1276168515.4639.08.dbg
          247M jún. 10 13:30 ube.1276168515.4639.09.s
          
          ./ir2hf_ss12_1:
          total 3253936
             0 jún. 10 15:37 ccfe.04867.0.err
          799M jún. 10 16:22 ccfe.05037.0.ir
          8,8M jún. 10 16:24 ccfe.05037.1.s
             0 jún. 10 16:18 ccfe.05037.2.err
           23K jún. 10 15:37 ir2hf_ss12_1_stdcxx-preprocess.log
           24K jún. 10 16:25 ir2hf_ss12_1_stdcxx.log
          778M jún. 10 16:24 iropt.05037.3.ir
             0 jún. 10 16:22 iropt.05037.4.err
          2,1M jún. 10 15:34 RigidRegistration.i
          7,6K jún. 10 15:37 ss12_1-rigidreglib-stdcxx-keepfiles-preprocess.txt
          7,7K jún. 10 14:12 ss12_1-rigidreglib-stdcxx-keepfiles.txt
          
          ./ir2hf_ss12_stdcxx:
          total 2200096
          362M jún. 10 13:37 ccfe.04674.0.ir
          7,2M jún. 10 13:37 ccfe.04674.1.s
             0 jún. 10 13:36 ccfe.04674.2.err
           23K jún. 10 15:22 ir2hf_ss12_stdcxx-preprocess.log
           24K jún. 10 13:52 ir2hf_ss12_stdcxx.log
          294M jún. 10 13:38 ir2hf.04674.5.o
          376M jún. 10 13:38 iropt.04674.3.ir
             0 jún. 10 13:37 iropt.04674.4.err
           32M jún. 10 13:52 RigidRegistration_ir2hf_ss12_stdcxx.cxx.o
          2,1M jún. 10 14:50 RigidRegistration.i
          7,7K jún. 10 15:38 ss12-rigidreglib-stdcxx-keepfiles-preprocess.txt
          7,7K jún. 10 14:53 ss12-rigidreglib-stdcxx-keepfiles-preprocess.txt~
          7,7K jún. 10 13:53 ss12-rigidreglib-stdcxx-keepfiles.txt
          
          ./ir2hf_ss12_stlport:
          total 2872862
          679M jún. 10 16:28 ccfe.05042.0.ir
          7,2M jún. 10 16:28 ccfe.05042.1.s
             0 jún. 10 16:25 ccfe.05042.2.err
           23K jún. 10 15:31 ir2hf_ss12_stlport-preprocess.log
           24K jún. 10 16:31 ir2hf_ss12_stlport.log
          714M jún. 10 16:30 iropt.05042.3.ir
             0 jún. 10 16:28 iropt.05042.4.err
          2,0M jún. 10 15:32 RigidRegistration.i
          7,5K jún. 10 15:38 ss12-rigidreglib-stlport-keepfiles-preprocess
          7,6K jún. 10 12:36 ss12-rigidreglib-stlport-keepfiles.txt
          
          ./ir2hf_ssx:
          total 1329058
          646M jún. 10 14:00 ccfe.1276170995.4728.01.ir
            30 jún. 10 14:00 ccfe.1276170995.4728.03.err
             0 jún. 10 15:55 ccfe.1276178112.5002.01.err
           23K jún. 10 15:55 ir2hf_ssx_stdcxx-preprocess.log
           24K jún. 10 14:00 ir2hf_ssx_stdcxx.log
          2,1M jún. 10 15:55 RigidRegistration.i
          7,6K jún. 10 15:53 ssx-rigidreglib-stdcxx-keepfiles-preprocess.txt
          7,7K jún. 10 12:36 ssx-rigidreglib-stdcxx-keepfiles.txt
          A recipe for download:
          www.orl.szote.u-szeged.hu/~attila/ssbughunt/ < subdir name from above>/< filename from above >
          (directory browsing is disabled)

          Happy browsing! :)

          Thanks for the help,
          Attila
          • 17. Re: ir2hf out of memory
            807575
            Some more infos:

            with Studio 12 I did tests with both stlport4 and stdcxx, just to be sure that we don't miss errors. With Studio 12.1 and StudioExpress 1006 I only used stdcxx, since I/we only use this STL implementation, and will continue to do so in the future (Qt!).
            The files:
            - foo *.log* files contain - logs... :) I saved the preprocessed files too (thanks to Stefan Teleman for the hint!!! :)), so the foo-preprocessed.log files contain the logs for those operations.
            - foo *.txt* files contain the actual command lines I used during the work (I admit that the .txt extension may be misleading, but we're on unix, and I simply chmod'ed them to be executable :)).
            - foo *.i* files are of course the preprocessed sources.
            All the other files are the files that were generated during the builds.

            Summaries:+*

            Corrupted object files:
            - I could not+ generate corrupted object files with neither Studio 12 & stlport4 nor Studio12 & stdcxx4. ( corrupted_ss12_stdcxx and corrupted_ss12_stlport directories)
            (see this thread for some info, but we didn't dig this deep back then: [C++ forum thread: [...] has invalid sh_link: 42164|http://forums.sun.com/thread.jspa?threadID=5413456] )
            - I could* generate corrupted object files with both Studio 12.1 & stdcxx4 and Studio express & stdcxx4. (directories corrupted_ss12_1 and corrupted_ssx)

            ir2hf & libsunir errors:
            - ir2hf ran out of memory: Studio 12 & stlport4 (see the log file " [ir2hf_ss12_stlport/ir2hf_ss12_stlport.log|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/ir2hf_ss12_stlport/ir2hf_ss12_stlport.log] ")
            - NO+ ir2hf error with Studio 12 & stdcxx4 (see the log file " [ir2hf_ss12_stdcxx/ir2hf_ss12_stdcxx.log|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/ir2hf_ss12_stdcxx/ir2hf_ss12_stdcxx.log] ")
            - ir2hf ran out of memory (with a detailed output) with Studio 12.1 & stdcxx4 (log file here: "[ir2hf_ss12_1/ir2hf_ss12_1_stdcxx.log|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/ir2hf_ss12_1/ir2hf_ss12_1_stdcxx.log] ")
            - libsunir error with Studio Express 1006 & stdcxx4 (see log here: "[ir2hf_ssx/ir2hf_ssx_stdcxx.log|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/ir2hf_ssx/ir2hf_ssx_stdcxx.log] ")

            I think that's it for now.

            Let me know if you need anything else!

            (To the developers: should I update the bug per e-mail? I think I should, but what and how should I include?)

            One more thing: as 10GB is pretty much data, I'd be glad if you could let me know what files do you need (or download it), so that I can remove at least part of the files from our server - we're pretty low on space... :)) - it's not a problem for a good few days or maybe a few weeks (it's summer, so no big amounts of data is generated), so it's not that urgent.

            Thanks you very much,
            Attila

            ps.: One more thing I forgot to add earlier, and this may be important.
            I discovered in my own notes (I always make notes and save most of the build logs), that RigidRegistration.cxx (I worked with that file, by the way) WITH -xO3 gives the known ir2hf out of mem error, while it compiles with lower optimization levels, only to make a broken object file. So with -xO2 ... -xO0 ldd/ld "complains". This is the only file I observed this behavior with, but I might have left out a few "experiments" int this regard.

            Edited by: -aspirin- on Jun 11, 2010 3:02 PM
            • 18. Re: ir2hf out of memory
              807575
              >
              (To the developers: should I update the bug per e-mail? I think I should, but what and how should I include?)
              I put link to this thread in bug comments. That should be enough
              One more thing: as 10GB is pretty much data, I'd be glad if you could let me know what files do you need (or download it), so that I can remove at
              least part of the files from our server - we're pretty low on space... :)) - it's not a problem for a good few days or maybe a few weeks (it's summer, so no
              big amounts of data is generated), so it's not that urgent.
              Well, you can remove all ss12 stuff.
              It either works OK, or has same issues as more recent releases, so it's not that interesting :-)
              I could get distracted by other things, so could you keep

              corrupted_ssx
              ir2hf_ss12_1
              ir2hf_ssx

              directories for some time? It may be easier to tar and (b)zip them into single files.
              • 19. Re: ir2hf out of memory
                807575
                No problem on my side, I can remove SS12-related stuff, and tar & bzip(2) the needed subdirectories, see the new links below.
                I keep all the files on my HDD, so in case needed I can put them on-line by demand (but please note that I'll leave for the next week, so I'll be off-line, or read only at best)

                The new links are here (I thought in fact that these files are less "compressible", but - to my surprise - they weren't :))

                [corrupted_ssx.tar.bz2|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/corrupted_ssx.tar.bz2] (197.3 MB; *1.9 GB extracted*)
                [ir2hf_ss12_1.tar.bz2|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/ir2hf_ss12_1.tar.bz2] (199.8 MB; *1.6GB extracted*)
                [ir2hf_ssx.tar.bz2|http://www.orl.szote.u-szeged.hu/~attila/ssbughunt/ir2hf_ssx.tar.bz2] (81.2 MB; *648.6 MB extracted*)

                Thanks for the feedback, and for taking the time and looking into this! :)
                1 2 Previous Next