5 Replies Latest reply: May 7, 2014 12:56 PM by Michael Peel-Oracle RSS

    XMLParser: failed to read records

    John P

      I am facing this issue while baseline. Any help is highly appreciated.

       

      \logs\dgidxs\Dgidx\Dgidx.log

      ...

      ...

      XMLParser: failed to read record data 

      ERROR 05/05/14 14:27:23.199 UTC (1399300043199) DGIDX {dgidx,baseline} XMLParser: failed to read records 

      FATAL 05/05/14 14:27:23.199 UTC (1399300043199)

      DGIDX {dgidx,baseline}

      Binary record parsing of file 'E:\Endeca\apps\app2012\data\forge_output\\data2011-sgmt0.records.binary' unsuccessful.

       

       

      Note: app2011-sgmt0.records is a 1.2 GB file and I suspect whether the size of this file is causing this issue?

       

       

       

      \app2012\logs\app2012.0.0.log

      ...

      ...

      SEVERE: Batch component  'Dgidx' failed. Refer to component logs in E:\Endeca\apps\nfm2012\config\script\..\..\.\logs\dgidxs\Dgidx on host ITLHost.
      Occurred while executing line 32 of valid BeanShell script:
      [[

      29|        Forge.archiveLogDir();
      30|        Forge.run();
      31|        Dgidx.archiveLogDir();
      32|        Dgidx.run();
      33|       
      34|        // distributed index, update Dgraphs  //jpp
      35|        //DistributeIndexAndApply.run();

      ]]

      May 5, 2014 9:27:27 AM com.endeca.soleng.eac.toolkit.Controller execute
      SEVERE: Caught an exception while invoking method 'run' on object 'BaselineUpdate'. Releasing locks.
      java.lang.reflect.InvocationTargetException
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at com.endeca.soleng.eac.toolkit.Controller.invokeRequestedMethod(Controller.java:925)
      at com.endeca.soleng.eac.toolkit.Controller.execute(Controller.java:210)
      at com.endeca.soleng.eac.toolkit.Controller.main(Controller.java:89)
      Caused by: com.endeca.soleng.eac.toolkit.exception.AppControlException: Error executing valid BeanShell script.
      at com.endeca.soleng.eac.toolkit.script.Script.runBeanShellScript(Script.java:134)
      at com.endeca.soleng.eac.toolkit.script.Script.run(Script.java:82)
      ... 7 more
      Caused by: com.endeca.soleng.eac.toolkit.exception.EacComponentControlException: Batch component  'Dgidx' failed. Refer to component logs in E:\Endeca\apps\nfm2012\config\script\..\..\.\logs\dgidxs\Dgidx on host ITLHost.
      at com.endeca.soleng.eac.toolkit.component.BatchComponent.run(BatchComponent.java:79)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at bsh.Reflect.invokeMethod(Unknown Source)
      at bsh.Reflect.invokeObjectMethod(Unknown Source)
      at bsh.Name.invokeMethod(Unknown Source)
      at bsh.BSHMethodInvocation.eval(Unknown Source)
      at bsh.BSHPrimaryExpression.eval(Unknown Source)
      at bsh.BSHPrimaryExpression.eval(Unknown Source)
      at bsh.BSHBlock.evalBlock(Unknown Source)
      at bsh.BSHBlock.eval(Unknown Source)
      at bsh.BSHBlock.eval(Unknown Source)
      at bsh.BSHIfStatement.eval(Unknown Source)
      at bsh.BSHBlock.evalBlock(Unknown Source)
      at bsh.BSHBlock.eval(Unknown Source)
      at bsh.BSHBlock.eval(Unknown Source)
      at bsh.BSHIfStatement.eval(Unknown Source)
      at bsh.Interpreter.eval(Unknown Source)
      at bsh.Interpreter.eval(Unknown Source)
      at bsh.Interpreter.eval(Unknown Source)
      at com.endeca.soleng.eac.toolkit.script.Script.runBeanShellScript(Script.java:120)
      ... 8 more
      May 5, 2014 9:27:27 AM com.endeca.soleng.eac.toolkit.base.LockManager releaseLock
      INFO: Released lock 'update_lock'.

        • 1. Re: XMLParser: failed to read records
          Michael Peel-Oracle

          Don't think I've seen this before.  Are your versions of Platform Services and MDEX compatible?  The Forge component that generates this file is shipped with Platform Services, while the Dgidx component that reads this file ships with the MDEX (not the most intuitive, I know) - possibly these are incompatible?

           

          There isn't anything else in the Dgidx log referring to this?  Also check the forge log (./logs/Forges/Forge/Forge.x) to see if there is any issues with the generation of this file that might assist.

           

          Michael

          • 2. Re: XMLParser: failed to read records
            TimK

            I'm noticing an extra slash in your path - linux might be ok with this, but maybe windows is not. Can you fix that and try again?

            Binary record parsing of file 'E:\Endeca\apps\app2012\data\forge_output\\data2011-sgmt0.records.binary' unsuccessful.

            • 3. Re: XMLParser: failed to read records
              John P

              Michael,

               

              Thank you for your response, this is a strange behavior and we did recently upgraded MDEX to 6.2.2 from 6.1.4

               

              This is the current setup - (assuming its compatible.. lol)

              PlatformServices - 6.1.0

              Mdex - 6.2.2

               

              I quickly checked my Forge as you mentioned and I only have bunch of warnings and 0 errors

               

              WARN 05/06/14 18:31:19.398 UTC (1399401079383) FORGE {config}: Root dimension value name 'Water and Ice Dispenser' for dimension 'Water and Ice Dispenser' is not a valid NCName.

              WARN 05/06/14 18:31:19.398 UTC (1399401079383) FORGE {config}: Root dimension value name 'Solar Lights' for dimension 'Solar Lights' is not a valid NCName.

              WARN 05/06/14 18:31:19.398 UTC (1399401079383) FORGE {config}: Root dimension value name 'Shade Color' for dimension 'Shade Color' is not a valid NCName.

              WARN 05/06/14 18:31:19.398 UTC (1399401079383) FORGE {config}: Root dimension value name 'Comic Character' for dimension 'Comic Character' is not a valid NCName.

              WARN 05/06/14 18:35:05.057 UTC (1399401305057) FORGE {baseline}: Source property 'features_abbreviated' value contains binary data (char 0x1c) when mapping to 'p_features_abbreviated' on record number 195293

              WARN 05/06/14 18:35:31.089 UTC (1399401331089) FORGE {baseline}: Forge completed with 0 errors and 929 warnings.

               

              Other thought:

              Every Dgidx log and Endeca_Conf/log/shell logs point to the not able to read the below mentioned binary file.

              forge_output has this file xxx2011-sgmt0.records.binary is 1+ gb in size and this is the first time we are hitting above 1 gb.

               

              my attempts so far:

              Tried adding some memory to TomCat - setenv.bat - no luck

               

              When I grabbed the incoming data files to another instance on the same box and the baseline works fine.

               

              More logs:

              Endeca_Conf\logs\process.0.log

               

              May 6, 2014 2:22:15 PM com.endeca.esf.delegate.procctrl.ProcessControlImpl startProcess

              INFO: Starting process for TaskId(COMPONENT,nfm2012prod,Dgidx): E:\Endeca\MDEX\6.2.2\bin\dgidx.exe "-v" "--sort" "p_sort_default|desc" "--out" "E:\Endeca\apps\nfm2012prod\logs\dgidxs\Dgidx\Dgidx.log" "--dtddir" "E:\Endeca\MDEX\6.2.2\conf\dtd" "--tmpdir" "E:\Endeca\apps\nfm2012prod\data\temp" "E:\Endeca\apps\nfm2012prod\data\forge_output\nfm2011" "E:\Endeca\apps\nfm2012prod\data\dgidx_output\nfm2011"    STARTDIR: E:\Endeca\apps\nfm2012prod\config\script\..\..   ENV VARS: ENDECA_MDEX_ROOT=E:\Endeca\MDEX\6.2.2

              May 6, 2014 2:22:15 PM com.endeca.esf.delegate.task.ProcessHandlePollingTask startProcess

              INFO: Started process 3532 for dgidx 'Dgidx'

               

              Interested to hear different views or any other pointers to dig into this issue. This happens in our Stage box (prod is good so far) and little concerned if this gets replicated to prod

               

              Thanks

              John.P

              • 4. Re: XMLParser: failed to read records
                John P

                Thanks Michael and Tim,

                 

                This error was very unique and I was able to bring some production pipeline files downs to stage and compared to see one of the extra file (<appname>.xml sitting in pipeline which had the reference to the above mentioned .binary file). I paid close attention and check with the team and were able to safely guess this file was accidently copied from processing or a backed up copy of files as part of the recent upgrade.

                 

                Once I cleared this reference and cleared forge_output, dgidx_output, complete_index, processing folders out I was able to get baseline run now. Big relief

                 

                Appreciate your inputs as always (from the good old Eden days).

                 

                Tim, the double slash thing I believe is a bug in logging java code or setup of ours but this was always been this way in log files. Good catch though.

                 

                thanks

                John.P

                • 5. Re: XMLParser: failed to read records
                  Michael Peel-Oracle

                  Glad you got it sorted.  Was the data prefix in AppConfig.xml different to the OUTPUT_PREFIX in pipeline.epx then?  Worth double-checking that ./config/script/set_environment.[bat|sh] has the right PROJECT_NAME setting too.

                   

                  Thanks

                   

                  Michael