6 Replies Latest reply: Sep 26, 2013 8:19 AM by user464965 RSS

    JMS VAM Capture Startup Error

    user464965

      I have a JMS queue that I want to capture into a GoldenGate trail (which will later be pumped into an Oracle DB), but I'm getting the following error when I try to start it up:

       

      EXTRACT JMSVAM started.

      Recovery initialization completed for target file /u01/trails/va000007, at RBA 774.

      Output file /u01/trails/va is using format RELEASE 10.4/11.1.

      Rolling over remote file /u01/trails/va000007.

      Recovery completed for target file /u01/trails/va000008, at RBA 774.

      Recovery completed for all targets.

      VAM API running in single-threaded mode.

      Positionin to start.

      VAM Client Report < ***** Oracle GoldenGate Adapter for Java Version 11.1.1.0.0 Build: 007 SunOS_5-9 [sparcv9 (64-bit); Java 5]; GG Java API: 11_1_1_0_0_006 (20100729_085631) ***** >.

      VAM Client Report <Error occured in parser.c[243]: The type of parser must be specified in property parser.type >.

      VAM Client Report <Error occured in javavamimpl.c[114]: Could not initialize parser >.

      VAM function VAMInitialize returned unexpected result: error 600 - VAM Client Report <Could not initialize Generic Vam>.

      PROCESS ABENDING.

       

      I *do* have "parser.type=xml" defined in my VAM message capture properties file, but I'm sure some of my dyntxrule.*, dynoprule.*, and dyncolrule.* rules are not all there, or at least may not all be properly defined yet.

       

      My question is, will I continue to see the "parser must be specified in property parser.type" error until I have all of these other properties correct?? ... or is there some other problem going on?  I'm guessing the latter, since I've gone so far as to delete all of my dyn*.* rules ... so that I'm left with something like this, and I'm still getting the "parser must be specified in property parser.type" error ... eg:

       

       

      log.level=INFO

      log.logname=e_jms

      log.tostdout=true

      log.tofile=true

       

      provider.type=jms

       

      gg.source=ms

      gg.report.time=30sec

       

      java.naming.provider.url=t3://myhost.mydom.com:7001

      java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory

      java.naming.security.principal=myUser

      java.naming.security.credentials=myPW

       

      gg.jms.destination=jms/myJmsQ

      gg.jms.connectionFactory=ConnectionFactory

      gg.jms.id=time

       

      parser.type=xml

       

       

      Thanks for any assistance/clarification on this VAM capture error.

        • 1. Re: JMS VAM Capture Startup Error
          MikeN

          Verify that your properties file is being used, by checking the prm file; e.g.,

           

           

          Extract e_jms
          -- (1) Before starting mgr, put the JVM in the library path (libjvm.so)
          --   (Solaris/Linux=> LD_LIBRARY_PATH, z/OS,AIX=> LIBPATH, Windows=> PATH)
          --     sh> echo LD_LIBRARY_PATH=$LD_LIBRARY_PATH
          --     ggsci> start mgr
          --
          --   (Optional) When testing, clean up previous tests
          --     sh> rm dirdat/tz*
          --     ggsci> delete extract e_jms
          --
          -- (2) Add the JMS VAM capture
          --     ggsci> add extract e_jms, VAM
          --     ggsci> info extract e_jms
          --
          -- (3) Add the target trail (both in ggsci, and the prm file)
          --     ggsci> add extTrail dirdat/tz, extract e_jms, megabytes 200
          
          VAM libggjava_vam.so PARAMS (dirprm/jmsvam.properties)
          -- VAM ggjava_vam.dll PARAMS (dirprm/jmsvam.properties)
          
          ReportCount Every 10000 Records, Rate
          
          GetUpdateBefores
          
          TranLogOptions VAMCOMPATIBILITY 1
          TranLogOptions GETMETADATAFROMVAM
          
          ExtTrail dirdat/tz
          
          Table GG.*;
          
          
          
          

           

          Then the corresponding (sample) jmsvam.properties file would look like this (here, with lots of commentary):

           

          ########################################################################
          ## connection factory name (JNDI)
          # gg.jms.connectionFactory=...
          ## queue/topic name (JNDI)
          # gg.jms.destination=...
          ## destination username/password (not JNDI authentication)
          # gg.jms.user=...
          # gg.jms.password=...
          ## JNDI properties
          # java.naming.provider.url=...
          # java.naming.factory.initial=...
          ## put java.naming.security.* properties in dirprm/jndi.properties
          # java.naming.security.principal=...
          # java.naming.security.credentials=...
          ########################################################################
          
          ## example for WebLogic
          gg.jms.destination=examples.MyQueue
          gg.jms.connectionFactory=weblogic.examples.jms.QueueConnectionFactory
          java.naming.provider.url=t3://localhost:7001
          java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory
          ## if JNDI requires authentication (add to dirprm/jndi.properties)
          # java.naming.security.principal=weblogic
          # java.naming.security.credentials=welcome1
          
          ## example for ActiveMQ 
          #gg.jms.connectionFactory=ConnectionFactory
          #gg.jms.destination=dynamicQueues/testQ
          #java.naming.provider.url=tcp://localhost:61616?jms.prefetchPolicy.all=1000
          #java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory
          #...
          
          
          # set localtx=false to disable JMS transactions, using client_acknowledge instead
          gg.source=jms
          gg.jms.localtx=false
          gg.jms.id=time
          # reporting frequency to log file
          gg.report.time=1 hr
          
          ## Configuration for native JNI library
          # for logging, set the following (put the config in dirprm):
          #  -Dlog4j.configuration={optionally this file} (log4j is the v11.1 default; v11.2+ uses slf4j)
          # jvm.bootoptions=-Djava.class.path=dirprm:ggjava/ggjava.jar:/opt/activemq/activemq-all.jar
          jvm.bootoptions=-Djava.class.path=dirprm:ggjava/ggjava.jar:/opt/weblogic/wlfullclient.jar
          
          
          ## native JNI library logging configuration
          # logfile path and filename prefix
          goldengate.log.logname=dirout/jmsvam_jni
          goldengate.log.level=INFO
          goldengate.log.tostdout=false
          goldengate.log.tofile=true
          goldengate.log.modules=GENDEF,GENERICVAM,JAVAVAMIMPL,JMSPROVIDER,PARSER,PROVIDER,SFXML,XMLPARSER,XMLRULES
          
          ########################################################################
          # XML parsing properties.
          #  The properties below can be used to parse the GG JMS delivery (user-exit)
          #  built-in "format=xml2" message format (less verbose than "format=xml").
          #  The "tcustmer" and "tcustord" demo SQL can be used as an example.
          ########################################################################
          parser.type=xml
          # generate source-definitions based on target database schema,
          # and create appropriately formatted trail
          
          xml.sourcedefs=dirdef/tc.def
          
          # xml.rules = tx_rule => op_rule => col_rule
          
          xml.rules=tx_rule
          
          ##### transactions
          tx_rule.type=tx
          tx_rule.match=/transaction
          tx_rule.subrules=op_rule
          tx_rule.txid=*txid
          
          ##### operations
          op_rule.type=op
          op_rule.match=./operation
          op_rule.seqid=*seqid
          op_rule.timestamp=@ts
          
          op_rule.schemaandtable=@table
          op_rule.optype=@type
          op_rule.optype.insertval=INSERT
          op_rule.optype.updateval=UPDATE
          op_rule.optype.deleteval=DELETE
          op_rule.subrules=col_rule
          
          ##### subrules.columns
          col_rule.type=col
          col_rule.match=./column
          col_rule.name=@name
          col_rule.index=@index
          col_rule.before.value=./before-value/text()
          col_rule.before.isnull=./before-value[@isNull=true]/@isNull
          col_rule.after.value=./after-value/text()
          col_rule.before.isnull=./after-value[@isNull=true]/@isNull
          
          ## note: checking element existence is faster than getting attributes;
          ## so the following rules are preferable, but the XML format would
          ## have to be:  <after-value><isNull/></after-value>"
          ## instead of:  <after-value isNull='true'/>
          #col_rule.before.isnull=./before-value/isNull/exists()
          #col_rule.after.isnull=./after-value/isNull/exists()
          
          ## transaction ID notes:
          ## the following uses a system-generated tx ID
          #    tx_rule.txid=*txid
          ## examples for txid, using message properties or headers:
          #    tx_rule.txid=$GG_XID
          #    tx_rule.txid=$JMSMessageID
          #
          ## sequence ID notes:
          ## if seqid=$jmsid, use gg.jms.id
          #    op_rule.seqid=$jmsid
          ## can use header/property for seqid
          #    op_rule.seqid=$GG_ID
          #    op_rule.timestamp=$JMSTimestamp
          ## for timestamp=$GG_JMS_READ_TIMESTAMP, use timestamp.format=JMS (posix)
          #    op_rule.timestamp=$GG_JMS_READ_TIMESTAMP
          #    op_rule.timestamp.format=JMS
          
          
          
          

           

           

           

          If you configure the JMS delivery (user-exit) to send messages using "format=xml2", then the above configuration should be able to parse it; or just send a message like:

           

          <transaction ts='2013-09-15 22:39:49.000000' numOps='1' startPosition='00000000000000000000' endPosition='00000000000000000528'>
            <operation table='MNIELSEN.TCUSTMER' type='INSERT' opType='INSERT' txInd='BEGIN' ts='2013-09-15 22:39:49.000000' numCols='4' position='00000000000000000000'>
              <column name='CUST_CODE' index='0'>
                <after-value><![CDATA[WILL]]></after-value>
              </column>
              <column name='NAME' index='1'>
                <after-value><![CDATA[BG SOFTWARE CO.]]></after-value>
              </column>
              <column name='CITY' index='2'>
                <after-value><![CDATA[SEATTLE]]></after-value>
              </column>
              <column name='STATE' index='3'>
                <after-value><![CDATA[WA]]></after-value>
              </column>
            </operation>
          </transaction>
          
          
          
          

           

           

           

          (caveat lector: that is just an example I pieced to together w/o running it; some tweaking may still be necessary to get it working.)


          Hope it helps,
          -Michael

           

          Message was edited by: MikeN -- adding code formatting  (this new forum software really hates source code.)

          • 2. Re: JMS VAM Capture Startup Error
            user464965

            There is a bit more information about where in the source code I'm failing in the logging that gets written to the ./dirrpt/JMSVAM.rpt file :

             

            VAM Client Report <Error occured in parser.c[243]: The type of parser must be specified in property parser.type >.

             

            VAM Client Report <Error occured in javavamimpl.c[114]: Could not initialize parser >.

             

            Source Context:
              SourceModule   : [ggvam.gen]
              SourceID       : [/home/ecloud/workspace/Build_OpenSys_r11.1.1.0.0_078_[34109]/perforce/src/gglib/ggvam/cvamgen.cpp]
              SourceFunction : [vamInitialize]
              SourceLine     : [418]

             

            VAM function VAMInitialize returned unexpected result: error 600 - VAM Client Report <Could not initialize Generic Vam>.

             

            PROCESS ABENDING.

            • 3. Re: JMS VAM Capture Startup Error
              MikeN

              please post the full report file (dirrpt/*.rpt), and the full properties (dirprm/*.properties), and the tail any relevant log4j/java application log file & tail of the native library log file.

               

              (Tip: use the forum code syntax highlite option by clicking on "use the advanced editor", ">>" , "highlighting","plain" (or "SQL" or for prm files, "ruby" for properties files)

              • 4. Re: JMS VAM Capture Startup Error
                user464965

                I'm past my original error ... the cause of that was improper syntax for the VAM PARAMS argument (although it wasn't fatal!).  Here's what I had when I was getting the error ... note that I had argument to the PARAMS (the part after the comma) on a new line:

                 

                VAM libggjava_vam.so PARAMS,

                dirprm/jmsvam.properties

                 

                Once I changed the two lines above to the following single line (and included the argument to PARAMS so that it was within parentheses) ... like this:

                 

                VAM libggjava_vam.so PARAMS, (dirprm/jmsvam.properties)

                 

                ... my original error went away.

                 

                The error I'm getting now  is further along ... actually reading a message from the JMS queue :

                 

                (... a bunch of initial startup output skipped  here)

                XMLRULES(xmlrules.c:477)  [INFO]: XML Rules Initialized

                PROVIDER(provider.c:40) [INFO]: Using jms provider

                JMSPROVIDER(jmsprovider.c:491) [INFO]: Initializing JMS Provider

                JMSPROVIDER(jmsprovider.c:75) [INFO]:Initializing JVM: -Dcom.goldengate.atg.conf=jmsVam.properties -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/wlfullclient.jar

                JMSPROVIDER(jmsprovider.c:95) [INFO]: Added JVM option: -Dcom.goldengate.atg.conf=jmsVam.properties

                JMSPROVIDER(jmsprovider.c:95) [INFO]: Added JVM option: -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/wlfullclient.jar

                JMSPROVIDER(jmsprovider.c:378) [INFO]: Calling 'main' with JVM options: [-Dcom.goldengate.atg.conf=jmsVam.properties -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/wlfullclient.jar], passing args to Java application: []

                JMSPROVIDER(jmsprovider.c534) [INFO]: JMS Provider Initialized

                JAVAVAMIMPL(javavamimpl.c:129) [INFO]: Property checkpoint.mode not specified, will checkpoint on tx end

                JAVAVAM(genericvam.c:386) [INFO]: Initialized Generic Vam

                JAVAVAM(genericvam.c:392) [INFO]: Exclude User:  Len 0

                JAVAVAM(genericvam.c:410) [INFO]: Position: Default start mode, going to EOF

                XMLRULES(xmlrules.c:650) [ERROR]: Operation type INSERT does not match any valid operation types

                 

                 

                The operation type "INSERT" is the value that appears in the @opType attribute of an operation node in the XML of my JMS message ... which is generated came from a GoldenGate Trail, where the opType value is populated from an operation object's opType property!

                • 5. Re: JMS VAM Capture Startup Error
                  MikeN

                  Try running the example I've included above.  And instead of sending messages from a GoldenGate trail, send the same, single hard-coded message to the JMS VAM until you get the properties correct to parse the message.  (Note: you have "jmsvam.properties" in your example, but "jmsVam.properties" in the report. Perhaps unrelated, but eventually will cause problems.)

                  • 6. Re: JMS VAM Capture Startup Error
                    user464965

                    By default the JMS VAM Capture is expecting the following opType values : I, U, D (... for Insert, Update, and Delete, respectively).  The values I'm getting out of the JMS messages for these three operations are INSERT, UPDATE_FIELDCOMP, and DELETE.  So all I needed to do to resolve this problem in the JMS VAM Capture was to add the following three lines to my jmsVam.properties file:

                     

                    dynoprule.optype.insertval=INSERT

                    dynoprule.optype.updateval=UPDATE_FIELDCOMP

                    dynoprule.optype.deleteval=DELETE