This discussion is archived
6 Replies Latest reply: Sep 26, 2013 6:19 AM by user464965 RSS

JMS VAM Capture Startup Error

user464965 Newbie
Currently Being Moderated

I have a JMS queue that I want to capture into a GoldenGate trail (which will later be pumped into an Oracle DB), but I'm getting the following error when I try to start it up:

 

EXTRACT JMSVAM started.

Recovery initialization completed for target file /u01/trails/va000007, at RBA 774.

Output file /u01/trails/va is using format RELEASE 10.4/11.1.

Rolling over remote file /u01/trails/va000007.

Recovery completed for target file /u01/trails/va000008, at RBA 774.

Recovery completed for all targets.

VAM API running in single-threaded mode.

Positionin to start.

VAM Client Report < ***** Oracle GoldenGate Adapter for Java Version 11.1.1.0.0 Build: 007 SunOS_5-9 [sparcv9 (64-bit); Java 5]; GG Java API: 11_1_1_0_0_006 (20100729_085631) ***** >.

VAM Client Report <Error occured in parser.c[243]: The type of parser must be specified in property parser.type >.

VAM Client Report <Error occured in javavamimpl.c[114]: Could not initialize parser >.

VAM function VAMInitialize returned unexpected result: error 600 - VAM Client Report <Could not initialize Generic Vam>.

PROCESS ABENDING.

 

I *do* have "parser.type=xml" defined in my VAM message capture properties file, but I'm sure some of my dyntxrule.*, dynoprule.*, and dyncolrule.* rules are not all there, or at least may not all be properly defined yet.

 

My question is, will I continue to see the "parser must be specified in property parser.type" error until I have all of these other properties correct?? ... or is there some other problem going on?  I'm guessing the latter, since I've gone so far as to delete all of my dyn*.* rules ... so that I'm left with something like this, and I'm still getting the "parser must be specified in property parser.type" error ... eg:

 

 

log.level=INFO

log.logname=e_jms

log.tostdout=true

log.tofile=true

 

provider.type=jms

 

gg.source=ms

gg.report.time=30sec

 

java.naming.provider.url=t3://myhost.mydom.com:7001

java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory

java.naming.security.principal=myUser

java.naming.security.credentials=myPW

 

gg.jms.destination=jms/myJmsQ

gg.jms.connectionFactory=ConnectionFactory

gg.jms.id=time

 

parser.type=xml

 

 

Thanks for any assistance/clarification on this VAM capture error.

  • 1. Re: JMS VAM Capture Startup Error
    MikeN Journeyer
    Currently Being Moderated

    Verify that your properties file is being used, by checking the prm file; e.g.,

     

     

    Extract e_jms
    -- (1) Before starting mgr, put the JVM in the library path (libjvm.so)
    --   (Solaris/Linux=> LD_LIBRARY_PATH, z/OS,AIX=> LIBPATH, Windows=> PATH)
    --     sh> echo LD_LIBRARY_PATH=$LD_LIBRARY_PATH
    --     ggsci> start mgr
    --
    --   (Optional) When testing, clean up previous tests
    --     sh> rm dirdat/tz*
    --     ggsci> delete extract e_jms
    --
    -- (2) Add the JMS VAM capture
    --     ggsci> add extract e_jms, VAM
    --     ggsci> info extract e_jms
    --
    -- (3) Add the target trail (both in ggsci, and the prm file)
    --     ggsci> add extTrail dirdat/tz, extract e_jms, megabytes 200
    VAM libggjava_vam.so PARAMS (dirprm/jmsvam.properties)
    -- VAM ggjava_vam.dll PARAMS (dirprm/jmsvam.properties)
    ReportCount Every 10000 Records, Rate
    GetUpdateBefores
    TranLogOptions VAMCOMPATIBILITY 1
    TranLogOptions GETMETADATAFROMVAM
    ExtTrail dirdat/tz
    Table GG.*;
    
    
    

     

    Then the corresponding (sample) jmsvam.properties file would look like this (here, with lots of commentary):

     

    ########################################################################
    ## connection factory name (JNDI)
    # gg.jms.connectionFactory=...
    ## queue/topic name (JNDI)
    # gg.jms.destination=...
    ## destination username/password (not JNDI authentication)
    # gg.jms.user=...
    # gg.jms.password=...
    ## JNDI properties
    # java.naming.provider.url=...
    # java.naming.factory.initial=...
    ## put java.naming.security.* properties in dirprm/jndi.properties
    # java.naming.security.principal=...
    # java.naming.security.credentials=...
    ########################################################################
    ## example for WebLogic
    gg.jms.destination=examples.MyQueue
    gg.jms.connectionFactory=weblogic.examples.jms.QueueConnectionFactory
    java.naming.provider.url=t3://localhost:7001
    java.naming.factory.initial=weblogic.jndi.WLInitialContextFactory
    ## if JNDI requires authentication (add to dirprm/jndi.properties)
    # java.naming.security.principal=weblogic
    # java.naming.security.credentials=welcome1
    ## example for ActiveMQ 
    #gg.jms.connectionFactory=ConnectionFactory
    #gg.jms.destination=dynamicQueues/testQ
    #java.naming.provider.url=tcp://localhost:61616?jms.prefetchPolicy.all=1000
    #java.naming.factory.initial=org.apache.activemq.jndi.ActiveMQInitialContextFactory
    #...# set localtx=false to disable JMS transactions, using client_acknowledge instead
    gg.source=jms
    gg.jms.localtx=false
    gg.jms.id=time
    # reporting frequency to log file
    gg.report.time=1 hr
    ## Configuration for native JNI library
    # for logging, set the following (put the config in dirprm):
    #  -Dlog4j.configuration={optionally this file} (log4j is the v11.1 default; v11.2+ uses slf4j)
    # jvm.bootoptions=-Djava.class.path=dirprm:ggjava/ggjava.jar:/opt/activemq/activemq-all.jar
    jvm.bootoptions=-Djava.class.path=dirprm:ggjava/ggjava.jar:/opt/weblogic/wlfullclient.jar
    ## native JNI library logging configuration
    # logfile path and filename prefix
    goldengate.log.logname=dirout/jmsvam_jni
    goldengate.log.level=INFO
    goldengate.log.tostdout=false
    goldengate.log.tofile=true
    goldengate.log.modules=GENDEF,GENERICVAM,JAVAVAMIMPL,JMSPROVIDER,PARSER,PROVIDER,SFXML,XMLPARSER,XMLRULES
    ########################################################################
    # XML parsing properties.
    #  The properties below can be used to parse the GG JMS delivery (user-exit)
    #  built-in "format=xml2" message format (less verbose than "format=xml").
    #  The "tcustmer" and "tcustord" demo SQL can be used as an example.
    ########################################################################
    parser.type=xml
    # generate source-definitions based on target database schema,
    # and create appropriately formatted trail
    xml.sourcedefs=dirdef/tc.def
    # xml.rules = tx_rule => op_rule => col_rule
    xml.rules=tx_rule
    ##### transactions
    tx_rule.type=tx
    tx_rule.match=/transaction
    tx_rule.subrules=op_rule
    tx_rule.txid=*txid
    ##### operations
    op_rule.type=op
    op_rule.match=./operation
    op_rule.seqid=*seqid
    op_rule.timestamp=@ts
    op_rule.schemaandtable=@table
    op_rule.optype=@type
    op_rule.optype.insertval=INSERT
    op_rule.optype.updateval=UPDATE
    op_rule.optype.deleteval=DELETE
    op_rule.subrules=col_rule
    ##### subrules.columns
    col_rule.type=col
    col_rule.match=./column
    col_rule.name=@name
    col_rule.index=@index
    col_rule.before.value=./before-value/text()
    col_rule.before.isnull=./before-value[@isNull=true]/@isNull
    col_rule.after.value=./after-value/text()
    col_rule.before.isnull=./after-value[@isNull=true]/@isNull
    ## note: checking element existence is faster than getting attributes;
    ## so the following rules are preferable, but the XML format would
    ## have to be:  <after-value><isNull/></after-value>"
    ## instead of:  <after-value isNull='true'/>
    #col_rule.before.isnull=./before-value/isNull/exists()
    #col_rule.after.isnull=./after-value/isNull/exists()
    ## transaction ID notes:
    ## the following uses a system-generated tx ID
    #    tx_rule.txid=*txid
    ## examples for txid, using message properties or headers:
    #    tx_rule.txid=$GG_XID
    #    tx_rule.txid=$JMSMessageID
    #
    ## sequence ID notes:
    ## if seqid=$jmsid, use gg.jms.id
    #    op_rule.seqid=$jmsid
    ## can use header/property for seqid
    #    op_rule.seqid=$GG_ID
    #    op_rule.timestamp=$JMSTimestamp
    ## for timestamp=$GG_JMS_READ_TIMESTAMP, use timestamp.format=JMS (posix)
    #    op_rule.timestamp=$GG_JMS_READ_TIMESTAMP
    #    op_rule.timestamp.format=JMS
    
    
    

     

     

     

    If you configure the JMS delivery (user-exit) to send messages using "format=xml2", then the above configuration should be able to parse it; or just send a message like:

     

    <transaction ts='2013-09-15 22:39:49.000000' numOps='1' startPosition='00000000000000000000' endPosition='00000000000000000528'>
      <operation table='MNIELSEN.TCUSTMER' type='INSERT' opType='INSERT' txInd='BEGIN' ts='2013-09-15 22:39:49.000000' numCols='4' position='00000000000000000000'>
        <column name='CUST_CODE' index='0'>
          <after-value><![CDATA[WILL]]></after-value>
        </column>
        <column name='NAME' index='1'>
          <after-value><![CDATA[BG SOFTWARE CO.]]></after-value>
        </column>
        <column name='CITY' index='2'>
          <after-value><![CDATA[SEATTLE]]></after-value>
        </column>
        <column name='STATE' index='3'>
          <after-value><![CDATA[WA]]></after-value>
        </column>
      </operation>
    </transaction>
    
    
    

     

     

     

    (caveat lector: that is just an example I pieced to together w/o running it; some tweaking may still be necessary to get it working.)


    Hope it helps,
    -Michael

     

    Message was edited by: MikeN -- adding code formatting  (this new forum software really hates source code.)

  • 2. Re: JMS VAM Capture Startup Error
    user464965 Newbie
    Currently Being Moderated

    There is a bit more information about where in the source code I'm failing in the logging that gets written to the ./dirrpt/JMSVAM.rpt file :

     

    VAM Client Report <Error occured in parser.c[243]: The type of parser must be specified in property parser.type >.

     

    VAM Client Report <Error occured in javavamimpl.c[114]: Could not initialize parser >.

     

    Source Context:
      SourceModule   : [ggvam.gen]
      SourceID       : [/home/ecloud/workspace/Build_OpenSys_r11.1.1.0.0_078_[34109]/perforce/src/gglib/ggvam/cvamgen.cpp]
      SourceFunction : [vamInitialize]
      SourceLine     : [418]

     

    VAM function VAMInitialize returned unexpected result: error 600 - VAM Client Report <Could not initialize Generic Vam>.

     

    PROCESS ABENDING.

  • 3. Re: JMS VAM Capture Startup Error
    MikeN Journeyer
    Currently Being Moderated

    please post the full report file (dirrpt/*.rpt), and the full properties (dirprm/*.properties), and the tail any relevant log4j/java application log file & tail of the native library log file.

     

    (Tip: use the forum code syntax highlite option by clicking on "use the advanced editor", ">>" , "highlighting","plain" (or "SQL" or for prm files, "ruby" for properties files)

  • 4. Re: JMS VAM Capture Startup Error
    user464965 Newbie
    Currently Being Moderated

    I'm past my original error ... the cause of that was improper syntax for the VAM PARAMS argument (although it wasn't fatal!).  Here's what I had when I was getting the error ... note that I had argument to the PARAMS (the part after the comma) on a new line:

     

    VAM libggjava_vam.so PARAMS,

    dirprm/jmsvam.properties

     

    Once I changed the two lines above to the following single line (and included the argument to PARAMS so that it was within parentheses) ... like this:

     

    VAM libggjava_vam.so PARAMS, (dirprm/jmsvam.properties)

     

    ... my original error went away.

     

    The error I'm getting now  is further along ... actually reading a message from the JMS queue :

     

    (... a bunch of initial startup output skipped  here)

    XMLRULES(xmlrules.c:477)  [INFO]: XML Rules Initialized

    PROVIDER(provider.c:40) [INFO]: Using jms provider

    JMSPROVIDER(jmsprovider.c:491) [INFO]: Initializing JMS Provider

    JMSPROVIDER(jmsprovider.c:75) [INFO]:Initializing JVM: -Dcom.goldengate.atg.conf=jmsVam.properties -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/wlfullclient.jar

    JMSPROVIDER(jmsprovider.c:95) [INFO]: Added JVM option: -Dcom.goldengate.atg.conf=jmsVam.properties

    JMSPROVIDER(jmsprovider.c:95) [INFO]: Added JVM option: -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/wlfullclient.jar

    JMSPROVIDER(jmsprovider.c:378) [INFO]: Calling 'main' with JVM options: [-Dcom.goldengate.atg.conf=jmsVam.properties -Djava.class.path=dirprm:ggjava/ggjava.jar:dirprm/wlfullclient.jar], passing args to Java application: []

    JMSPROVIDER(jmsprovider.c534) [INFO]: JMS Provider Initialized

    JAVAVAMIMPL(javavamimpl.c:129) [INFO]: Property checkpoint.mode not specified, will checkpoint on tx end

    JAVAVAM(genericvam.c:386) [INFO]: Initialized Generic Vam

    JAVAVAM(genericvam.c:392) [INFO]: Exclude User:  Len 0

    JAVAVAM(genericvam.c:410) [INFO]: Position: Default start mode, going to EOF

    XMLRULES(xmlrules.c:650) [ERROR]: Operation type INSERT does not match any valid operation types

     

     

    The operation type "INSERT" is the value that appears in the @opType attribute of an operation node in the XML of my JMS message ... which is generated came from a GoldenGate Trail, where the opType value is populated from an operation object's opType property!

  • 5. Re: JMS VAM Capture Startup Error
    MikeN Journeyer
    Currently Being Moderated

    Try running the example I've included above.  And instead of sending messages from a GoldenGate trail, send the same, single hard-coded message to the JMS VAM until you get the properties correct to parse the message.  (Note: you have "jmsvam.properties" in your example, but "jmsVam.properties" in the report. Perhaps unrelated, but eventually will cause problems.)

  • 6. Re: JMS VAM Capture Startup Error
    user464965 Newbie
    Currently Being Moderated

    By default the JMS VAM Capture is expecting the following opType values : I, U, D (... for Insert, Update, and Delete, respectively).  The values I'm getting out of the JMS messages for these three operations are INSERT, UPDATE_FIELDCOMP, and DELETE.  So all I needed to do to resolve this problem in the JMS VAM Capture was to add the following three lines to my jmsVam.properties file:

     

    dynoprule.optype.insertval=INSERT

    dynoprule.optype.updateval=UPDATE_FIELDCOMP

    dynoprule.optype.deleteval=DELETE

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points