This discussion is archived
12 Replies Latest reply: Feb 3, 2012 6:52 AM by Nayas RSS

ebs r12 oacore memory

Nayas Explorer
Currently Being Moderated
Hi All,

Ebs r12.1.3
rdbms 11.2.0.2

I would like to know if there is a way to monitor oacore memory usage and if there is a way to alert us when the memory is close to the maximum memory available (-Xmx parameter)?
I also would like to know, our oacore seems to fill during the week and at one point, doesn't have enough memory and they froze/crash. We have to bounce them to fix the issue.
Does any of you do on a regular basic a bounce of OC4J components (adopmnctl.sh stop / start) ?

Thank you,
Felix
  • 1. Re: ebs r12 oacore memory
    EBSDBA Expert
    Currently Being Moderated
    Hi,
    Please see if it helps

    http://riktutorials.blogspot.com/2010/04/how-to-monitor-memory-usage-in-oc4j-as.html

    Simply source the appl. env file and run jconsole (using vnc or any other graphical tool).
    It will list you the local processes.
    To find the oacore process, do the following
    applmgr$ ps -ef|grep oacorestd.err
    It'll show you the PID, The same pid you can see it jconsole.

    More information:

    How to create a Java heapdump on e-Business Suite ? [ID 835909.1]
    Guidelines to setup the JVM in Apps Ebusiness Suite 11i and R12 [ID 362851.1]

    http://blogs.oracle.com/stevenChan/entry/using_jconsole_to_monitor_apps
    OACore monitoring question (A previous thread of yourself :) )

    Thanks
  • 2. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi,

    Oh, I forgot I already asked the same question :S
    I am trying to use jconsole but in the local tab, I do not have any oacore process, only GSM.

    I am getting the PID from $INST_TOP/admin/scripts/adopmnctl.sh status.
    When I try to do jconsole PID, I am getting:
    [server.domain.com]:> $INST_TOP/admin/scripts/adopmnctl.sh status
    
    You are running adopmnctl.sh version 120.6.12010000.5
    
    Checking status of OPMN managed processes...
    
    Processes in Instance: SID_server.server.domain.com
    ---------------------------------+--------------------+---------+---------
    ias-component                    | process-type       |     pid | status
    ---------------------------------+--------------------+---------+---------
    OC4JGroup:default_group          | OC4J:oafm          | 2891892 | Alive
    OC4JGroup:default_group          | OC4J:oacore        | 104695~ | Alive
    HTTP_Server                      | HTTP_Server        | 101213~ | Alive
    
    [server.domain.com]:> which jconsole
    $IAS_ORACLE_HOME/10.1.3/appsutil/jdk/bin/jconsole
    
    [server.domain.com]:> jconsole 10469588
    com.ibm.tools.attach.AttachNotSupportedException: target not found
            at com.ibm.tools.attach.javaSE.VirtualMachineImpl.tryAttachTarget(VirtualMachineImpl.java:303)
            at com.ibm.tools.attach.javaSE.VirtualMachineImpl.attachTarget(VirtualMachineImpl.java:86)
            at com.ibm.tools.attach.javaSE.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:37)
            at ibm.tools.attach.J9AttachProvider.attachVirtualMachine(J9AttachProvider.java:60)
            at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:221)
            at sun.tools.jconsole.LocalVirtualMachine.getLocalVirtualMachine(LocalVirtualMachine.java:205)
            at sun.tools.jconsole.JConsole.main(JConsole.java:924)
    Invalid process id:10469588
    But:
    [server.domain.com]:> ps -ef | grep 104695
     applmgr 9469982 10244248   0 15:23:29  pts/2  0:00 grep 104695
     applmgr 10469588  1806434   0 14:18:34      -  0:13 $IAS_ORACLE_HOME/10.1.3/appsutil/jdk/bin/java -DCLIENT_PROCESSID=10469588 -verbose:gc -Xmx512M -Xms128M -Dcom.sun.management.jmxremote -Djava.security.policy=$IAS_ORACLE_HOME/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doracle.security.jazn.config=$IAS_ORACLE_HOME/j2ee/oacore/config/jazn.xml -Dhttp.cookie.ignoreCommaInCookiesNamed=X_NoMatchingCookies -Doracle.ons.oraclehome=$IAS_ORACLE_HOME -Doracle.home=$ISA_ORACLE_HOME -Doracle.ons.oracleconfighome=$INST_TOP/ora/10.1.3 -Doracle.ons.clustername=default -Doracle.ons.instancename=SID_server.server.domain.com -Dopmn.compatible=904 -Doracle.ons.indexid=oacore.default_group.1 -Doracle.ons.numprocs=1 -Doracle.ons.uid=745050216 -Doracle.oc4j.groupname=default_group -Doracle.oc4j.instancename=oacore -Doracle.oc4j.islandname=default_group -Doracle.opmn.routingid=g_rt_id -DOPMN=true -jar oc4j.jar -config $INST_TOP/ora/10.1.3/j2ee/oacore/config/server.xml -properties -out $INST_TOP/logs/ora/10.1.3/opmn/oacorestd.out -err $INST_TOP/logs/ora/10.1.3/opmn/oacorestd.err -ports default-web-site:ajp:21575,rmi:20075,jms:23075
    I do not understand why it says invalid process id.

    If I simply start jconsole without providing PID, I can see the local JVM. All I see is the FNDOPP / FNDCPGSC processes, nothing else.

    Any idea?

    Thank you,
    Felix
  • 3. Re: ebs r12 oacore memory
    EBSDBA Expert
    Currently Being Moderated
    Hi Feix,
    Just type jconsole

    ok, you will list of processes right.

    1. do

    $ps -ef|grep oacorestd.err
    IT shows you the process id for oacore.

    Match that process in the list of jconsole for oacore

    2.

    or you can put the cursor on each of the processes, it shows you the complete path (helper) . Like oacore,forms etc.

    Thanks
  • 4. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi,

    Problem with simply typing jconsole is that my oacore processes are not displayed in the local tab. Only FNDOPP and FNDGSM.

    Thanks
  • 5. Re: ebs r12 oacore memory
    EBSDBA Expert
    Currently Being Moderated
    Hi,
    I checked it on one of my instances (12.1.3 with 11.2.0.2). It displays here fine here.

    when you do ps -ef|grep oacorestd.err does it return any processes.

    Thanks
  • 6. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi,

    Note: in our tests instance, we only have 1 oacore process where as in prod, we have 2
    [server.domain.com]:> ps -fu applmgr | grep oacorestd.err
     applmgr 10469588  1806434   0 14:18:34      -  0:37 $IAS_ORACLE_HOME/appsutil/jdk/bin/java -DCLIENT_PROCESSID=10469588 -verbose:gc -Xmx512M -Xms128M -Dcom.sun.management.jmxremote -Djava.security.policy=$IAS_ORACLE_HOME/j2ee/oacore/config/java2.policy -Djava.awt.headless=true -Dhttp.webdir.enable=false -Doracle.security.jazn.config=$INST_TOP/ora/10.1.3/j2ee/oacore/config/jazn.xml -Dhttp.cookie.ignoreCommaInCookiesNamed=X_NoMatchingCookies -Doracle.ons.oraclehome=$IAS_ORACLE_HOME -Doracle.home=$IAS_ORACLE_HOME -Doracle.ons.oracleconfighome=$INST_TOP/ora/10.1.3 -Doracle.ons.clustername=default -Doracle.ons.instancename=IMGCPU_server.server.domain.com -Dopmn.compatible=904 -Doracle.ons.indexid=oacore.default_group.1 -Doracle.ons.numprocs=1 -Doracle.ons.uid=745050216 -Doracle.oc4j.groupname=default_group -Doracle.oc4j.instancename=oacore -Doracle.oc4j.islandname=default_group -Doracle.opmn.routingid=g_rt_id -DOPMN=true -jar oc4j.jar -config $INST_TOP/ora/10.1.3/j2ee/oacore/config/server.xml -properties -out $INST_TOP/logs/ora/10.1.3/opmn/oacorestd.out -err $INST_TOP/logs/ora/10.1.3/opmn/oacorestd.err -ports default-web-site:ajp:21575,rmi:20075,jms:23075
    Now if I do
    [server.domain.com]:> jconsole 1806434
    com.ibm.tools.attach.AttachNotSupportedException: target not found
            at com.ibm.tools.attach.javaSE.VirtualMachineImpl.tryAttachTarget(VirtualMachineImpl.java:303)
            at com.ibm.tools.attach.javaSE.VirtualMachineImpl.attachTarget(VirtualMachineImpl.java:86)
            at com.ibm.tools.attach.javaSE.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:37)
            at ibm.tools.attach.J9AttachProvider.attachVirtualMachine(J9AttachProvider.java:60)
            at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:221)
            at sun.tools.jconsole.LocalVirtualMachine.getLocalVirtualMachine(LocalVirtualMachine.java:205)
            at sun.tools.jconsole.JConsole.main(JConsole.java:924)
    Invalid process id:1806434
    If I simply type jconsole, in Java Monitoring & Management Console, I do not have my pid 1806434...

    Any idea?

    Thank you,
    Felix
  • 7. Re: ebs r12 oacore memory
    EBSDBA Expert
    Currently Being Moderated
    Hi Felix,
    applmgr 10469588 1806434
    Sorry the pid is 10469588 not 1806434

    Thanks
  • 8. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi,

    Sorry, my mistake.

    Same thing with the PID and not the PPID:
    [server.domain.com]:> jconsole 10469588
    com.ibm.tools.attach.AttachNotSupportedException: target not found
            at com.ibm.tools.attach.javaSE.VirtualMachineImpl.tryAttachTarget(VirtualMachineImpl.java:303)
            at com.ibm.tools.attach.javaSE.VirtualMachineImpl.attachTarget(VirtualMachineImpl.java:86)
            at com.ibm.tools.attach.javaSE.AttachProviderImpl.attachVirtualMachine(AttachProviderImpl.java:37)
            at ibm.tools.attach.J9AttachProvider.attachVirtualMachine(J9AttachProvider.java:60)
            at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:221)
            at sun.tools.jconsole.LocalVirtualMachine.getLocalVirtualMachine(LocalVirtualMachine.java:205)
            at sun.tools.jconsole.JConsole.main(JConsole.java:924)
    Invalid process id:10469588
    Thanks
  • 9. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi,

    Strange, I tried to same on a R12.1.1 lab we have at work and it works correctly. I can connect no problem to the oacore with jconsole <oacore_pid> ...

    I do not get why it doesn't work for that specific instance. Any special settings to allow jconsole to connect to oacore process?

    Thanks
  • 10. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi

    I start believing it's an issue with AIX JRE ... works greate on linux but not on AIX...
  • 11. Re: ebs r12 oacore memory
    EBSDBA Expert
    Currently Being Moderated
    Hi,
    Yeah you are right.

    This could be a bug in AIX.

    Thanks
  • 12. Re: ebs r12 oacore memory
    Nayas Explorer
    Currently Being Moderated
    Hi,
    FYI - we fixed jconsole issue.

    With AIX, JDK creates a folder under /tmp call .bom_ibm_tools_attach. Inside this folder, there are folders, each corresponding to the PID of jvm. The owner of the directory is the owner of the JVM.
    Our issue with jconsole was that, we changed, in applmgr .profile the variable TMPDIR to another directory than /tmp (for example, let say we use appldev, /apps/ebs/DEV/devtmp).

    When we go to that directory, there is also a .com_ibm_tools_attach directory with sub folders for each JVM running under appldev BESIDE oacore jvm. Oacore jvm is under /tmp.
    So when doing "jconsole OACORE_PID" we are getting "Invalid process id" because it cannot find the directory /apps/ebs/DEV/devtmp/.com_ibm_tools/attach/OACORE_PID because it's under /tmp.
    The workaround is to do TMPDIR=tmp jconsole OACORE_PID. Bingo, now it works.

    Now, where is the variable that says oacore tmp folder goes to /tmp if we overwrite everything to go to /apps/ebs/DEV/devtmp ...?
    We tried to variable in opmn.xml the following line but it didn't work:

    From
    <ias-instance id="SID_sever.server.domain.com" name="SID_sever.server.domain.com">
      <environment>
        <variable id="TMP" value="/tmp" />
      </environment>
    To
    <ias-instance id="SID_sever.server.domain.com" name="SID_sever.server.domain.com">
      <environment>
        <variable id="TMP" value="/apps/ebs/DEV/devtmp" />
      </environment>
    But regardless of jconsole and our issue with jconsole, i cannot believe we are the only one trying to monitor oacore and alert us when oacore memory is close to the maximum allowed?
    I cannot find anything to monitor and alert when memory gets close to the mamimum allowed...

    Thank you,
    Felix

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points