3 Replies Latest reply: Oct 24, 2012 8:48 AM by Iordan Iotzov RSS

    Oracle pmon process is not shown

    477504
      Hi facing a peculiar issue ? 10g RAC has been set up & I see all services running in crs_stat -t
      But I do not see any pmon,smon process ..


      [root@rac2 ~]# /u01/app/oracle/product/10.2.0/crs/bin/crs_stat -t
      Name Type Target State Host
      ------------------------------------------------------------
      ora....DB1.srv application ONLINE ONLINE rac2
      ora....TPAY.cs application ONLINE ONLINE rac1
      ora....DB2.srv application ONLINE ONLINE rac2
      ora....VICE.cs application ONLINE ONLINE rac2
      ora....B1.inst application ONLINE ONLINE rac1
      ora....B2.inst application ONLINE ONLINE rac2
      ora.RACDB.db application ONLINE ONLINE rac1
      ora....SM1.asm application ONLINE ONLINE rac1
      ora....C1.lsnr application ONLINE ONLINE rac1
      ora.rac1.gsd application ONLINE ONLINE rac1
      ora.rac1.ons application ONLINE ONLINE rac1
      ora.rac1.vip application ONLINE ONLINE rac1
      ora....SM2.asm application ONLINE ONLINE rac2
      ora....C2.lsnr application ONLINE ONLINE rac2
      ora.rac2.gsd application ONLINE ONLINE rac2
      ora.rac2.ons application ONLINE ONLINE rac2
      ora.rac2.vip application ONLINE ONLINE rac2
      [root@rac2 ~]# su - oracle


      SQL> select INSTANCE_NAME,VERSION,STATUS from gv$instance;

      INSTANCE_NAME     VERSION     STATUS
      ---------------- ----------------- ------------
      RACDB2          10.2.0.1.0     OPEN
      RACDB1          10.2.0.1.0     OPEN

      [oracle@rac2 ~]$ ps -ef|grep pmon
      oracle 14088 1333 0 21:22 pts/1 00:00:00 grep pmon

      [oracle@rac2 ~]$ ps -ef|grep ora
      oracle 714 1 0 10:43 ? 00:00:42 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (LOCAL=NO)
      root 1332 12793 0 21:13 pts/1 00:00:00 su - oracle
      oracle 1333 1332 0 21:13 pts/1 00:00:00 -bash
      root 2540 2497 0 Oct22 ? 00:02:43 hald-addon-storage: polling /dev/sr0
      oracle 5626 1 0 10:06 ? 00:00:52 /u01/app/oracle/product/10.2.0/db_1/bin/racgimon startd RACDB
      oracle 5706 1 0 10:06 ? 00:00:47 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5708 1 0 10:06 ? 00:01:49 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5710 1 0 10:06 ? 00:00:07 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5712 1 0 10:06 ? 00:06:17 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5714 1 0 10:06 ? 00:04:54 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5716 1 1 10:06 ? 00:13:04 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5720 1 0 10:06 ? 00:00:08 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5722 1 0 10:06 ? 00:00:42 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5724 1 0 10:06 ? 00:00:27 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5726 1 0 10:06 ? 00:00:47 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5728 1 0 10:06 ? 00:00:10 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5730 1 0 10:06 ? 00:00:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5732 1 0 10:06 ? 00:01:11 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5734 1 0 10:06 ? 00:00:52 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5736 1 0 10:06 ? 00:00:47 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5738 1 0 10:06 ? 00:00:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5740 1 0 10:06 ? 00:00:01 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5770 1 0 10:06 ? 00:00:50 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5803 1 0 10:07 ? 00:00:09 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5805 1 0 10:07 ? 00:00:15 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
      oracle 5807 1 0 10:07 ? 00:00:07 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 5896 1 0 10:07 ? 00:00:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 6147 1 0 10:07 ? 00:00:01 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 6151 1 0 10:07 ? 00:00:07 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      root 6355 1 0 Oct22 ? 00:00:00 /bin/su -l oracle -c sh -c 'ulimit -c unlimited; cd /u01/app/oracle/product/10.2.0/crs/log/rac2/evmd; exec /u01/app/oracle/product/10.2.0/crs/bin/evmd '
      root 6357 1 0 Oct22 ? 00:12:59 /u01/app/oracle/product/10.2.0/crs/bin/crsd.bin reboot
      oracle 6372 1 0 21:17 ? 00:00:00 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 6374 1 0 21:17 ? 00:00:00 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 6610 6355 0 Oct22 ? 00:00:53 /u01/app/oracle/product/10.2.0/crs/bin/evmd.bin
      root 6694 6608 0 Oct22 ? 00:00:00 /bin/su -l oracle -c /bin/sh -c 'ulimit -c unlimited; cd /u01/app/oracle/product/10.2.0/crs/log/rac2/cssd; /u01/app/oracle/product/10.2.0/crs/bin/ocssd || exit $?'
      oracle 6695 6694 0 Oct22 ? 00:00:00 /bin/sh -c ulimit -c unlimited; cd /u01/app/oracle/product/10.2.0/crs/log/rac2/cssd; /u01/app/oracle/product/10.2.0/crs/bin/ocssd || exit $?
      oracle 6717 6695 0 Oct22 ? 00:08:07 /u01/app/oracle/product/10.2.0/crs/bin/ocssd.bin
      oracle 6852 1 0 10:07 ? 00:00:09 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 6911 1 0 10:07 ? 00:01:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (LOCAL=NO)
      oracle 7067 1 0 10:07 ? 00:00:18 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (LOCAL=NO)
      oracle 7090 6610 0 Oct22 ? 00:00:01 /u01/app/oracle/product/10.2.0/crs/bin/evmlogger.bin -o /u01/app/oracle/product/10.2.0/crs/evm/log/evmlogger.info -l /u01/app/oracle/product/10.2.0/crs/evm/log/evmlogger.log
      oracle 7147 1 0 10:07 ? 00:00:02 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 7149 1 0 10:07 ? 00:00:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 7179 1 0 10:07 ? 00:00:19 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 7214 1 0 10:07 ? 00:00:01 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
      oracle 7287 1 0 10:07 ? 00:00:51 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      oracle 7362 1 0 10:07 ? 00:00:11 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
      oracle 7363 1 0 10:07 ? 00:00:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
      oracle 7386 1 0 10:07 ? 00:00:00 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))
      oracle 9030 1 0 10:08 ? 00:00:47 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (LOCAL=NO)
      oracle 9035 1 0 10:08 ? 00:00:37 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (LOCAL=NO)
      oracle 9037 1 0 10:08 ? 00:01:52 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin (LOCAL=NO)
      oracle 9358 1 0 Oct22 ? 00:00:24 /u01/app/oracle/product/10.2.0/db_1/perl/bin/perl /u01/app/oracle/product/10.2.0/db_1/bin/emwd.pl dbconsole /u01/app/oracle/product/10.2.0/db_1/rac2_RACDB2/sysman/log/emdb.nohup
      oracle 9426 9358 0 Oct22 ? 00:02:58 /u01/app/oracle/product/10.2.0/db_1/bin/emagent
      oracle 14139 1333 0 21:22 pts/1 00:00:00 ps -ef
      oracle 14140 1333 0 21:22 pts/1 00:00:00 grep ora
      oracle 14704 1 0 01:02 ? 00:00:23 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin (LOCAL=NO)
      oracle 17635 1 0 Oct22 ? 00:00:00 /u01/app/oracle/product/10.2.0/crs/opmn/bin/ons -d
      oracle 17637 17635 0 Oct22 ? 00:00:10 /u01/app/oracle/product/10.2.0/crs/opmn/bin/ons -d
      oracle 23505 1 0 09:00 ? 00:00:00 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29197 1 0 Oct22 ? 00:00:25 /u01/app/oracle/product/10.2.0/asm/bin/tnslsnr LISTENER_RAC2 -inherit
      oracle 29837 1 0 00:51 ? 00:01:16 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29839 1 0 00:51 ? 00:03:18 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29841 1 0 00:51 ? 00:00:14 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29843 1 0 00:51 ? 00:08:51 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29845 1 0 00:51 ? 00:08:09 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29847 1 0 00:51 ? 00:08:05 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29851 1 0 00:51 ? 00:00:15 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29853 1 0 00:51 ? 00:00:18 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29855 1 0 00:51 ? 00:00:21 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29857 1 0 00:51 ? 00:00:32 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29859 1 0 00:51 ? 00:00:37 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29861 1 0 00:51 ? 00:00:17 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29863 1 0 00:51 ? 00:00:34 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 29947 1 0 00:51 ? 00:01:01 /u01/app/oracle/product/10.2.0/asm/bin/oracle.bin
      oracle 30200 1 0 00:52 ? 00:00:54 /u01/app/oracle/product/10.2.0/asm/bin/racgimon daemon ora.rac2.ASM2.asm
      oracle 31959 1 0 17:57 ? 00:00:05 /u01/app/oracle/product/10.2.0/db_1/bin/oracle.bin
      [oracle@rac2 ~]$

      Kindly let me know if there are any ways to resolve this ? btw I had to use the below for ASM & Oracle Homes as this is a 10.2.0.1 on OEL5 setup

      mv oracle oracle.bin
      cat >oracle <<"EOF"
      #!/bin/bash
      export DISABLE_HUGETLBFS=1
      exec $ORACLE_HOME/bin/oracle.bin $@
      EOF
      chmod +x oracle

      Is upgrading to 10.2.0.5 the only option ?

      -abhilash
        • 1. Re: Oracle pmon process is not shown
          Iordan Iotzov
          Welcome to the forum,

          It is likely that your pmon processes are in place. You can connect to ASM and DB instances and issue
          select program from v$session
          where type = 'BACKGROUND'
          to check.

          I believe this behavior is a result of redefining the “oracle” executable.

          Is there another way of disabling hugepages?

          Iordan Iotzov
          http://iiotzov.wordpress.com/
          • 2. Re: Oracle pmon process is not shown
            477504
            Thanks,
            Oracle Processes seems to work fine..
            I having a slight issue though connecting to ASM..

            SQL> select program from v$session where type = 'BACKGROUND';

            PROGRAM
            ------------------------------------------------
            oracle@rac1.fugoconsulting.com (q001)
            oracle@rac1.fugoconsulting.com (q000)
            oracle@rac1.fugoconsulting.com (QMNC)
            oracle@rac1.fugoconsulting.com (ARC1)
            oracle@rac1.fugoconsulting.com (ARC0)
            oracle@rac1.fugoconsulting.com (O001)
            oracle@rac1.fugoconsulting.com (RBAL)
            oracle@rac1.fugoconsulting.com (ASMB)
            oracle@rac1.fugoconsulting.com (LCK0)
            oracle@rac1.fugoconsulting.com (MMNL)
            oracle@rac1.fugoconsulting.com (MMON)
            oracle@rac1.fugoconsulting.com (CJQ0)
            oracle@rac1.fugoconsulting.com (RECO)
            oracle@rac1.fugoconsulting.com (SMON)
            oracle@rac1.fugoconsulting.com (CKPT)
            oracle@rac1.fugoconsulting.com (LGWR)
            oracle@rac1.fugoconsulting.com (DBW0)
            oracle@rac1.fugoconsulting.com (MMAN)
            oracle@rac1.fugoconsulting.com (LMS0)
            oracle@rac1.fugoconsulting.com (LMD0)
            oracle@rac1.fugoconsulting.com (LMON)
            oracle@rac1.fugoconsulting.com (PSP0)
            oracle@rac1.fugoconsulting.com (DIAG)
            oracle@rac1.fugoconsulting.com (PMON)

            24 rows selected.

            SQL> exit
            Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - 64bit Production
            With the Partitioning, Real Application Clusters, OLAP and Data Mining options
            *[oracle@rac1 ~]$ export ORACLE_SID=+ASM1*
            *[oracle@rac1 ~]$ export ORACLE_HOME=/u01/app/oracle/product/10.2.0/asm/*
            *[oracle@rac1 ~]$ sqlplus*

            SQLPlus: Release 10.2.0.1.0 - Production on Wed Oct 24 13:16:42 2012*

            Copyright (c) 1982, 2005, Oracle.  All rights reserved.

            Enter user-name: sys as sysdba
            Enter password:
            Connected to an idle instance.

            SQL> exit
            Disconnected
            *[oracle@rac1 ~]$ asmcmd*
            ORA-01034: ORACLE not available
            ORA-27101: shared memory realm does not exist
            Linux-x86_64 Error: 2: No such file or directory (DBD ERROR: OCISessionBegin)
            [oracle@rac1 ~]$ more /home/oracle/.bash_profile
            # .bash_profile

            # Get the aliases and functions
            if [ -f ~/.bashrc ]; then
                 . ~/.bashrc
            fi

            # User specific environment and startup programs

            PATH=$PATH:$HOME/bin
            export PATH

            # Oracle Settings
            TMP=/tmp; export TMP
            TMPDIR=$TMP; export TMPDIR

            ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE
            ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db_1; export ORACLE_HOME
            ORA_ASM_HOME=$ORACLE_BASE/product/10.2.0/asm; export ORA_ASM_HOME
            ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs; export ORA_CRS_HOME
            ORACLE_SID=RACDB1; export ORACLE_SID
            ORACLE_TERM=xterm; export ORACLE_TERM
            PATH=/usr/sbin:$PATH; export PATH
            PATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$ORA_ASM_HOME/bin:$PATH; export PATH

            LD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORA_CRS_HOME/lib:$ORA_ASM_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH
            CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPATH

            if [ $USER = "oracle" ]; then
            if [ $SHELL = "/bin/ksh" ]; then
            ulimit -p 16384
            ulimit -n 65536
            else
            ulimit -u 16384 -n 65536
            fi
            fi
            [oracle@rac1 ~]$

            Any way I can connect to ASM ??

            -abhilash
            • 3. Re: Oracle pmon process is not shown
              Iordan Iotzov
              Try setting the environment like this:
              ORACLE_HOME=/u01/app/oracle/product/10.2.0/asm; export ORACLE_HOME
              ORACLE_SID=+ASM1; export ORACLE_SID
              Iordan Iotzov
              http://iiotzov.wordpress.com/