This discussion is archived
4 Replies Latest reply: May 26, 2013 1:31 AM by oracle.dba.89 RSS

Faced an issue while setting up RAC on my virtual machine

oracle.dba.89 Newbie
Currently Being Moderated
Hi All,

While setting up RAC 10g on VMWare with Cent OS 4, i ran into a few issues which were solved by a reboot (last resort)

The problem was after installing the database, while checking the cluster resources using crs_stat -t, the status was showing offline for 2 resources.
[oracle@rac1 bin]$ ./crs_stat -t
Name           Type           Target    State     Host
ora....SM1.asm application    ONLINE    OFFLINE   rac1
ora....C1.lsnr application    ONLINE    OFFLINE   rac1
ora.rac1.gsd   application    ONLINE    ONLINE    rac1
ora.rac1.ons   application    ONLINE    ONLINE    rac1   application    ONLINE    ONLINE    rac1
ora....SM2.asm application    ONLINE    ONLINE    rac2
ora....C2.lsnr application    ONLINE    ONLINE    rac2
ora.rac2.gsd   application    ONLINE    ONLINE    rac2
ora.rac2.ons   application    ONLINE    ONLINE    rac2   application    ONLINE    ONLINE    rac2
The +ASM1 instance and listener on RAC1 was not started when I checked. So i started them manually.
But while starting the instance, i got an error
WARNING: cache failed to read fn=3  indblk=0 from disk(s): 2
ORA-15196: invalid ASM block header [kfc.c:7910] [endian_kfbh] [3] [2147483648] [0 != 1]
NOTE: a corrupted block was dumped to the trace file
System State dumped to trace file /u01/app/oracle/admin/+ASM/bdump/+asm1_lgwr_12518.trc
NOTE: cache initiating offline of disk 2  group 1
WARNING: offlining disk 2.3912953495 (DATA_0002) with mask 0x3
NOTE: PST update: grp = 1, dsk = 2, mode = 0x6
Wed Sep 12 22:43:44 2012
ERROR: too many offline disks in PST (grp 1)
Wed Sep 12 22:43:44 2012
NOTE: PST not enabling heartbeating (grp 1): group dismounted
I was unable to dismount and remount the disk. It was giving an error saying
ORA-15130: diskgroup "DISK" is being dismounted
First of all can anyone tell me what "ora....SM1.asm" and "ora....C1.lsnr" are? Why was the TARGET column showing as online?

I also tried shutting down the resources and then restarting the rawdevice services. But after doing that, while trying to start the resources (crsctl start resources) the command was just hung on both the instances.

I then didn't know what to do and just rebooted both the machines and then saw that everything was up. I managed to create the database and everything is running fine now.

I didn't know where else to look for errors. Can anyone help me figure out the issue please?

Also now after setting everything up, I see the below error in the ASM alert log for both nodes.
Errors in file /u01/app/oracle/admin/+ASM/bdump/+asm1_rbal_9071.trc:
ORA-15183: ASMLIB initialization error [/opt/oracle/extapi/32/asm/orcl/1/]
ORA-15183: ASMLIB initialization error [driver/agent not installed]
I checked oracle support doc ID 1054033.1 and tried checking the permissions. But I don't see the file asmadmin present in $ORACLE_HOME/bin

From the support doc

1. Manually change the oracle binary ownership and permission:
as root user:
# cd $ORACLE_HOME/bin
# chgrp asmadmin oracle
# chmod 6751 oracle
# ls -l oracle
  • 1. Re: Faced an issue while setting up RAC on my virtual machine
    damorgan Oracle ACE Director
    Currently Being Moderated
    I only read your first sentence ... let me repeat what I think I just read.
    While setting up RAC using a version of Oracle currently in desupport in a virtual environment never certified or supported by Oracle 
    with an operating system never certified or supported by Oracle, you ran into a few issues.
    did I miss anything?

    My recommendation, without reading past what I just parodied above, is that you format the drive, install Oracle Enterprise Linux, Oracle VM, and or above following the installation documentation published by Oracle Corp.

    Then if you have issues people like me will read your entire post.

    My point is not to be rude but rather the point out what should be obvious. If you don't follow the docs, don't use supported software and configurations, and have problems it is essentially impossible for us to know what might have gone wrong. We can guess ... but we can not do more than that.
  • 2. Re: Faced an issue while setting up RAC on my virtual machine
    945629 Newbie
    Currently Being Moderated

    I agree with damorgan.

    we should follow Oracle Docs guideline and use only Cerrtified OS for Production and other environment.

    Edited by: 942626 on Sep 12, 2012 11:47 PM
  • 3. Re: Faced an issue while setting up RAC on my virtual machine
    Rajesh.Rathod Explorer
    Currently Being Moderated

    Need to start servuces in sequence.

    can u stop service properly as follows and make sure they are stopped properly by checking status using crscrtl.

    (1) stop db services ,instance services
    (2) stop asm
    (3) stop nodeapps services
    (4) stop CRS SERVICES

    now start services in seqeunce.

    (1) start CRS services
    (2) start nodeapp services
    (3) start asm
    (4) start instance ,db services

    Edited by: Rajesh.Rathod on Sep 12, 2012 11:50 PM

    Edited by: Rajesh.Rathod on Sep 12, 2012 11:51 PM

    Edited by: Rajesh.Rathod on Sep 13, 2012 2:51 AM
  • 4. Re: Faced an issue while setting up RAC on my virtual machine
    oracle.dba.89 Newbie
    Currently Being Moderated
    I am gonna find a supported virtual solution for this and set up everything on that :)

    I'll close this thread for now


  • Correct Answers - 10 points
  • Helpful Answers - 5 points