4 Replies Latest reply: Nov 13, 2013 1:49 AM by Anar Godjaev RSS

    ORA-29701: unable to connect to Cluster Synchronization Service

    EdStevens

      Oracle GI 11.2.0.2.0, standalone server, no cluster no RAC

      Oracle Linux 5.6 x-64

       

      Building a new sandbox system under VBox to mimic one of my production servers.  Have done this many times before, and currently have a total of 7 other VM's on my Win7 Pro desktop. (not all running concurrently)

      It has been my habit in the past that when shutting down one of these machines I don't bother shutting down the db's or ASM instance, I just have vbox send and ACPI shutdown.  Of course that causes the systems to have a shutdown abort, but for my purposes that has never caused an issue.  I would NOT do that on a live system.

       

      With this system I decided to start doing that the right way.  I got the ASM instance built (still no databsases), added a second DG with two failgroups, modified a few init parms to more closely align with the live system.  Shutdown and startup from sqlplus works fine.  But when I shutdown the ASM instance and then send the machine an ACPI shutdown, the next time I start the machine, the ASM instance will not restart.  Instead I get

       

       

      oracle:+ASM$ sqlplus / as sysasm

       

      SQL*Plus: Release 11.2.0.2.0 Production on Fri Sep 27 13:09:26 2013

       

      Copyright (c) 1982, 2010, Oracle.  All rights reserved.

       

      Connected to an idle instance.

       

      SQL> startup

      ORA-01078: failure in processing system parameters

      ORA-29701: unable to connect to Cluster Synchronization Service

      SQL>

       

      It appears "has" started ok - if I try to start it manually, it tells me it is already active, and as a side observation, it did automatically start the listener.

       

      As a side note that may or may not bear on this, the spfile is not on the ASM disk, but is in $ORACLE_HOME/dbs.  This is how the live system is configured.

       

      Am willing to do some digging, but need some direction.

        • 1. Re: ORA-29701: unable to connect to Cluster Synchronization Service
          EdStevens

          Aha!

           

          Immediate solution was 'crsctl start resource'.

           

          I knew that with oracle restart, if I manually stopped a particular service (such as a db or asm instance), that would be reflected in restart, and it would not attempt to auto restart that particular service until such time as I manually restarted it.  But I didn't think stopping the asm instance would cause the cssd service to also be affected.

          • 2. Re: ORA-29701: unable to connect to Cluster Synchronization Service
            Levi Pereira

            Hi EdStevens,

             

            With Oracle 11.2 some resources have auto start policy set to restore, which means that GI will remember the last state of the resource. If the resource was stopped normally then on the next restart of clusterware it won’t be started. Otherwise if the server crashes or by some reason the OS is rebooted then clusterware will start the database because last state was ONLINE (running).

             

            By default with Oracle 11.2 several important resources come in the profile with attribute AUTO_START=restore, so if Oracle database server is restarted for some reason , it will keep and restore the last state.


            The resource ora.cssd has the attribute AUTO_START=never by default and the resouce ora.asm has the attribute AUTO_START=restore by default.

             

            That means which ora.cssd will never start does not matter what was last state of this resource.

             

            So, How and when ora.cssd start?

            It's ora.asm which have attribute START_DEPENDENCIES=hard(ora.cssd).

            So, if the ora.asm  resource don't start at startup (for some reason) the resource ora.cssd will stay down until manually start it or start ora.asm resource.

             

            Check it:

            $ crsctl stat res ora.cssd -p |grep AUTO_START

            $ crsctl stat res ora.asm -p |grep AUTO_START

            $ crsctl stat res ora.asm -p |grep START_DEPENDENCIES

             

            If you wish you can change it. (Is not supported by Oracle, unless if you have a SR)

             

            $ crsctl modify resource ora.cssd -attr AUTO_START=always

            But I don't see any problem doing this.

             

             

            Regards,

            Levi Pereira

            • 3. Re: ORA-29701: unable to connect to Cluster Synchronization Service
              Jacob Morake

              Why would I get the message:

              ERROR: unrecoverable error ORA-29701 raised in ASM I/O path; terminating process 6898

               

              in the alert.log and still have all services (css, evm, has)  running and available?

               

              We are running Oracle 11.2.0.3.0 on Red Hat Enterprise Linux Server release 5.5 (Tikanga)  x86_64

              • 4. Re: ORA-29701: unable to connect to Cluster Synchronization Service
                Anar Godjaev

                Hi,

                 

                Could you please check OTN form discussions: https://forums.oracle.com/thread/2384764?start=0&tstart=0    (Maybe this link help you)

                 

                Thank you