1 Reply Latest reply: Apr 2, 2012 10:22 AM by 927646 RSS

    rac node failure crs cleanup failing

    927646
      I have a three node rac database, 10.2.0.4 running on Windows server 2008. I lost a hard drive on one of the servers and it corrupted the mirror disk as well so I am having to rebuild. I am going through these procedures, RAC on Windows: How to Cleanup When A Node Has Been Disconnected or The OS Rebuilt (Doc ID 742737.1) and am running into a problem once I tried to delete the listener and then on to crs to delete the nodeapps for node3.

      For the listener, I go into netca and the option to delete a listener is grayed out. When I run crs_stat I can still see the ora.node3.lsnr there. Does this mean that I just need to update tnsnames.ora or is there another place the information would be held? I hate to manually delete because I am afraid I won't get it cleaned out from everywhere. Any idea why that option would not be there?

      My second issue is when I run this:
      srvctl stop nodeapps -n node3
      The nodeapps stop doesn't return any output and then when I try to remove nodeapps it gives me PRKO-2112: Some or all node applications are not removed successfully on node.

      I have searched metalink for that error with no success as the document I found also says that you must stop nodeapps. I have already deleted the node from the db and asm and updated the appropriate inventory. I just need to finish the listener and crs and update the inventory for crs. Also, I noticed that the vip for the failed node was reassigned to node2 and I show that it has been released when i run cluvfy to check. Would crs give me errors on this if that was not the case?

      I appreciate any help or guidance!
        • 1. Re: rac node failure crs cleanup failing
          927646
          Wanted to post a follow up in case any others are interested in the results...

          I had tried to add the listener back to one of the remaining nodes .ora file and then delete but that didn't work. Also, remove nodeapps continued to throw an error that it could not stop the listener or vip for the failed node.

          After a few days of reading I make a decision to just unregister the abandoned services from crs. I made sure to backup the OCRCONFIG before I ran crs_unregister and was able to successfully remove the listener and vip services from the failed node.

          This eliminated my issue with netca, the node did not show up there anymore. I then went on to remove nodeapps and it failed saying it could not find the resource vip. I then ran olsnodes -n and used crssetup to remove the node entirely. Everything showed removed and I went and updated the crs inventory to finish.

          All looks good and now I am working to rebuild and add the node back in.