2 Replies Latest reply: Mar 5, 2013 4:45 AM by 963732 RSS

    ORA-00304: requested INSTANCE_NUMBER is busy

      Hi guys,

      Need help to fix the subject problem, what I did is to add a new node to an existing cluster with the following steps:

      1. To add clusterware on the new node :
      [xml]./addNode.sh -silent "CLUSTER_NEW_NODES={Sol4}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={Sol4-vip}" "CLUSTER_NEW_PRIVATE_NODE_NAMES={Sol4-priv}"[xml]

      2. To add database softwre to the new node :
      [xml]./addNode.sh -silent "CLUSTER_NEW_NODES={Sol4}"[xml]

      3. To create an instance for the new node :
      [xml]dbca -silent -addInstance -nodeList Sol4 -gdbName Sols.localdomain -instanceName Sols2 -sysDBAUserName sys -sysDBAPassword ************[xml]

      srvctl config database -d sols
      [xml]Database unique name: Sols
      Database name:
      Oracle home: /u01/app/oracle/product/11.2.0/dbhome_1
      Oracle user: oracle
      Spfile: +DGDATA/sols/spfilesols.ora
      Domain: localdomain
      Start options: open
      Stop options: immediate
      Database role: PHYSICAL_STANDBY
      Management policy: AUTOMATIC
      Server pools: Sols
      Database instances: Sols1,Sols2
      Disk Groups: DGDATA,DGFRA
      Mount point paths:
      Type: RAC
      Database is administrator managed[xml]

      Here is the output from ADRCI for the alerts
      [xml]2013-03-05 14:10:06.986000 +05:00
      Starting ORACLE instance (normal)
      2013-03-05 14:10:08.551000 +05:00
      2013-03-05 14:10:15.605000 +05:00
      Private Interface 'net4:1' configured from GPnP for use as a private interconnect.
      [name='net4:1', type=1, ip=, mac=00-50-56-3a-a2-f7-00-00-00-00-00-00-00-00-2f-00-00-00-00-00, net=, mask=,

      Public Interface 'net0' configured from GPnP for use as a public interface.
      [name='net0', type=1, ip=, mac=00-50-56-30-5b-35-00-00-00-00-00-00-00-00-2f-00-00-00-00-00, net=, mask=,

      Picked latch-free SCN scheme 3
      Autotune of undo retention is turned on.
      SYS auditing is disabled
      2013-03-05 14:10:16.909000 +05:00
      Starting up:
      Oracle Database 11g Enterprise Edition Release - 64bit Production
      With the Partitioning, Real Application Clusters, Oracle Label Security, OLAP,
      Data Mining, Oracle Database Vault and Real Application Testing options.
      ORACLE_HOME = /u01/app/oracle/product/11.2.0/dbhome_1
      System name:SunOS
      Node name:Sol4
      Using parameter settings in server-side pfile /u01/app/oracle/product/11.2.0/dbhome_1/dbs/initSols2.ora
      System parameters with non-default values:
      processes = 150
      spfile = "+DGDATA/sols/spfilesols.ora"
      memory_target = 900M
      control_files = "+DGDATA/sols/controlfile/current.260.808586739"
      control_files = "+DGFRA/sols/controlfile/current.259.808586739"
      db_file_name_convert = "+DATA/Sol"
      db_file_name_convert = "+DGDATA/Sols"
      db_file_name_convert = "+FRA/Sol"
      db_file_name_convert = "+DGFRA/Sols"
      db_file_name_convert = "+DATA/sol"
      db_file_name_convert = "+DGDATA/sols"
      db_file_name_convert = "+FRA/sol"
      db_file_name_convert = "+DGFRA/sols"
      log_file_name_convert = "+DATA/Sol"
      log_file_name_convert = "+DGDATA/Sols"
      log_file_name_convert = "+FRA/sol"
      log_file_name_convert = "+DGFRA/sols"
      db_block_size = 8192
      compatible = ""
      log_archive_dest_1 = "location=USE_DB_RECOVERY_FILE_DEST"
      log_archive_dest_1 = "valid_for=(ALL_LOGFILES, ALL_ROLES)"
      log_archive_dest_2 = "service="sol""
      log_archive_dest_2 = "LGWR ASYNC NOAFFIRM delay=0 optional compression=disable max_failure=0 max_connections=1 reopen=300

      db_unique_name="Sol" net_timeout=30"
      log_archive_dest_2 = "valid_for=(all_logfiles,primary_role)"
      log_archive_dest_state_2 = "ENABLE"
      log_archive_min_succeed_dest= 1
      fal_client = "Sols"
      fal_server = "sol"
      log_archive_config = "dg_config=(Sol,Sols)"
      log_archive_format = "%t_%s_%r.dbf"
      log_archive_max_processes= 8
      archive_lag_target = 0
      cluster_database = TRUE
      db_create_file_dest = "+DGDATA"
      db_recovery_file_dest = "+DGFRA"
      db_recovery_file_dest_size= 15000M
      standby_file_management = "AUTO"
      instance_number = 1 <<<--------------<<<<<----
      remote_login_passwordfile= "EXCLUSIVE"
      db_domain = "localdomain"
      dispatchers = "(PROTOCOL=TCP) (SERVICE=SolsXDB)"
      remote_listener = "sols-scan:1521"
      audit_file_dest = "/u01/app/oracle/admin/Sols/adump"
      audit_trail = "DB"
      db_name = "Sol"
      db_unique_name = "Sols"
      open_cursors = 300
      dg_broker_start = TRUE
      dg_broker_config_file1 = "+DGDATA/sols/dr1sols.dat"
      dg_broker_config_file2 = "+DGDATA/sols/dr2sols.dat"
      diagnostic_dest = "/u01/app/oracle"
      Cluster communication is configured to use the following interface(s) for this instance
      cluster interconnect IPC version:Oracle UDP/IP (generic)
      IPC Vendor 1 proto 2
      2013-03-05 14:10:42.602000 +05:00
      PMON started with pid=2, OS id=18601
      PSP0 started with pid=3, OS id=18603
      2013-03-05 14:10:44.019000 +05:00
      VKTM started with pid=4, OS id=18609 at elevated priority
      VKTM running at (1)millisec precision with DBRM quantum (100)ms
      GEN0 started with pid=5, OS id=18613
      DIAG started with pid=6, OS id=18615
      DBRM started with pid=7, OS id=18617
      PING started with pid=8, OS id=18619
      ACMS started with pid=9, OS id=18621
      DIA0 started with pid=10, OS id=18624
      2013-03-05 14:10:45.003000 +05:00
      LMON started with pid=11, OS id=18626
      LMD0 started with pid=12, OS id=18628
      * Load Monitor used for high load check
      * New Low - High Load Threshold Range = [1920 - 2560]
      LMS0 started with pid=13, OS id=18630 at elevated priority
      RMS0 started with pid=14, OS id=18634
      LMHB started with pid=15, OS id=18636
      MMAN started with pid=16, OS id=18638
      2013-03-05 14:10:46.022000 +05:00
      DBW0 started with pid=17, OS id=18641
      LGWR started with pid=18, OS id=18643
      CKPT started with pid=19, OS id=18645
      SMON started with pid=20, OS id=18647
      RECO started with pid=21, OS id=18649
      RBAL started with pid=22, OS id=18651
      ASMB started with pid=23, OS id=18654
      MMON started with pid=24, OS id=18656
      2013-03-05 14:10:47.119000 +05:00
      starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
      MMNL started with pid=25, OS id=18660
      starting up 1 shared server(s) ...
      NOTE: initiating MARK startup
      Starting background process MARK
      DISM started, OS id=18667
      MARK started with pid=28, OS id=18668
      2013-03-05 14:10:48.236000 +05:00
      NOTE: MARK has subscribed
      2013-03-05 14:11:03.704000 +05:00
      USER (ospid: 18546): terminating the instance due to error 304
      2013-03-05 14:11:05.121000 +05:00
      Instance terminated by USER, pid = 18546[xml]

      As you can see its using the instance number = 1 for the new instance which is already running on the other node.

      What I would like to know :

      1. Is it a bug in oracle ? shouldn't it have created 2nd instance with a different instance number ?
      2. How do I fix it now ?

      Thank you very much for your help in advance.

      Best regards.