1 Reply Latest reply on Nov 24, 2012 8:30 AM by 746690

    Manage servers with virtual IPs/ Host names won't start

      Hello everyone,

      I have installed a load balanced OSB cluster and a load balanced SOA (10.3.6) on two separate servers.

      - Admin SOA server (runs on server #1)
      - Admin OSB server (runs on server #2)

      I'm using virtual host names for my managed SOA and OSB servers, as follows :

      - Managed OSB server #1 : OSBHOST1VHN - ip : xxx.xxx.xxx.xxx (runs on server #1)
      - Managed OSB server #2 : OSBHOST2VHN - ip : yyy.yyy.yyy.yyy (runs on server #2)

      - Managed SOA server #1 : SOAHOST1VHN - ip : www.www.www.www (runs on server #1)
      - Managed SOA server #2 : SOAHOST2VHN - ip : zzz.zzz.zzz.zzz (runs on server #2)

      - Managed BAM server #1 : BAMHOST1VHN - ip : nnn.nnn.nnn.nnn (runs on server #1)
      - Managed BAM server #2 : ip : zzz.zzz.zzz.zzz - no virtual IP name here (runs on server #2)

      I was able to start/stop the admin and managed server with node manager so far on each server.
      Everything worked just fine until we decided to stop the machines to add more memory.
      After the reboot, the admin servers started but not the managed servers.
      Well, all the managed servers with vIPs failed but the BAM server #2 started without any problem.

      For the other managed servers, I get the following errors in the log files.

      ####<20 nov. 2012 15 h 56 GMT> <Error> <ALSB Statistics Manager> <svlvsoa08.vlv.arvato.fr> <ms_prd_osb2> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <9583013bf964d764:32a0a378:13b1e86ef73:-8000-0000000000000010> <1353426960019> <BEA-473003> <Serveur d'agrégation non disponible. Failed to get remote aggregator
      java.rmi.UnknownHostException: Could not discover URL for server 'ms_prd_osb1'
      at weblogic.protocol.URLManager.findURL(URLManager.java:145)
      at com.bea.alsb.platform.weblogic.topology.WlsRemoteServerImpl.getInitialContext(WlsRemoteServerImpl.java:120)
      at com.bea.alsb.platform.weblogic.topology.WlsRemoteServerImpl.lookupJNDI(WlsRemoteServerImpl.java:76)
      at com.bea.wli.monitoring.statistics.ALSBStatisticsManager.getRemoteAggregator(ALSBStatisticsManager.java:291)
      at com.bea.wli.monitoring.statistics.ALSBStatisticsManager.access$000(ALSBStatisticsManager.java:38)
      at com.bea.wli.monitoring.statistics.ALSBStatisticsManager$RemoteAggregatorProxy.send(ALSBStatisticsManager.java:55)
      at com.bea.wli.monitoring.statistics.collection.Collector.sendRawSnaphotToAggregator(Collector.java:284)
      at com.bea.wli.monitoring.statistics.collection.Collector.doCheckpoint(Collector.java:243)
      at com.bea.wli.monitoring.statistics.collection.Collector$CheckpointThread.doWork(Collector.java:68)
      at com.bea.wli.monitoring.utils.Schedulable.timerExpired(Schedulable.java:68)
      at com.bea.wli.timer.ClusterTimerImpl$InternalTimerListener.timerExpired(ClusterTimerImpl.java:254)
      at weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)
      at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:531)
      at weblogic.work.ExecuteThread.execute(ExecuteThread.java:252)
      at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)

      ####<20 nov. 2012 15 h 56 GMT> <Emergency> <Security> <svlvsoa08.vlv.arvato.fr> <ms_prd_osb2> <[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <9583013bf964d764:32a0a378:13b1e86ef73:-8000-0000000000000015> <1353426963376> <BEA-090087> <Server failed to bind to the configured Admin port. The port may already be used by another process.>
      ####<20 nov. 2012 15 h 56 GMT> <Error> <Server> <svlvsoa08.vlv.arvato.fr> <ms_prd_osb2> <DynamicListenThread[Default]> <<WLS Kernel>> <> <9583013bf964d764:32a0a378:13b1e86ef73:-8000-0000000000000016> <1353426963376> <BEA-002606> *<Unable to create a server socket for listening on channel "Default". The address might be incorrect or another process is using port 7000: java.net.BindException: Cannot assign requested address.>####<20 nov. 2012 15 h 56 GMT> <Critical> <WebLogicServer> <svlvsoa08.vlv.arvato.fr> <ms_prd_osb2> <main> <<WLS Kernel>> <> <9583013bf964d764:32a0a378:13b1e86ef73:-8000-0000000000000012> <1353426963377> <BEA-000362> <Server failed. Reason: Server failed to bind to any usable port. See preceeding log message for details.>####<20 nov. 2012 15 h 56 GMT> <Error> <WebLogicServer> <svlvsoa08.vlv.arvato.fr> <ms_prd_osb2> <main> <<WLS Kernel>> <> <9583013bf964d764:32a0a378:13b1e86ef73:-8000-0000000000000012> <1353426963387> <BEA-000383> <A critical service failed. The server will shut itself down>

      I tried to ping my vHOSTS :
      ping OSBHOST2VHN
      I manage to resolve my vIPs, so I assume they are working properly.
      I checked the etc/hosts and everything is still correctly configured.
      I also checked config.xml (which I haven't modified at all) and everything seems correctly set (host, servers, IP, ports, ...).

      The only that changed since the server rebooted is the setDomainEnv.sh in which I modified Xms and Xms values to increase the heap size.
      I only changed this on one server to check if the settings we properly applied.

      Can anyone help me with this ?

      Edited by: user13619168 on 21 nov. 2012 08:09

      Edited by: user13619168 on 21 nov. 2012 08:10

      Edited by: user13619168 on 21 nov. 2012 08:16

      Edited by: user13619168 on 21 nov. 2012 08:17