4 Replies Latest reply on Jul 27, 2010 2:21 PM by alpyanar

    I cannot start weblogic due to IOException

    gregauv
      Hello,
      Can you tell me how to sole this problem. After starting weblogic , I got an exception :

      *weblogic.diagnostics.lifecycle.DiagnosticComponentLifecycleException: weblogic.store.PersistentStoreException: java.io.IOException: [Store:280021]There was an error while opening the file store file "WLS_DIAGNOSTICS000000.DAT"*

      bash-3.00$ ./startWebLogic.sh
      .
      .
      JAVA Memory arguments: -Xms256m -Xmx512m -XX:CompileThreshold=8000 -XX:PermSize=48m -XX:MaxPermSize=128m
      .
      WLS Start Mode=Development
      .
      CLASSPATH=/product/gemaltovalidation/3rdps/bea/patch_wls1032/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/product/gemaltovalidation/3rdps/java/jdk1.6.0_21/lib/tools.jar:/product/gemaltovalidation/3rdps/bea/utils/config/10.3/config-launch.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/weblogic.jar:/product/gemaltovalidation/3rdps/bea/modules/features/weblogic.server.modules_10.3.2.0.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/webservices.jar:/product/gemaltovalidation/3rdps/bea/modules/org.apache.ant_1.7.0/lib/ant-all.jar:/product/gemaltovalidation/3rdps/bea/modules/net.sf.antcontrib_1.0.0.0_1-0b2/lib/ant-contrib.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/common/eval/pointbase/lib/pbclient57.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/xqrl.jar
      .
      PATH=/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/bin:/product/gemaltovalidation/3rdps/bea/modules/org.apache.ant_1.7.0/bin:/product/gemaltovalidation/3rdps/java/jdk1.6.0_21/jre/bin:/product/gemaltovalidation/3rdps/java/jdk1.6.0_21/bin:/opt/java1.6/bin:/opt/oracle/10.2.0/bin:/opt/java1.5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/quest/bin
      .
      ***************************************************
      * To start WebLogic Server, use a username and *
      * password assigned to an admin-level user. For *
      * server administration, use the WebLogic Server *
      * console at http://hostname:port/console *
      ***************************************************
      starting weblogic with Java version:
      java version "1.6.0_21"
      Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
      Java HotSpot(TM) Client VM (build 17.0-b16, mixed mode, sharing)
      Starting WLS with line:
      /product/gemaltovalidation/3rdps/java/jdk1.6.0_21/bin/java -client -Xms256m -Xmx512m -XX:CompileThreshold=8000 -XX:PermSize=48m -XX:MaxPermSize=128m -Dweblogic.Name=AdminServer -Djava.security.policy=/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/product/gemaltovalidation/3rdps/bea/wlserver_10.3 -Dwls.home=/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server -Dweblogic.home=/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/product/gemaltovalidation/3rdps/bea/patch_wls1032/profiles/default/sysext_manifest_classpath weblogic.Server
      <Jul 23, 2010 3:28:10 PM MEST> <Notice> <WebLogicServer> <BEA-000395> <Following extensions directory contents added to the end of the classpath:
      /product/gemaltovalidation/wls/domain/gemalto_domain/lib/commons-logging-1.1.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/eps-event-core-api.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/eps-ws-client.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/gemalto-domain-logging-startup.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/log4j.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/oaps-api.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/smsgw-api.jar:/product/gemaltovalidation/wls/domain/gemalto_domain/lib/smsgw-driver-api.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/wlcommons-logging.jar:/product/gemaltovalidation/3rdps/bea/wlserver_10.3/server/lib/wllog4j.jar>
      <Jul 23, 2010 3:28:10 PM MEST> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) Client VM Version 17.0-b16 from Sun Microsystems Inc.>
      <Jul 23, 2010 3:28:11 PM MEST> <Info> <Management> <BEA-141107> <Version: WebLogic Server 10.3.2.0 Tue Oct 20 12:16:15 PDT 2009 1267925 >
      <Jul 23, 2010 3:28:13 PM MEST> <Info> <Management> <BEA-141227> <Making a backup copy of the configuration at /product/gemaltovalidation/wls/domain/gemalto_domain/config-original.jar.>
      <Jul 23, 2010 3:28:13 PM MEST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
      <Jul 23, 2010 3:28:13 PM MEST> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
      <Jul 23, 2010 3:28:13 PM MEST> <Notice> <LoggingService> <BEA-320400> <The log file /product/gemaltovalidation/wls/domain/gemalto_domain/servers/AdminServer/logs/AdminServer__yyyy_MM_dd_hh_mm.log will be rotated. Reopen the log file if tailing has stopped. This can happen on some platforms like Windows.>
      <Jul 23, 2010 3:28:13 PM MEST> <Notice> <LoggingService> <BEA-320401> <The log file has been rotated to /product/gemaltovalidation/wls/domain/gemalto_domain/servers/AdminServer/logs/AdminServer__2010_07_23_03_28.log00010. Log messages will continue to be logged in /product/gemaltovalidation/wls/domain/gemalto_domain/servers/AdminServer/logs/AdminServer__yyyy_MM_dd_hh_mm.log.>
      <Jul 23, 2010 3:28:13 PM MEST> <Notice> <Log Management> <BEA-170019> <The server log file /product/gemaltovalidation/wls/domain/gemalto_domain/servers/AdminServer/logs/AdminServer__%yyyy%_%MM%_%dd%_%hh%_%mm%.log is opened. All server side log events will be written to this file.>
      <Jul 23, 2010 3:28:13 PM MEST> <Critical> <WebLogicServer> <BEA-000362> <Server failed. Reason:
      There are 1 nested errors:
      weblogic.diagnostics.lifecycle.DiagnosticComponentLifecycleException: weblogic.store.PersistentStoreException: java.io.IOException: [Store:280021]There was an error while opening the file store file "WLS_DIAGNOSTICS000000.DAT"
      at weblogic.diagnostics.lifecycle.ArchiveLifecycleImpl.initialize(ArchiveLifecycleImpl.java:66)
      at weblogic.diagnostics.lifecycle.DiagnosticFoundationService.start(DiagnosticFoundationService.java:107)
      at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
      at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
      at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
      Caused by: weblogic.store.PersistentStoreException: java.io.IOException: [Store:280021]There was an error while opening the file store file "WLS_DIAGNOSTICS000000.DAT"
      at weblogic.store.io.file.Heap.open(Heap.java:185)
      at weblogic.store.io.file.FileStoreIO.open(FileStoreIO.java:88)
      at weblogic.store.internal.PersistentStoreImpl.recoverStoreConnections(PersistentStoreImpl.java:332)
      at weblogic.store.internal.PersistentStoreImpl.open(PersistentStoreImpl.java:323)
      at weblogic.store.xa.PersistentStoreManagerXA.createFileStore(PersistentStoreManagerXA.java:104)
      at weblogic.store.xa.PersistentStoreManagerXA.createFileStore(PersistentStoreManagerXA.java:177)
      at weblogic.diagnostics.archive.DiagnosticStoreRepository.getStore(DiagnosticStoreRepository.java:85)
      at weblogic.diagnostics.lifecycle.ArchiveLifecycleImpl.initialize(ArchiveLifecycleImpl.java:64)
      ... 4 more
      Caused by: java.io.IOException: [Store:280021]There was an error while opening the file store file "WLS_DIAGNOSTICS000000.DAT"
      at weblogic.store.io.file.StoreDir.throwIOException(StoreDir.java:214)
      at weblogic.store.io.file.StoreDir.open(StoreDir.java:104)
      at weblogic.store.io.file.Heap.open(Heap.java:183)
      ... 11 more
      Caused by: java.io.IOException: Error from fcntl() for file locking, Resource temporarily unavailable, errno=11
      at weblogic.store.io.file.direct.DirectIONative.open(Native Method)
      at weblogic.store.io.file.direct.DirectFileChannel.<init>(DirectFileChannel.java:52)
      at weblogic.store.io.file.direct.DirectIOManager.open(DirectIOManager.java:139)
      at weblogic.store.io.file.StoreFile.openInternal(StoreFile.java:118)
      at weblogic.store.io.file.StoreFile.open(StoreFile.java:148)
      at weblogic.store.io.file.StoreDir.open(StoreDir.java:96)
      ... 12 more
      >
      ***************************************************************************
      The WebLogic Server encountered a critical failure
      Reason: Assertion violated
      ***************************************************************************
      Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class weblogic.protocol.ServerIdentityManager$Initializer
      at weblogic.protocol.ServerIdentityManager.findServerIdentity(ServerIdentityManager.java:64)
      at weblogic.protocol.URLManager.findAdministrationURL(URLManager.java:172)
      at weblogic.server.ServerLifeCycleRuntime.getLifeCycleOperationsRemote(ServerLifeCycleRuntime.java:801)
      at weblogic.t3.srvr.ServerRuntime.sendStateToAdminServer(ServerRuntime.java:423)
      at weblogic.t3.srvr.ServerRuntime.updateRunState(ServerRuntime.java:409)
      at weblogic.t3.srvr.T3Srvr.setState(T3Srvr.java:187)
      at weblogic.t3.srvr.T3Srvr.run(T3Srvr.java:458)
      at weblogic.Server.main(Server.java:67)

      Thanks in advance
      Grégory
        • 1. Re: I cannot start weblogic due to IOException
          690392
          delete the "tmp" and "data" directories in the managed servers and try to start it again.


          Regards,
          Harsha
          • 2. Re: I cannot start weblogic due to IOException
            486763
            The diagnostics store is not transactional, it is written at the OS discretion.

            The exception is ultimately caused by java.io.IOException: Error from fcntl() for file locking, Resource temporarily unavailable, errno=11.

            Make sure (using an OS process list) that there is no accidentally started WLS instance of this server.

            If you ruled out another running instance of WLS and If your domain is stored on NFS, you might have run into a dangling lock issue after an NFS client crash. It is safe to get rid of the diagnostics store DAT files to recover from your situation (remove <DOMAIN>/servers/<SERVER>/data/store/diagnostics/WLS_DIAGNOSTICS<FILENUM>.DAT). If it were a store that contains useful data, such as the default store, custom JMS Server File Store, etc, we recommend copying this file within the same dir to some tmp name (e.g., <original-name>.copy), delete the original file, and rename <original-name>.copy back to <original-name>.
            • 3. Re: I cannot start weblogic due to IOException
              786714
              Hai,

              I also faced the same issue in your error is the weblogic admin console port is already using the existing java object. You can use the below common to find out the background java port number in the linux environment.

              ( The same port number is using the existing objects )

              Example:

              [root@localhost bin]# lsof -i
              java 3917 root 472u IPv6 106757 TCP localhost.sis.ad:afs3-callback->localhost.sis.ad:42939 (ESTABLISHED)
              java 3917 root 535u IPv6 106743 TCP localhost.localdomain:afs3-callback (LISTEN)
              java 3917 root 536u IPv6 106744 TCP localhost.sis.ad:afs3-callback (LISTEN)
              java 3917 root 537u IPv6 106745 TCP localhost:afs3-callback (LISTEN)

              In your admin port number shows in the below list means then you can give the other port number then only you can start your Admin server in weblogic 10.3.3.

              Regards,
              S.Vinoth Babu
              • 4. Re: I cannot start weblogic due to IOException
                alpyanar
                Which user do you use to start weblogic? And does this user have needed privileges to open .DAT file?
                Please check the permissions.