This discussion is archived
7 Replies Latest reply: Sep 24, 2012 7:26 PM by Kalyan Pasupuleti-Oracle RSS

Weblogic Error(unable to start Admin Server)

887012 Newbie
Currently Being Moderated
"Error Message"
In the linux terminal I tried to start the Weblogic Adminserver. While starting the weblogic server it is showing the following error.


Java HotSpot(TM) Client VM (build 17.0-b17, mixed mode)
Starting WLS with line:
/mwhome/BEA/Middleware/jdk160_21/bin/java -client -Xms768m -Xmx1536m -XX:PermSize=512m -XX:MaxPermSize=1024m -Dweblogic.Name=AdminServer -Djava.security.policy=/mwhome/BEA/Middleware/wlserver_10.3/server/lib/weblogic.policy -Xverify:none -da -Dplatform.home=/mwhome/BEA/Middleware/wlserver_10.3 -Dwls.home=/mwhome/BEA/Middleware/wlserver_10.3/server -Dweblogic.home=/mwhome/BEA/Middleware/wlserver_10.3/server -Dcommon.components.home=/mwhome/BEA/Middleware/oracle_common -Djrf.version=11.1.1 -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.Jdk14Logger -Ddomain.home=/mwhome/BEA/Middleware/user_projects/domains/soa_domain -Djrockit.optfile=/mwhome/BEA/Middleware/oracle_common/modules/oracle.jrf_11.1.1/jrocket_optfile.txt -Doracle.server.config.dir=/mwhome/BEA/Middleware/user_projects/domains/soa_domain/config/fmwconfig/servers/AdminServer -Doracle.domain.config.dir=/mwhome/BEA/Middleware/user_projects/domains/soa_domain/config/fmwconfig -Digf.arisidbeans.carmlloc=/mwhome/BEA/Middleware/user_projects/domains/soa_domain/config/fmwconfig/carml -Digf.arisidstack.home=/mwhome/BEA/Middleware/user_projects/domains/soa_domain/config/fmwconfig/arisidprovider -Doracle.security.jps.config=/mwhome/BEA/Middleware/user_projects/domains/soa_domain/config/fmwconfig/jps-config.xml -Doracle.deployed.app.dir=/mwhome/BEA/Middleware/user_projects/domains/soa_domain/servers/AdminServer/tmp/_WL_user -Doracle.deployed.app.ext=/- -Dweblogic.alternateTypesDirectory=/mwhome/BEA/Middleware/oracle_common/modules/oracle.ossoiap_11.1.1,/mwhome/BEA/Middleware/oracle_common/modules/oracle.oamprovider_11.1.1 -Djava.protocol.handler.pkgs=oracle.mds.net.protocol|oracle.fabric.common.classloaderurl.handler|oracle.fabric.common.uddiurl.handler|oracle.bpm.io.fs.protocol -Dweblogic.jdbc.remoteEnabled=false -da:org.apache.xmlbeans... -Dsoa.archives.dir=/mwhome/BEA/Middleware/Oracle_SOA1/soa -Dsoa.oracle.home=/mwhome/BEA/Middleware/Oracle_SOA1 -Dsoa.instance.home=/mwhome/BEA/Middleware/user_projects/domains/soa_domain -Dtangosol.coherence.clusteraddress=227.7.7.9 -Dtangosol.coherence.clusterport=9778 -Dtangosol.coherence.log=jdk -Djavax.xml.soap.MessageFactory=oracle.j2ee.ws.saaj.soap.MessageFactoryImpl -Dweblogic.transaction.blocking.commit=true -Dweblogic.transaction.blocking.rollback=true -Djavax.net.ssl.trustStore=/mwhome/BEA/Middleware/wlserver_10.3/server/lib/DemoTrust.jks -Dem.oracle.home=/mwhome/BEA/Middleware/oracle_common -Djava.awt.headless=true -Dums.oracle.home=/mwhome/BEA/Middleware/Oracle_SOA1 -Dweblogic.management.discover=true -Dwlw.iterativeDev= -Dwlw.testConsole= -Dwlw.logErrorsToConsole= -Dweblogic.ext.dirs=/mwhome/BEA/Middleware/patch_wls1034/profiles/default/sysext_manifest_classpath:/mwhome/BEA/Middleware/patch_ocp360/profiles/default/sysext_manifest_classpath weblogic.Server
<Sep 8, 2011 5:41:07 PM IST> <Info> <Security> <BEA-090905> <Disabling CryptoJ JCE Provider self-integrity check for better startup performance. To enable this check, specify -Dweblogic.security.allowCryptoJDefaultJCEVerification=true>
<Sep 8, 2011 5:41:07 PM IST> <Info> <Security> <BEA-090906> <Changing the default Random Number Generator in RSA CryptoJ from ECDRBG to FIPS186PRNG. To disable this change, specify -Dweblogic.security.allowCryptoJDefaultPRNG=true>
<Sep 8, 2011 5:41:07 PM IST> <Info> <WebLogicServer> <BEA-000377> <Starting WebLogic Server with Java HotSpot(TM) Client VM Version 17.0-b17 from Sun Microsystems Inc.>
<Sep 8, 2011 5:41:07 PM IST> <Info> <Management> <BEA-141107> <Version: WebLogic Server 10.3.4.0 Fri Dec 17 20:47:33 PST 2010 1384255 >
<Sep 8, 2011 5:41:09 PM IST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to STARTING>
<Sep 8, 2011 5:41:09 PM IST> <Info> <WorkManager> <BEA-002900> <Initializing self-tuning thread pool>
<Sep 8, 2011 5:41:09 PM IST> <Notice> <LoggingService> <BEA-320400> <The log file /mwhome/BEA/Middleware/user_projects/domains/soa_domain/servers/AdminServer/logs/AdminServer.log will be rotated. Reopen the log file if tailing has stopped. This can happen on some platforms like Windows.>
<Sep 8, 2011 5:41:09 PM IST> <Notice> <LoggingService> <BEA-320401> <The log file has been rotated to /mwhome/BEA/Middleware/user_projects/domains/soa_domain/servers/AdminServer/logs/AdminServer.log00033. Log messages will continue to be logged in /mwhome/BEA/Middleware/user_projects/domains/soa_domain/servers/AdminServer/logs/AdminServer.log.>
<Sep 8, 2011 5:41:09 PM IST> <Notice> <Log Management> <BEA-170019> <The server log file /mwhome/BEA/Middleware/user_projects/domains/soa_domain/servers/AdminServer/logs/AdminServer.log is opened. All server side log events will be written to this file.>
Sep 8, 2011 5:41:10 PM oracle.security.jps.internal.credstore.ssp.CsfWalletManager openWallet
WARNING: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
oracle.security.jps.service.credstore.CredStoreException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
at oracle.security.jps.internal.credstore.ssp.CsfWalletManager.openWallet(CsfWalletManager.java:177)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.doInit(SspCredentialStore.java:217)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.<init>(SspCredentialStore.java:139)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.<init>(SspCredentialStore.java:126)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStoreProvider.getInstance(SspCredentialStoreProvider.java:114)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStoreProvider.getInstance(SspCredentialStoreProvider.java:52)
at oracle.security.jps.internal.core.runtime.ContextFactoryImpl.findServiceInstance(ContextFactoryImpl.java:139)
at oracle.security.jps.internal.core.runtime.ContextFactoryImpl.getContext(ContextFactoryImpl.java:170)
at oracle.security.jps.internal.core.runtime.ContextFactoryImpl.getContext(ContextFactoryImpl.java:191)
at oracle.security.jps.internal.core.runtime.JpsContextFactoryImpl.getContext(JpsContextFactoryImpl.java:132)
at oracle.security.jps.internal.core.runtime.JpsContextFactoryImpl.getContext(JpsContextFactoryImpl.java:127)
at oracle.security.jps.internal.policystore.PolicyUtil$1.run(PolicyUtil.java:798)
at oracle.security.jps.internal.policystore.PolicyUtil$1.run(PolicyUtil.java:792)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.internal.policystore.PolicyUtil.getDefaultPolicyStore(PolicyUtil.java:792)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:289)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:282)
at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:261)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1339)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1018)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
at weblogic.security.SecurityService.start(SecurityService.java:141)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:176)
<Sep 8, 2011 5:41:10 PM IST> <Error> <Security> <BEA-090892> <The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider>
<Sep 8, 2011 5:41:10 PM IST> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1398)
at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1018)
at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:873)
at weblogic.security.SecurityService.start(SecurityService.java:141)
at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.JpsRuntimeException: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:291)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:282)
at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:261)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.JpsException: [PolicyUtil] Exception while getting default policy Provider
at oracle.security.jps.internal.policystore.PolicyUtil.getDefaultPolicyStore(PolicyUtil.java:847)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:289)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:282)
at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:261)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
Truncated. see log file for complete stacktrace
Caused By: java.security.PrivilegedActionException: oracle.security.jps.JpsException: [PolicyUtil] Unable to obtain default JPS Context!
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.internal.policystore.PolicyUtil.getDefaultPolicyStore(PolicyUtil.java:792)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:289)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:282)
at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:261)
Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.JpsException: [PolicyUtil] Unable to obtain default JPS Context!
at oracle.security.jps.internal.policystore.PolicyUtil$1.run(PolicyUtil.java:808)
at oracle.security.jps.internal.policystore.PolicyUtil$1.run(PolicyUtil.java:792)
at java.security.AccessController.doPrivileged(Native Method)
at oracle.security.jps.internal.policystore.PolicyUtil.getDefaultPolicyStore(PolicyUtil.java:792)
at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:289)
Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.service.credstore.CredStoreException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.
at oracle.security.jps.internal.credstore.ssp.CsfWalletManager.openWallet(CsfWalletManager.java:177)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.doInit(SspCredentialStore.java:217)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.<init>(SspCredentialStore.java:139)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStore.<init>(SspCredentialStore.java:126)
at oracle.security.jps.internal.credstore.ssp.SspCredentialStoreProvider.getInstance(SspCredentialStoreProvider.java:114)
Truncated. see log file for complete stacktrace
>
<Sep 8, 2011 5:41:10 PM IST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FAILED>
<Sep 8, 2011 5:41:10 PM IST> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
<Sep 8, 2011 5:41:10 PM IST> <Notice> <WebLogicServer> <BEA-000365> <Server state changed to FORCE_SHUTTING_DOWN>

"Hope I'll get a solution soon"
  • 1. Re: Weblogic Error(unable to start Admin Server)
    RenévanWijk Oracle ACE
    Currently Being Moderated
    You probably run the WebLogic once as the super user and are now trying to run it as another.

    you could issue a chown or chmod -R 777

    Also a remark that provided here: Opening of wallet based credential store failed. Reason java.io.IOException

    "It appears the my config/fmwconfig/cwallet.sso file was corrupted, possibly by a previous WebLogic crash. The cwallet.sso file in my fmwconfig directory was only 77bytes, but it should have been larger. I was able to find a previous version of the cwallet.sso file in the /tmp/cred* directories, and restored the file. That allowed me to start up WebLogic, but I had lost some data in cwallet.sso that I needed to re-enter in FMW Control."
  • 2. Re: Weblogic Error(unable to start Admin Server)
    skt Newbie
    Currently Being Moderated
    Can you add the following system property to the startup script "-Doracle.pki.debug=true" and attempt to restart the server. This should produce more information that ought to tell which wallet file was being accessed and what was the real cause.
    There could be several causes
    - user that the weblogic is starting as doesn't have sufficient file access permissions to domain and bootstrap wallet files. Typically happens when software is installed and domain created using User1 and server is started as User2.
    - domain directory is in NFS mounted partition that doesn't support file locking
    - cwallet.sso was potentially created using a different version of the software and there is some version incompatibility.

    Either way, the debug flag mentioned above will help.
  • 3. Re: Weblogic Error(unable to start Admin Server)
    HypSvc Newbie
    Currently Being Moderated
    replaceing cwallet.sso file under config folder should resolved the issue.
  • 4. Re: Weblogic Error(unable to start Admin Server)
    432524 Oracle ACE
    Currently Being Moderated
    The innermost error says:
    Caused By: oracle.security.jps.service.credstore.CredStoreException: JPS-01050: Opening of wallet based credential store failed. Reason java.io.IOException: PKI-02002: Unable to open the wallet. Check password.

    It seems somehow that your cwallet.sso is the reason for your error.
    Can you try to use the keytool or orapki to figure out if your wallet is still OK? Both tools offer basic wallet management possibilities. So you can see if you can open your wallet (e.g. is your password correct), which certificates are in there and so on.

    Based on this information you will probably find more possibilities to dig deeper and solve your problem.

    cu
    Andreas
  • 5. Re: Weblogic Error(unable to start Admin Server)
    903894 Newbie
    Currently Being Moderated
    Hi All,

    We could resolve this issue.
    What we did was .. deleted the existing Admin security folder ( or back it up) (/domain/server/your_servername/security)
    Then we started the server without the nohup and &( we run the startWeblogic.sh script to start and not the Nodemanager). While starting , it prompted for the credentials.( Give here the original credentials only)
    Admin server came up with no issues.
    Looks like some other user has tried to bring up the Admin server without having related access.
    Hence security credentials might have got corrupted.
    Before this I tried restoring a good copy of system_jazn file from the original copy. But still that dint work.

    Hope this helps you too,
    Madhav
  • 6. Re: Weblogic Error(unable to start Admin Server)
    767663 Newbie
    Currently Being Moderated
    Also, try this step to see if this works

    sudo chmod -R 777 /tmp

    I retrieved the solution from here: https://kr.forums.oracle.com/forums/thread.jspa?threadID=1078487
  • 7. Re: Weblogic Error(unable to start Admin Server)
    Kalyan Pasupuleti-Oracle Expert
    Currently Being Moderated
    Hi,


    Look for following file system-jazn-data.xml whether this is empty or not.

    you will find this one <domain>/config/fmwconfig/system-jazn-data.xml.

    There isn't a tool/utility that allows to re-generate this file from scratch.

    This file can be copied from any of the managed servers in the same domain.

    If there isn't any other copy in the domain it must be recovered from recent backup.


    Regards,
    Kal

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points