Oracle BI wsm-pm and obisch1 does not start, not sure what and which caused which to stop first?

vfan

Current setup: Windows 7 /BI 11.1.1.16/Oracle 11G

1) The first thing I noticed is the unable to login to Analytics even if I have the correct username and password.

2) logged in EM page and found BI/COREAPPLICATION is down, it shows the BIPUBLISHER and oracleRTD are both down

3) went to check opmnctl status and it said OBISCH1 is "down" (all others are alive) or sometimes it showed "none"

4) Logged into CONSOLE page, and found wsm-pm and oracleRTD are both "failed".

Things I tried.

1) restarted the OPMNCTL multiple times, but results were all the same.  OBISCH1 is down.

2) EM page error shows the below.  It seems many are failing and I don't know what and which casued which to fail?

Message: An error occurred for port: {http://oracle/bi/security/}SecurityServicePort: oracle.fabric.common.PolicyEnforcementException: PolicySet Invalid: WSM-06102 PolicyReference The policy reference URI "oracle/wss_username_token_service_policy" is not valid.

Message: Servlet: "PolicyManagerValidator" failed to preload on startup in Web application: "/wsm-pm".

oracle.adf.share.ADFShareException: MDSConfigurationException encountered in parseADFConfiguration

Message:  MDSConfigurationException encountered in parseADFConfigurationMDS-01330: unable to load MDS configuration document   

Supplemental Detail    MDS-01329: unable to load element "persistence-config"

MDS-01370: MetadataStore configuration for metadata-store-usage "OWSM_TargetRepos" is invalid.   

MDS-01377: Unable to get database connection from data source configured with JNDI name "jdbc/mds/owsm".    

weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified

oracle.mds.config.MDSConfigurationException: MDS-01330: unable to load MDS configuration document   

MDS-01329: unable to load element "persistence-config"

Message:  Authentication Failure.

Message: Processing Event: SEVERE: Element Type: BI_INSTANCE, Element Id: coreapplication, Operation Result: FAILED_TO_START, Detail Message: Error in starting one or more components of BI Instance; Please check the server log files for detailed message.

Message: Supplementary information regarding operation: DOMAIN;FAILED_TO_START;Failed to start J2EE BI component: AdminServer:bipublisher#11.1.1

Message: Unable to get list of partitions for: mds-owsm

Supplemental Detail    oracle.mds.persistence.MDSIOException: MDS-01377: Unable to get database connection from data source configured with JNDI name "jdbc/mds/owsm".    

weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified

at oracle.mds.persistence.stores.db.AbstractDBMetadataRepository.getAllPartitionNames(AbstractDBMetadataRepository.java:328)

MDS-01370: MetadataStore configuration for metadata-store-usage "OWSM_TargetRepos" is invalid.   

MDS-01377: Unable to get database connection from data source configured with JNDI name "jdbc/mds/owsm".    

weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified

3) I went into CONSOLE and update the WSM-PM and start "servicing all requests", but error shows:

weblogic.application.ModuleException: [HTTP:101216]Servlet: "PolicyManagerValidator" failed to preload on startup in Web application: "/wsm-pm". oracle.adf.share.ADFShareException: MDSConfigurationException encountered in parseADFConfiguration at oracle.adf.share.config.ADFMDSConfig.parseADFConfiguration(ADFMDSConfig.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at oracle.adf.share.config.ADFConfigImpl.getResultFromComponent(ADFConfigImpl.java:497) at oracle.adf.share.config.ADFConfigImpl.getConfigObject(ADFConfigImpl.java:568) at oracle.adf.share.config.ADFConfigImpl.getConfigObject(ADFConfigImpl.java:546) at oracle.adf.share.config.ADFConfigImpl.getMDSInstance(ADFConfigImpl.java:611) at oracle.adf.share.config.ADFConfigImpl.getMDSInstance(ADFConfigImpl.java:606) at oracle.adf.share.config.ADFContextMDSConfigHelperImpl.getMDSInstance(ADFContextMDSConfigHelperImpl.java:277) at oracle.adf.share.ADFContext.getMDSInstanceAsObject(ADFContext.java:1674) at oracle.wsm.repository.mds.MDSInstanceFactory.getMDSInstance(MDSInstanceFactory.java:98) at oracle.wsm.policymanager.bean.AbstractBean.getRepository(AbstractBean.java:350) at oracle.wsm.policymanager.bean.AbstractBean.createSession(AbstractBean.java:233) at oracle.wsm.policymanager.bean.UpgradeManagerBean.seedWSMPolicyRepository(UpgradeManagerBean.java:588) at oracle.wsm.policymanager.web.PolicyManagerValidator.init(PolicyManagerValidator.java:113) at weblogic.servlet.internal.StubSecurityHelper$ServletInitAction.run(StubSecurityHelper.java:283) at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120) at weblogic.servlet.internal.StubSecurityHelper.createServlet(StubSecurityHelper.java:64) at weblogic.servlet.internal.StubLifecycleHelper.createOneInstance(StubLifecycleHelper.java:58) at weblogic.servlet.internal.StubLifecycleHelper.<init>(StubLifecycleHelper.java:48) at weblogic.servlet.internal.ServletStubImpl.prepareServlet(ServletStubImpl.java:539) at weblogic.servlet.internal.WebAppServletContext.preloadServlet(WebAppServletContext.java:1985) at weblogic.servlet.internal.WebAppServletContext.loadServletsOnStartup(WebAppServletContext.java:1959) at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1878) at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:3153) at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1508) at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:482) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119) at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:200) at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:247) at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119) at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:27) at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:636) at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52) at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:205) at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:58) at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161) at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79) at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:569) at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:150) at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:116) at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:140) at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:323) at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:844) at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1253) at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:440) at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:163) at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195) at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13) at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68) at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528) at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209) at weblogic.work.ExecuteThread.run(ExecuteThread.java:178) Caused by: oracle.mds.config.MDSConfigurationException: MDS-01330: unable to load MDS configuration document MDS-01329: unable to load element "persistence-config" MDS-01370: MetadataStore configuration for metadata-store-usage "OWSM_TargetRepos" is invalid. MDS-01377: Unable to get database connection from data source configured with JNDI name "jdbc/mds/owsm". weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified at oracle.mds.config.PConfig.loadFromBean(PConfig.java:959) at oracle.mds.config.PConfig.<init>(PConfig.java:758) at oracle.mds.config.MDSConfig.loadFromBean(MDSConfig.java:787) at oracle.mds.config.MDSConfig.loadFromElement(MDSConfig.java:848) at oracle.mds.config.MDSConfig.<init>(MDSConfig.java:491) at oracle.mds.core.MDSInstance.getMDSConfigFromDocument(MDSInstance.java:2690) at oracle.mds.core.MDSInstance.findAndStoreMDSInstanceWithCustConfig(MDSInstance.java:2523) at oracle.mds.core.MDSInstance.getOrCreateInstanceInternal(MDSInstance.java:2031) at oracle.mds.core.MDSInstance.getOrCreateInstance(MDSInstance.java:740) at oracle.adf.share.config.ADFMDSConfig.parseADFConfiguration(ADFMDSConfig.java:139) ... 59 more Caused by: oracle.mds.exception.MDSExceptionList: MDS-01329: unable to load element "persistence-config" MDS-01370: MetadataStore configuration for metadata-store-usage "OWSM_TargetRepos" is invalid. MDS-01377: Unable to get database connection from data source configured with JNDI name "jdbc/mds/owsm". weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified at oracle.mds.config.PConfig.loadFromBean(PConfig.java:953) ... 68 more :oracle.mds.exception.MDSExceptionList:MDS-01329: unable to load element "persistence-config" MDS-01370: MetadataStore configuration for metadata-store-usage "OWSM_TargetRepos" is invalid. MDS-01377: Unable to get database connection from data source configured with JNDI name "jdbc/mds/owsm". weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified

4) checked database and it is running.  I can log in to it.  Pinging server is good, but TSNPING the SID shows this error:

TNS-12545: Connect failed because target host or object does not exist

I checked the LISTENER is listening, and TNSNAMES.ORA is correct.

I have googled about others having such issues, but almost all pointed to the password expired & locked.  However, this is not my case.  I verified my username/password is not expired nor locked.

I appreciate any pointers.  Please let me know if I could provide more information to resolve this problem.

BTW, not sure if this is related.  The box was once shut down for 6 months.  Once it booted up, it ran out of disk space.  I have added space to it and leave it untouched for another 3 months.  I just recently logged into it to do so update.

Thank you very much.

Madasamy -Oracle

Hello,

For the issue on wsm-pm, please go to the console --> Data sources, and check if each data source is able to establish the JDBC connection to the target database, particularly the WSM schema.

This should resolve the issue ""jdbc/mds/owsm". weblogic.common.resourcepool.ResourceDeadException: 0:weblogic.common.ResourceException: Could not create pool connection. The DBMS driver exception was: IO Error: Unknown host specified"

and try to restart the services again.

We should see no errors on the AdmiNServer log regarding the data source connections and that would lead to some help

- Madasamy