Can't start OBIPS, OBISCH & OBIS on 2nd node after horizontal scaling

Adam Wickes

Hi all,

As you may have seen in my other threads, I have had nothing but trouble scaling our 12c environment from a fresh install.
I have used a combination of the following threads and am now facing an issue whereby 3 components of the 2nd biserver will not start.

OBIEE 12c - Horizontal Scaling Part 2 - Red Stack Tech

https://www.peakindicators.com/blog/installing-an-oracle-bi-12c-cluster

In each of the above steps, it talks about using the bi_clone_machine to be able to create a .jar file that can then be migrated to the 2nd server and unpacked.
In my install, the bi_clone_machine step fails saying that the -domain "bla" is an invalid argument.
Instead of attempting to run the bi_clone_machine again, I manually packed the .jar file using pack.cmd and then continued the steps by unpacking, running the midtier sync and then starting the two servers in the appropriate order.

I am not left with the following:

Logs on DLVOBI04 are as follows:

OBIPS - sawlog0.log

[2018-01-04T19:21:11+11:00] [OBIPS] [NOTIFICATION:1] [] [saw.sawserver] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Oracle BI Presentation Services (OBIPS) 12.2.1.3.0 (Build BIPS-20170820114118 64-bit) are starting up.[[File:sawserver.cppLine:670Location:  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:11+11:00] [OBIPS] [NOTIFICATION:1] [] [saw.sawserver] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Platform: Microsoft Windows 8 Server -- on Host -- DLVOBI04Version: 6.2.9200Host: DLVOBI04[[File:sawserver.cppLine:682Location:  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:11+11:00] [OBIPS] [WARNING:1] [] [saw.dmsintegration.dmswrapperapi.loaddmslibrary] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] DMS initialization - an error has occurred while loading : "loadLib".[[File:dmswrapperapi.cppLine:192Location:  saw.dmsintegration.dmswrapperapi.loaddmslibrary  saw.dmsintegration.dmswrapperapi.activate  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:11+11:00] [OBIPS] [NOTIFICATION:1] [] [saw.csf.cache] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Using credential store provider configured in jps configuration file 'D:\oracle\OBI\Middleware\Oracle_home\user_projects\domains\bi\config\fmwconfig\jps-config-jse.xml'.[[File:csfcredentialstore.cppLine:122Location:  saw.csf.cache  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:11+11:00] [OBIPS] [WARNING:16] [] [saw.csf.cache] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] OFM security reported error 43131[[File:csfwrapperimpl.cppLine:213Location:  saw.csf.cache  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:11+11:00] [OBIPS] [ERROR:1] [] [saw.sawserver] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] OFM security reported error 43131[[File:csfwrapperimpl.cppLine:213Location:  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:12+11:00] [OBIPS] [NOTIFICATION:1] [] [saw.nqscheduler] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Found valid endpoint to be DLVOBI03.bbldtl.int 9508[[File:rpcoptionsreader.cppLine:1007Location:  saw.nqscheduler  saw.webextensionbase.init  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:12+11:00] [OBIPS] [NOTIFICATION:1] [] [saw.nqscheduler] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Found valid endpoint to be 10.254.177.13 9508[[File:rpcoptionsreader.cppLine:1007Location:  saw.nqscheduler  saw.webextensionbase.init  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:12+11:00] [OBIPS] [WARNING:16] [] [saw.csf.cache] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] OFM security reported error 43131[[File:csfwrapperimpl.cppLine:213Location:  saw.csf.cache  saw.nqscheduler  saw.webextensionbase.init  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:12+11:00] [OBIPS] [ERROR:10] [] [saw.webextensionbase.shutdown] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Assertion failure: false at line 18 of handle.cppMessage: class saw::NQSchedulerManager * __ptr64[[File:handle.cppLine:18Location:  saw.webextensionbase.shutdown  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:12+11:00] [OBIPS] [WARNING:1] [] [saw.webextensionbase.shutdown] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Unexpected exception occured during WebExtension shutdown.[[File:webextensionbase.cppLine:1251Location:  saw.webextensionbase.shutdown  saw.sawserver  saw.sawserver.initializesawserver  saw.sawserver]][2018-01-04T19:21:12+11:00] [OBIPS] [NOTIFICATION:1] [] [saw.sawserver] [ecid: ] [tid: ] [SI-Name: ] [IDD-Name: ] [IDD-GUID: ] [userId: ] Oracle BI Presentation Services are shutting down.[[File:sawserver.cppLine:1031Location:  saw.sawserver  saw.sawserver.terminatesawserver  saw.sawserver]]

OBISCH - nqscheduler.log

[2018-01-04T19:21:55.748+11:00] [OBISCH] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: f80] [2018-01-04T19:21:55.748+11:00] [OBISCH] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: f80] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:21:55.748+11:00] [OBISCH] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: f80] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:21:55.748+11:00] [OBISCH] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: f80] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:21:55.826+11:00] [OBISCH] [NOTIFICATION:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68031] Scheduler process starting with pid 4240[2018-01-04T19:21:55.826+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: driver - dms wrapper initialization[2018-01-04T19:21:55.826+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: driver - register with OPMN[2018-01-04T19:21:55.842+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: driver - initialize scheduler[2018-01-04T19:21:55.842+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: Read credential store[2018-01-04T19:21:55.904+11:00] [OBISCH] [ERROR:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [46137] CSF error encountered. Error code: 43131. [[file: server\Utility\Generic\Src\SUGCSF.cpp; line: 133]][2018-01-04T19:21:55.904+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: Initialize SchScheduler failed[2018-01-04T19:21:55.904+11:00] [OBISCH] [ERROR:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [46137] CSF error encountered. Error code: 43131.[2018-01-04T19:21:55.904+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: driver - error before starting RPC Server[2018-01-04T19:21:55.904+11:00] [OBISCH] [ERROR:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68008] Scheduler Error: [46137] CSF error encountered. Error code: 43131.[2018-01-04T19:21:55.904+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: driver - stop RPC server on error[2018-01-04T19:21:55.920+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: SchScheduler destructor[2018-01-04T19:21:55.920+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: driver - ~SchLibLoader dms shutdown [2018-01-04T19:21:55.920+11:00] [OBISCH] [TRACE:1] [] [] [ecid: ] [sik: service1] [tid: f80]  [68032] Scheduler step: Driver - completed

OBIS - obis2-diagnostic.log

[2018-01-04T19:20:00.299+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] DB Max Execution Time::0 [[DBGateway Max Data Size::0DBGateway Max Rows::0DBGateway Max Threads::200Listen Port::9514MAXIMUM_FILE_HANDLES::503Machine::DLVOBI04.bbldtl.intMax Session Limit::2000Monitor Port::9515Number of Processors::4ORACLE_BI_HOME::D:\oracle\OBI\Middleware\Oracle_home\biORACLE_BI_LANG::enORACLE_COMPONENT::D:\oracle\OBI\Middleware\Oracle_home\user_projects\domains\bi\servers\obis2Repositories::{Star=liverpd.rpd}SQLBypass  Max Threads::200Server Max Threads::100Server Thread Stack Size::1048576Session TimeOut(Secs)::60Unique Identifier::obis2]][2018-01-04T19:20:00.314+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:20:00.314+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] Reading ServiceInstanceKey from file: \\dlcfil01\obiee$\bidata\components\OBIS\instance_mapping\obis2\mapping.xml [[file: server\Utility\Server\Config\SUSystemConfig.cpp; line: 952]][2018-01-04T19:20:00.330+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] Found service instance key: ssi for component:obis2 [[file: server\Utility\Server\Config\SUSystemConfig.cpp; line: 1116]][2018-01-04T19:20:00.330+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:20:00.330+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:20:00.330+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] LCMAPIsUtil.DoesSIMetadataNeedRefresh-> called with token 2018-01-04T17:57:42.003+11:00 : False[2018-01-04T19:20:00.346+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:20:00.455+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0]  [94598] Failed to locate the md5 hash file "D:\oracle\OBI\Middleware\Oracle_home\user_projects\domains\bi\tmp\ssi\datamodel\customizations\md5hash.bin" stored during last upload to data store for File System Mode.[2018-01-04T19:20:00.455+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0]  [94599] Datastore updated by external utility since last check-in. Refresh needs to be done for File System Mode.[2018-01-04T19:20:00.471+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ] [tid: fd0] LCMAPIsUtil.GetSDD-> \\dlcfil01\obiee$\bidata[2018-01-04T19:20:03.17+11:00] [OBIS] [ERROR:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [46137] CSF error encountered. Error code: 43131. [[file: server\Utility\Generic\Src\SUGCSF.cpp; line: 133]][2018-01-04T19:20:03.17+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47103] DDPATH value is  /oracle/OBI/Middleware/Oracle_home/bi/modules/oracle.bi.datadirect.odbc/8.0.2.[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT Btrieve (*.dta) DriverVersion:Oracle 5.3 32-BIT Btrieve (*.dta).[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT DB2 Wire Protocol DriverVersion:Oracle 5.3 32-BIT DB2 Wire Protocol.[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT FoxPro 3.0 Database (*.dbc) DriverVersion:Oracle 5.3 32-BIT FoxPro 3.0 Database (*.dbc).[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT Informix Wire Protocol DriverVersion:Oracle 5.3 32-BIT Informix Wire Protocol.[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT MySQL Wire Protocol DriverVersion:Oracle 5.3 32-BIT MySQL Wire Protocol.[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT Oracle Wire Protocol DriverVersion:Oracle 5.3 32-BIT Oracle Wire Protocol.[2018-01-04T19:20:03.33+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT ParadoxFile (*.db) DriverVersion:Oracle 5.3 32-BIT ParadoxFile (*.db).[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT SQL Server Wire Protocol DriverVersion:Oracle 5.3 32-BIT SQL Server Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT Sybase Wire Protocol DriverVersion:Oracle 5.3 32-BIT Sybase Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT TextFile (*.*) DriverVersion:Oracle 5.3 32-BIT TextFile (*.*).[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT XML DriverVersion:Oracle 5.3 32-BIT XML.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:32-BIT dBASEFile (*.dbf) DriverVersion:Oracle 5.3 32-BIT dBASEFile (*.dbf).[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Amazon RedShift Wire Protocol DriverVersion:Oracle 7.1 Amazon RedShift Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Amazon Redshift Wire Protocol DriverVersion:Oracle 7.1 Amazon Redshift Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Apache Hive Wire Protocol DriverVersion:Oracle 8.0 Apache Hive Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Apache Spark SQL DriverVersion:Oracle 8.0 Apache Spark SQL.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Cassandra DriverVersion:Oracle 8.0 Cassandra.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:DB2 Wire Protocol DriverVersion:Oracle 7.1 DB2 Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Greenplum Wire Protocol DriverVersion:Oracle 7.1 Greenplum Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Hive Wire Protocol DriverVersion:Oracle 7.1 Hive Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Impala Wire Protocol DriverVersion:Oracle 7.1 Impala Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Informix Wire Protocol DriverVersion:Oracle 7.1 Informix Wire Protocol.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:MongoDB DriverVersion:Oracle 8.0 MongoDB.[2018-01-04T19:20:03.48+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:MySQL Wire Protocol DriverVersion:Oracle 7.1 MySQL Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:OpenEdge Wire Protocol DriverVersion:Oracle 7.1 OpenEdge Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Oracle DriverVersion:Oracle 7.1 Oracle.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Oracle Service Cloud Wire Protocol DriverVersion:Oracle 7.1 Oracle Service Cloud Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Oracle Wire Protocol DriverVersion:Oracle 8.0 Oracle Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:PostgreSQL Wire Protocol DriverVersion:Oracle 7.1 PostgreSQL Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:SQL Server Legacy Wire Protocol DriverVersion:Oracle 7.1 SQL Server Legacy Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:SQL Server Wire Protocol DriverVersion:Oracle 7.1 SQL Server Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Salesforce DriverVersion:Oracle 7.1 Salesforce.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Sybase IQ Wire Protocol DriverVersion:Oracle 7.1 Sybase IQ Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Sybase Wire Protocol DriverVersion:Oracle 7.1 Sybase Wire Protocol.[2018-01-04T19:20:03.64+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [47104] DriverName:Teradata DriverVersion:Oracle 7.1 Teradata.[2018-01-04T19:20:03.80+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0] RPC Server: bind success with address :: port 9514 family AF_INET6 protocol TCP with dual stacks[2018-01-04T19:20:03.111+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0] LCMAPIsUtil.GetPreparedRPDPath-> [2018-01-04T19:20:03.111+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0] SUSystemConfig::GetConfiguredBaseRPDPath for : ssi is: \oracle\OBI\Middleware\Oracle_home\user_projects\domains\bi\tmp\ssi\datamodel\customizations\liverpd.rpd[2018-01-04T19:20:03.127+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [13079] Started reading RPD credentials for key metadata.encryption from CSF.[2018-01-04T19:20:03.127+11:00] [OBIS] [ERROR:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [46137] CSF error encountered. Error code: 43131. [[file: server\Utility\Generic\Src\SUGCSF.cpp; line: 133]][2018-01-04T19:20:03.158+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [14058] Unloaded all subject areas.[2018-01-04T19:20:04.205+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0]  [14058] Unloaded all subject areas. [[Properties: EnvShutdownCall=RpMgr::ShutDown(); EnvShutdownCallFileLine=server\NQSNative\NQSServEntry\Source\NQSQueryEnvironment.cpp:1468; ]][2018-01-04T19:20:04.205+11:00] [OBIS] [ERROR:1] [] [] [ecid: ] [sik: ssi] [tid: fd0] Oracle BI Server startup failed.[46137] CSF error encountered. Error code: 43131. [[file: server\NQSNative\Driver\Src\NQDDriverImpl.cpp; line: 488]][2018-01-04T19:20:04.205+11:00] [OBIS] [NOTIFICATION:1] [] [] [ecid: ] [sik: ssi] [tid: fd0] nqsserver: Oracle BI Server shutdown. [[file: server\NQSNative\Driver\Src\NQDDriverImpl.cpp; line: 66]]

The general theme through the logs is the "CSF error" but I can't find anything that talks about a fix.
Does anybody have any advice that may be able to set this right?
I feel like i'm super close now and really don't want to have deinstall and start again.... again!

Thanks in advance,
Adam

Adam Wickes

Environment is:

Windows 2012 server R2 (64bit)
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
Java 1.8.0_111

EDIT: Another thing I should mention is that I had to rename "simplefan.jar" on both nodes to avoid ONS issues.

Adam Wickes

Worked it out.

The answer was that I had to run sync_midtier_db.cmd on both nodes (while the nodeManager was running).