Skip to Main Content

Analytics Software

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Interested in getting your voice heard by members of the Developer Marketing team at Oracle? Check out this post for AppDev or this post for AI focus group information.

Applications Not Appearing in Workspace (9.3.1)

638106May 7 2008 — edited Aug 2 2008
Hello all,

We recently upgraded to system 9.3.1 and intermittently users are not seeing the applications when logging into Workspace.

Example: USER A logs in to Workspace, is able to perform all necessary duties by logging on to the application for days, sometimes weeks at a time. Suddenly, USER A logs in, and Workspace appears as if there are no applications registered at all. The next day USER A logs in and everything is back to normal.

Can anybody give me a clue as to where to look to find the source of this?

I have a sneaking suspicion it is because the applications were not properly de-registered from the previous version (System 9.3.0.1) which had completely different server names as the current system 9.3.1.

If that is the case, those references would be located in the tables in the main HFM database, but what, or should i even, modify/delete those references?

Thanks for any help,
Kelly

Comments

631379 May 7 2008
Is SharedServices or something else being restarted? If all of the server pieces are not restarted in the correct order strange things can happen.
638106 May 7 2008
I restart services (when necessary) in this order:

Shared Services/BI+ Core Server:
Open ldap service (native directory)
Shared Services (I run task manager and monitor the process, then wait until I can bring shared services up in a browser)
Core Services

BI+ Servers:
Apache service (usually already started)
Java RMI Registry
Workspace (in the same manner as SS, I wait until workspace can be brought up in a browser to start the following services)
Financial Reporting Report Server
Financial Reporting Scheduler Server
Financial Reporting Print Server
Financial Reporting Web application
Web Analysis service

Is there anything you can see wrong with my process?
602270 May 8 2008
Nothing wrong with your the startup order. Once OpenLDAP/HSS and Core are started everything else should be ok to start in more or less any order. The only difference I would make (purely logical) would be to start the "Hyperion S9 BI+ 9.3 Financial Reporting Java RMI Registry" just before the other freports services since these are all related. There is no dependency on Reports services and workspace startup so you do not really need to wait for Workspace to start before starting the FR services...anyway that is a different story.

The app list in workspace is generated by shared services. When the list does not appear have a look in the log: <hyperion home>\logs\server_messages_GSM.log are you getting messages like "Error in getting contents of Product Registration file"? Can you copy paste any errors for us to see?

When the list does not appear:

Can you log in to HSS form the BI+ Servers and from the Shared Services/BI+ Core Server:?
Can you drill down on your hfm projects?
Can you do through the motions of configuring role access?

How many BI+Servers do you have? If you have multiple Workspace/FR servers does it depend on which server the user is logged into as to whether or not the list appears?

Which web application are you using? Tomcat/Weblogic/Websphere?
638106 May 8 2008
cassidy,

many thanks for your response! i have looked at the logs you suggested and found below response in the server_messages_GSM.log.

Also, when registering an application in Workspace under consolidation administration tasks, the Financial Management Web Server URL for Security Administration URL defaults to this: httx://hyperion.intranet.com:19000/hfm

the only problem with that is that it is pointing to BI+ boxes that do not have HFM server installed, though they do have the hfm client files.

Is there a file that tells me what URL an application is registered to use for this information?

When the list does not appear have a look in the log: <hyperion home>\logs\server_messages_GSM.log are you getting messages like "Error in getting contents of Product Registration file"? Can you copy paste any errors for us to see?
<event logger="com.brio.one.services.globalservicemanager.impl.ProductRegInfoImpl" timestamp="1210236806252"
level="ERROR" thread="[ORB=_it_orb_id_1,Pool=1]::id-12" sequence_no="1289">
<time>08 May 2008 03:53:26,252</time>
<context originator_type="GSM" originator_name="GlobalServiceManager" host="SRVHOUHFM09"/>
<message><![CDATA[getProductRegFile(): HSS Error while retrieving registration information. Consult HSS logs for details.]]></message>
</event>

This message occurs frequently, I just posted one of them, they are all basically the same.

Can you log in to HSS form the BI+ Servers and from the Shared Services/BI+ Core Server:?
Yes

Can you drill down on your hfm projects?
Yes

Can you go through the motions of configuring role
access?

Yes

How many BI+Servers do you have?
2

If you have multiple Workspace/FR servers does it depend on which server the user is logged into as to whether or not the list appears?
No. We do have Microsoft's version of load balancing installed and have set up a VIP so all the users have to do is go to httx://hyperion.intranet.com, I then have a redirect set up which forwards all requests made to that URL to httx://hyperion.intranet.com:19000/workspace.

This is a place of possible concern as I have always pushed for the hardware solution for load balancing.

Which web application are you using? Tomcat/Weblogic/Websphere?
Weblogic Server Express 9.2 on the SS box and Weblogic Server Express 8.1 on the Reports boxes, which has always been a concern of mine from the beginning. When we performed the installation, the Oracle consultant on hand said it would not be a problem, and my knowledge of Java application servers is limited, so I went with his OK.

638106 May 8 2008
Also to be noted: When this problem does occur, if i restart the BI+ server services, the problem persists, but if I restart the HSS services, everything is fine until a certain time, which may have something to do with the MSAD refresh time, not sure though. but restarting the HSS services dos the trick for quick relief of the problem.


Checking the SharedServices_Security log, I find this message repeatedly:
2008-05-07 17:55:04,081 [[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] WARN com.hyperion.css.spi.impl.nv.NativeProvider.getHierarchicalRoleTree(Ljava.util.Map;Ljava.lang.String;Lcom.hyperion.css.common.CSSRoleNode;Ljava.lang.String;Lcom.hyperion.css.spi.util.jndi.CSSDirContext;Ljava.util.Locale;Ljava.util.ResourceBundle;)V(Unknown Source) - Exception getting Child Roles in hierarchy due to Illegal or invalid id.dflt passed in. Please check the argument.
602270 May 12 2008
Excellent restarting HSS fixes the issue, so we can maybe assume that HSS is the problem....

There is a fix for an HSS issue which could very well be related. The fix is HSS 9.3.1.0.3 ... and amongst other things the readme says:

The Shared Services User Management Console experienced intermittent issues with displaying the list of projects. Some projects were not shown in the project list after a period of time. There were also delays in displaying the list of projects.

I would consider installing this patch and see if the problem is resolved.
638106 May 13 2008
cassidy i apologize for not responding to your help sooner, I did not receive a notice in my email this time for some reason or i would have responded much sooner!

so you're saying the problem i am having may be related to a known issue... i will check on that immediately and get back soon with my results!

thank you so much for your advice!
638106 May 15 2008
I found the readme you were referring to, but it states that projects are sometimes not showing in SS, which is not our issue. The projects are always there, it's just the application that disappears...

I ran the WebDAV utility and looked over the properties of the registered applications and it appears that one of our applications, the one not showing, was registered using a URL that references our Reports boxes. When registering an application, the Hyperion Financial Management URL for security should be our HFM server, correct? My theory is the reason it works some of the time is that we have the HFM client installed on both of our BI+ Reports boxes, so it has the /hfm directory, just not all of the necessary files that a server would have, just a hunch...

Would re-registering an application to the proper HFM url erase all provisioning for it?
602270 May 20 2008
re-registering an hfm application should not wipe the provisioning.

The security url httx://hyperion.intranet.com:19000/hfm
should be ok, this is a mod_proxy redirect that gets setup in apache which redirects all /hfm requests to the real HFM IIS server, If you want to check the redirect information it can be found in the file:
<hyperion home>\common\httpServers\Apache\2.0.52\conf\hfm.conf

If there is a server specific problem you should be able to get the error by logginging on to each web server individually. One will work the ther will not.
638106 May 21 2008
The error is only occurring on our HSS/Core Services box.

One of the log files for BIPLUS on the HSS box grows to 4GB, cuts off then starts creating another one of itself and does this AT LEAST once a day!

here is a sampling of the error we are getting on the 'server_messages_usageservice' log file:

<event logger="com.brio.one.services.mgmt.usage.impl.UsageServiceImpl" timestamp="1210988764908" level="ALWAYS" thread="Foundation Server" sequence_no="390">
<time>16 May 2008 20:46:04,908</time>
<context originator_type="UsageService" originator_name="UT1_srvhouhfm09.###.com" host="SRVHOUHFM09"/>
<message><![CDATA[Starting Usage Service
]]></message>
</event>

<event logger="com.brio.one.services.mgmt.usage.impl.UsageServiceImpl" timestamp="1210988770426" level="WARN" thread="GCThread0" sequence_no="427">
<time>16 May 2008 20:46:10,426</time>
<context originator_type="UsageService" host="SRVHOUHFM09"/>
<message><![CDATA[1000165]]></message>
</event>

<event logger="com.brio.one.services.mgmt.usage.impl.GCThread" timestamp="1210988770426" level="WARN" thread="GCThread0" sequence_no="428">
<time>16 May 2008 20:46:10,426</time>
<context originator_type="UsageService" host="SRVHOUHFM09"/>
<message><![CDATA[1000165]]></message>
<throwable><![CDATA[May 16 20:46:10.426
Database Server error: DBConnectionPool::get: [Hyperion][SQLServer JDBC Driver]An error occurred while attempting to log onto the database.

com.sqribe.transformer.DBConnectionException: Database Server error: DBConnectionPool::get: [Hyperion][SQLServer JDBC Driver]An error occurred while attempting to log onto the database.

at com.sqribe.transformer.DBConnectionPool.get(Unknown Source)
at com.brio.one.services.impl.database.BaseDBManager.getUpdateableConnection(Unknown Source)
at com.brio.one.services.mgmt.usage.impl.database.UsageServiceDBManager.deleteEvents(Unknown Source)
at com.brio.one.services.mgmt.usage.impl.database.UsageServiceDBManager.deleteMessages(Unknown Source)
at com.brio.one.services.mgmt.usage.impl.GCThread.execute(Unknown Source)
at com.brio.one.services.mgmt.usage.impl.GCThread.run(Unknown Source)
[1000165]:
DBConnectionPool::get: [Hyperion][SQLServer JDBC Driver]An error occurred while attempting to log onto the database.


So in one of the log errors, it lists the user that is trying to connect. It is the SS admin I created during the installation, 'hypadmin'. This user of course does not have access to the BIPLUS db, which would cause these errors.

So I tried something, created a SQL user of the same name with the same password. Restarted services, and everything seemed to work fine, except this time when I tried to launch SS, it said a severe error had occurred, and to restart SS.

I restarted, same error. the horror consumes me...

so i went back into SQL, looked at the processes and lo and behold hypadmin was using one measely connection. I killed the process, killed the user ID in SQL, restarted all of my services and everything was golden again except, of course, the monstrosity that is the usageservice log file and the constant hangs while in the user management console.

does any of this ring a bell? how would a native directory user be configured to access the db???
602270 May 21 2008
A native user can not have been configured for access to the DB.

Determining HSS DB user:
Look in the file:
d:\Hyperion\deployments\Tomcat5\SharedServices9\config\Domain.xml

This will be a different folder since you are not using tomcat.
In the domain.xml file you will find the database connectivity information, user name something like:
<parameter name="user">HyperionSQLUser</parameter>

For BIPlus Database connection information this can be found in the file:
d:\Hyperion\BIPlus\common\config\config.dat

You should not edit the user/password information in these files.


Does the database get shutdown every night for backups and does BI get restarted after the database comes back up?

You will get errors like that when the database for BIPlus is not accessible,

Does the stdout_console.log log also have similar errors at the same time?
user486517 Jun 2 2008
Kelly, I'm having a similar problem with the server_messages_UsageService.log with an Oracle database. I keep getting invalid username/password errors every second and the file grows very large very quickly. Did you ever find out what the problem was? Any suggestions would be much appreciated. Thanks.

DaveC
638106 Jun 11 2008
Dave,

We resolved the issue yes.

The issue went to tier 3 support, he was dumbfounded as to what could have caused this as well.

What I did before he called was I checked the security audit log on my sql server to see which user name was attempting to connect. It was our Shared Services admin account, which we created called 'hypadmin'.

So what we had, and I am assuming you have as well, was one of the BI+ core services somehow using the wrong account to get db information.

My idea was to simply reconfigure Core Services using the configuration utility, but i did not want to do this without the 'blessing' of one of the upper tiered support personnel.

After about 4 hours of checking every possible configuration file where every one of them seemed the right settings, we finally decided on re-running the configuration tool. After completing the configuration we decided to go ahead and reboot all of our servers, and it worked!

Hope this helps, Dave. If you would like, you can PM me your details if you need help on your side.

Kelly
user486517 Jun 12 2008
Kelly,

Thanks for the response. Yes, this was baffling - it came out of nowhere. We tried a dozen different things to fix it but could not get past it. I finally restored the database repository schemas for both BI+ and Shared Services back to a point in time before the problem started occurring. I then reconfigured Shared Services and BI+ and that resolved the problem. However, I still have no idea what caused it or if there is any better fix than restoring and reconfiguring. Anyhow, I appreciate you getting back to me. Thanks again.

Dave
652550 Aug 2 2008
Kelly, could you drop me a line, I am interested in seeing I could request your services for a dev build.

I can be reached jconlee@sourceanalytics.com

thx.
1 - 15
Locked Post
New comments cannot be posted to this locked post.

Post Details

Locked on Aug 30 2008
Added on May 7 2008
15 comments
6,813 views