This discussion is archived
7 Replies Latest reply: Nov 21, 2012 8:35 AM by 974378 RSS

ESSSVR.exe crashes after service start

974378 Newbie
Currently Being Moderated
Greeting everyone,

We have a dedicated Essbase server. Today we started getting a TCP/IP protocol timeout error when trying to connect to EAS.
The first thing we noticed was the "OPMN service for Essbase" was actually running --this is the only Essbase service on this server.
At first we tried to restart the service, but the problem persisted. Then we tried to restart the server, but this didn't help neither.
Finally, we noticed that when you launch the service start (by using the oracle provided .bat files), while the service is starting, an ESSSVR.exe process shows up in the task manager, but once the service is started, the ESSSVER.exe process crashes and disappears from the task manager.

What can be wrong?

Thank you.

Regards,

G.Sanchez
  • 1. Re: ESSSVR.exe crashes after service start
    JohnGoodwin Oracle ACE
    Currently Being Moderated
    Have you checked any of the logs e.g essbase.log and SharedServices_Security_Client.log

    Cheers

    John
    http://john-goodwin.blogspot.com/
  • 2. Re: ESSSVR.exe crashes after service start
    974378 Newbie
    Currently Being Moderated
    Good morning John, and thanks for caring.

    I had already checked the essbase.log file. It shows nothing unusual.
    Following your advice I also checked SharedServices_Security_Client.log, but still nothing.

    Some research made me think that maybe I should set up the ESSBASEPATH and ARBORPATH variables in the System Variables of my Essbase server, which I did.
    After that I restarted services (the "OPMN service for Essbase" is the only Essbase service running on this server), but nothing happens.
    Then I tried restarting the server, but the problem persists.

    Is there anything else I could check to provide some clues about what's happening?

    Thank you.

    Regards,

    GSFeliu
  • 3. Re: ESSSVR.exe crashes after service start
    JohnGoodwin Oracle ACE
    Currently Being Moderated
    You shouldnt need to set the environment variables as they should be contained in the OPMN configuration file.
    You could also check the OPMN log files in <MIDDLEWARE_HOME>\user_projects\<instancename>\diagnostics\logs\OPMN\opmn

    Cheers

    John
    http://john-goodwin.blogspot.com/
  • 4. Re: ESSSVR.exe crashes after service start
    974378 Newbie
    Currently Being Moderated
    Thank you, John.

    Looking at the OPMN logs, I can see the following the problem started after a scheduled service restart. Prior to it and up to 11-20-2012 at 4:00 AM, everything was fine. When the service was automatically restarted at 4:40 AM after perfoming an automated back up (which is performed daily), the logs started to show errors and warnings. I pasted them here --maybe you can give me some ideas:

    opmn.log (nothing odd appears to be here)
    [2012-11-20T04:01:56][opmn][TRACE:1][668][OPMN]
    Request 3 Completed. Command: /shutdown
    [2012-11-20T04:01:56][opmn][TRACE:1][][OPMN][code:pm-requests]Http 3 
    EPM_epmsystem3~Essbase1~EssbaseAgent~AGENT~161747104:6416
      Status: Stopped
      Operation: request (time out while waiting for a managed process to stop)
      ErrFile: C:\Oracle\Middleware\user_projects\epmsystem3\diagnostics\logs\OPMN\opmn\console~Essbase1~EssbaseAgent~AGENT~1.log
      String: second stop attempted
    EPM_epmsystem3~Essbase1~EssbaseAgent~AGENT~161747104:6416
      Status: Stopped
      Operation: request (success)
      ErrFile: C:\Oracle\Middleware\user_projects\epmsystem3\diagnostics\logs\OPMN\opmn\console~Essbase1~EssbaseAgent~AGENT~1.log
    
    [2012-11-20T04:01:56][opmn][TRACE:1][666][OPMN][code:pm-process]Process Stopped: Essbase1~EssbaseAgent~AGENT~1 (161747104:6416)
    [2012-11-20T04:40:15][opmn][NOTIFICATION:1][90][OPMN][code:ons-internal]ONS server initiated
    [2012-11-20T04:40:15][opmn][NOTIFICATION:1][522][OPMN][code:pm-internal]PM state directory exists: C:\Oracle\Middleware\user_projects\epmsystem3\config\OPMN\opmn\states
    [2012-11-20T04:40:15][opmn][NOTIFICATION:1][675][OPMN][code:pm-internal]OPMN server ready.  Request handling enabled.
    [2012-11-20T04:40:15][opmn][TRACE:1][667][OPMN][code:pm-requests]Request 2 Started.  Command: /start
    [2012-11-20T04:40:15][opmn][TRACE:1][662][OPMN][code:pm-process]Starting Process: Essbase1~EssbaseAgent~AGENT~1 (161747105:0)
    [2012-11-20T04:40:35][opmn][TRACE:1][662][OPMN][code:pm-process]Starting Process: Essbase1~EssbaseAgent~AGENT~1 (161747105:0)
    [2012-11-20T04:40:46][opmn][TRACE:1][662][OPMN][code:pm-process]Starting Process: Essbase1~EssbaseAgent~AGENT~1 (161747105:0)
    [2012-11-20T04:40:54][opmn][TRACE:1][668][OPMN][code:pm-requests]Request 2 Completed. Command: /start
    
    *service.log*
    --------
    12/11/20 04:00:48 shutdown
    --------
    
    --------
    12/11/20 04:40:16 startproc
    --------
    ================================================================================
    opmn id=DESHYPDB02.codere.com:6712
        0 of 1 processes started.
    *
    ias-instance id=EPM_epmsystem3
    ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    --------------------------------------------------------------------------------
    ias-component/process-type/process-set:
        Essbase1/EssbaseAgent/AGENT/
    
    Error
    --> Process (index=1,uid=161747105,pid=6612)
        failed to start a managed process after the maximum retry limit
        Log:
        C:\Oracle\Middleware\user_projects\epmsystem3\diagnostics\logs\OPMN\opmn\console~Essbase1~EssbaseAgent~AGENT~1.log
    
    
    *console~Essbase1~EssbaseAgentÃGENT~1.log* (I copy what seems to be the most relevant information)
    Read Error:Fatal Error. C:\Oracle\Middleware\user_projects\epmsystem3\EssbaseServer\essbaseserver1\bin\*TEMPESSBASE.SEC is corrupt*
    
    Fatal Error: Essbase did not shutdown cleanly. Please remove TEMPESSBASE.SEC and restart Essbase with the ESSBASE.SEC file. Some security changes could have been lost
    
    According to this last error, shall I just remove (or back up) the TEMPESSBASE.SEC file, leave ESSBASE.SEC and try a restart?
    
    Many thanks,
    
    G.Sanchez                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
  • 5. Re: ESSSVR.exe crashes after service start
    JohnGoodwin Oracle ACE
    Currently Being Moderated
    Yes try removing tempessbase.sec or if that doesnt work try renaming essbase.bak to essbase.sec or revert from a backup of essbase.sec

    Cheers

    John
    http://john-goodwin.blogspot.com/
  • 6. Re: ESSSVR.exe crashes after service start
    974378 Newbie
    Currently Being Moderated
    Hi again, John.

    Thanks for your help. It really worked and Essbase.exe is now running as well as the Essbase applications.
    However, we are facing a second problem and maybe you can guide us here, too.

    Now, the development EAS console freezes up when you try to open the Essbase server node. This does not happen when you use the production console, from where we are able to see both the development and the production Essbase servers.

    It comes to my mind that this could be related to user credentials. For security reasons within the company the admin password was changed yesterday, but it was eventually reverted to the previous password.
    We have several admin accounts, but all of them experience the same issue.

    Thank you.

    Regards,

    G.Sanchez
  • 7. Re: ESSSVR.exe crashes after service start
    974378 Newbie
    Currently Being Moderated
    Ok, forget about it.

    I restarted services in all my Hyperion servers and now everything is fine.

    Thanks a lot.

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points