This discussion is archived
5 Replies Latest reply: Dec 5, 2012 1:12 AM by JohnGoodwin RSS

Planning EPMServer0 keeps crashing

user8769031 Newbie
Currently Being Moderated
Hi,
We raised a SR over 3 weeks ago and have been trying to identify why the EPMServer0 crashes every 3 to 4 days and we need to restart it.

The memory is set as -Xms3072m –Xmx8160m
Monitoring the server it generally stays with the min level. We never say an OutOfMemory errors in the logs.

Also when it crashes I noticed that is Middleware folder size increased by 5Gig (too much wasted space). I noticed there are 246 folders with different dates under ./servers/EPMServer0/adr/diag/ofm/EPMSystem/EPMServer0/incident. The readme.txt had:

Description
-----------
Incident detected using watch rule "UncheckedException":
Watch time: Dec 3, 2012 2:16:24 PM EST
Watch ServerName: EPMServer0
Watch RuleType: Log
Watch Rule: (SEVERITY = 'Error') AND ((MSGID = 'WL-101020') OR (MSGID = 'WL-101017') OR (MSGID = 'WL-000802') OR (MSGID = 'BEA-101020') O
R (MSGID = 'BEA-101017') OR (MSGID = 'BEA-000802'))
Watch DomainName: EPMSystem
Watch Data:
DATE : Dec 3, 2012 2:16:24 PM EST
SERVER : EPMServer0
MESSAGE : [ServletContext@880365594[app:EAS module:easconsole path:/easconsole spec-version:2.5 version:11.1.2.0]] Servlet failed with Exception
java.lang.IllegalStateException: Response already committed
at weblogic.servlet.internal.ServletResponseImpl.objectIfCommitted(ServletResponseImpl.java:1629)
at weblogic.servlet.internal.ServletResponseImpl.sendError(ServletResponseImpl.java:637)



Any suggestions on identifying the crash and recommendations to clean the extra folders created.


Thanks
  • 1. Re: Planning EPMServer0 keeps crashing
    JohnGoodwin Oracle ACE
    Currently Being Moderated
    You should be able to delete the incident dump files though I thought they were purged when they reached a certain size, how big is the incident directory?
    It makes it more of a problem trying to find the issue as it looks like you have a compact deployment, if one of the web apps crash then it takes everything out, are all the incidents relating to EAS, is there a reason why you picked a compact deployment?

    Cheers

    John
    http://john-goodwin.blogspot.com/
  • 2. Re: Planning EPMServer0 keeps crashing
    user8769031 Newbie
    Currently Being Moderated
    John,
    The incident total size is 562MB. Couple were created today, but the EPMServer0 is still running.
    Will check the logs under EPMServer0 and the AdminServer domain level and delete older ones.

    There are also many *.gz files under user_projects/domains/EPMSystem/servers/EPMServer0/logs/metrics but they total only 80MB (will delete older ones)

    We installed in compact mode for development purpose. But for the next environment we will separate the components. (Seems this separation is highly recommended. Thks)

    It is been hard to tell whether planning or Essbase is causing the issue, but believe it is Planning. When data is being entered in Planning forms is when it is the slowest.
    FYI: we did enlarge the JDBC connection pools and database processes (1000) and open_cursors (1000) which fixed many issues and don't have DB connections issues.

    We also modified planning JDBC connections:
    JDBC_MIN_CONNECTIONS = 10 (from 1)
    JDBC_MAX_CONNECTIONS = 45 (from 10 )

    Thanks
  • 3. Re: Planning EPMServer0 keeps crashing
    JohnGoodwin Oracle ACE
    Currently Being Moderated
    If you are experiencing issues then it may be worth moving from a compact deployment and deploy the web applications to their own managed server.
    There is no point in deleting files which will be rotated, large dump files that are causing space problems can be deleted.

    Cheers

    John
    http://john-goodwin.blogspot.com/
  • 4. Re: Planning EPMServer0 keeps crashing
    user8769031 Newbie
    Currently Being Moderated
    FYI: the version we have is 11.1.2.2 with weblogic server 10.3.6.0 and OHS.

    I agree that deploying to separate managed servers is recommended, and mainly focus on Planning application initially.
    see two options:
    1. Since Planning is already installed and configured, the idea is to rerun the configuration wizard and reconfigure Planning on separate managed server and use the new default port. Will the config wizard remove and reinstall the Planning app correctly?
    Note we currently have Planning applications already developed which will backup first.

    Are there additional configurations to allow the links in Workspace to point to the new port?
    I might need to modify mod_wl_ohs.conf to repoint <LocationMatch ^/HyperionPlanning>

    2. We also have a separate machine running windows where EPMA and FDM are running using IIS. Currently no weblogic server is installed. I see we can install planning and configure it here, but then need to un-configure it from the Unix server and repoint OHS. Not sure how the unconfigure can be done and whether is option will help.

    Thanks
  • 5. Re: Planning EPMServer0 keeps crashing
    JohnGoodwin Oracle ACE
    Currently Being Moderated
    You can certainly deploy planning to its own managed server though the issue is that the compact deployment will still contain the planning web application which have to be removed using the WebLogic admin console, it is documented here - http://docs.oracle.com/cd/E17236_01/epm.1112/epm_install_1112200/frameset.htm?ch09s05.html

    The OHS configuration should be updated if you run the configure web server again once planning has been deployed.

    Cheers

    John
    http://john-goodwin.blogspot.com/

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points