Frequent FR Server Crashes
866532 Sep 16, 2013 6:21 PMHello All,
We have a load balanced environment. This month being a busy time, we are experiencing frequent FR Server crashes. FR server1 is crashing more frequently than FR server2.
Our env:
Windows 2008 R2 64 bit
Hyperion 11.1.2.1.127
Heap size -> min : 1024 & max : 2048
readme.txt for Incident report on server2
Incident Id: 7831
Incident Source: SYSTEM
Create Time: Mon Sep 16 13:26:56 EDT 2013
Problem Key: OFM-99999 [FinancialReporting1][oracle.EPMRAF.com.brio.one.services.utils.css.CSSAccess]
Application Name: FINANCIALREPORTING
Error Message Id: OFM-99999
Description
-----------
The following log message matched one or more log detection conditions:
[2013-09-16T13:26:56.495-04:00] [FinancialReporting1] [INCIDENT_ERROR] [] [oracle.EPMRAF.com.brio.one.services.utils.css.CSSAccess] [host: HYP11WEB02] [nwaddr: 172.19.13.103] [tid: 95] [userId: <anonymous>] [ecid: 0000K4bEB296YNL6yRmZMG1IDlYu000001,1:22536] [APP: FINANCIALREPORTING#11.1.2.0] Error retrieving SAP login ticket[[
EPMCSS-00000: Failed to process request. Internal server error. Refer EPM Shared Services Security log file for related errors.
at com.hyperion.css.validate.Validator.validateToken(Validator.java:657)
at com.hyperion.css.CSSSystem.getSAPTicket(CSSSystem.java:431)
at com.brio.one.services.utils.css.CSSAccess.getSAPTicket(Unknown Source)
at com.brio.one.client.proxy.SessionProxy.getSAPTicket(Unknown Source)
at com.brio.one.client.proxy.SessionProxy.getSAPTicket(Unknown Source)
at com.hyperion.reporting.jsr283.foundation.FoundationSession.init(FoundationSession.java:139)
at com.hyperion.reporting.jsr283.foundation.FoundationSession.<init>(FoundationSession.java:123)
at com.hyperion.reporting.jsr283.foundation.FoundationRepository.joinSession(FoundationRepository.java:209)
at com.hyperion.reporting.jsr283.foundation.FoundationRepository.login(FoundationRepository.java:90)
at com.hyperion.reporting.jsr283.foundation.FoundationRepository.login(FoundationRepository.java:107)
at com.hyperion.reporting.security.impl.HsSecurityHelper.login(Unknown Source)
at com.hyperion.reporting.security.impl.HsSecurityHelper.authenticateUser(Unknown Source)
at com.hyperion.reporting.hsserver.impl.HsServer.openSession(Unknown Source)
at sun.reflect.GeneratedMethodAccessor3519.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:160)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:909)
at java.lang.Thread.run(Thread.java:619)
]]
Stack Trace
-----------
EPMCSS-00000: Failed to process request. Internal server error. Refer EPM Shared Services Security log file for related errors.
at com.hyperion.css.validate.Validator.validateToken(Validator.java:657)
at com.hyperion.css.CSSSystem.getSAPTicket(CSSSystem.java:431)
at com.brio.one.services.utils.css.CSSAccess.getSAPTicket(Unknown Source)
at com.brio.one.client.proxy.SessionProxy.getSAPTicket(Unknown Source)
at com.brio.one.client.proxy.SessionProxy.getSAPTicket(Unknown Source)
at com.hyperion.reporting.jsr283.foundation.FoundationSession.init(FoundationSession.java:139)
at com.hyperion.reporting.jsr283.foundation.FoundationSession.<init>(FoundationSession.java:123)
at com.hyperion.reporting.jsr283.foundation.FoundationRepository.joinSession(FoundationRepository.java:209)
at com.hyperion.reporting.jsr283.foundation.FoundationRepository.login(FoundationRepository.java:90)
at com.hyperion.reporting.jsr283.foundation.FoundationRepository.login(FoundationRepository.java:107)
at com.hyperion.reporting.security.impl.HsSecurityHelper.login(Unknown Source)
at com.hyperion.reporting.security.impl.HsSecurityHelper.authenticateUser(Unknown Source)
at com.hyperion.reporting.hsserver.impl.HsServer.openSession(Unknown Source)
at sun.reflect.GeneratedMethodAccessor3519.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:160)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:909)
at java.lang.Thread.run(Thread.java:619)
Diagnostic Dumps
----------------
Name: dms.ecidctx
Dump Files: dms_ecidctx107_i7831.dmp
Name: jvm.threads
Dump Files: jvm_threads108_i7831.dmp
Name: dms.metrics
Dump Files: dms_metrics109_i7831.dmp
Name: odl.quicktrace
Dump Files: odl_quicktrace110_i7831.dmp
Name: odl.logs
Dump Arguments: ecid=0000K4bEB296YNL6yRmZMG1IDlYu000001
Dump Files: odl_logs111_i7831.dmp
Name: odl.logs
Dump Arguments: timestamp=1379352416495
Dump Files: odl_logs112_i7831.dmp
Name: wls.image
Dump Files: wls_image113_i7831.zip
2