suddenly we have started getting this error.
we have weblogic 126.96.36.199, soa suite 188.8.131.52.0 osb 184.108.40.206.0 installed.
in summary of deployments in Admin console, i can see em is in failed state.
we tried starting em from summary of deployments but its giving below error:
<Apr 13, 2012 10:43:15 AM BST> <Error> <HTTP> <BEA-101008> <[ServletContext@229185004[app:em module:/em path:/em spec-version:2.5]] Cannot get InputStream of resource "/WEB-INF/config/consoleConfig.xml".
java.io.FileNotFoundException: /u01/oracle/MW_HOME/user_projects/domains/fusion_domain/servers/AdminServer/tmp/_WL_user/emcore/28c293/WEB-INF/config/consoleConfig.xml (Too many open files)
at java.io.FileInputStream.open(Native Method)
Truncated. see log file for complete stacktrace
The common reason of running into FD limit exception is the increase in the FD consumption or low value configured for the ulimit.
In case increasing the ulimit value fixes the issue (which will only be the case if we have not properly configured the system) then we need to collect some lsof() output to check what is the state of the socket and in case there is not issue with the socket management.
Thus to start with tune ulimit and if this does not fix the issue collect lsof() output to check where are the FD's/socket consumed.
ulimit value was 1024 and after changing it to 4096 i am able to start em console but there is another issue.
I am changing this ulimit from putty and starting the server, so whenever that putty session gets closed, ulimit value is again getting reset to 1024 and em again goes to failed state.
Please let me know how to permanently change ulimit value.
To make this change permanent login with root user and edit file /etc/security/limits.conf
Add below fields for the user (for example here I have installed MW with weblogic user):
weblogic hard nofile 131072
weblogic soft nofile 131072
weblogic hard nproc 131072
weblogic soft nproc 131072
weblogic soft core unlimited
weblogic hard core unlimited
weblogic soft memlock 3500000
weblogic hard memlock 3500000