This discussion is archived
14 Replies Latest reply: Oct 21, 2012 12:02 PM by 896779 RSS

Getting error when tried to create new user role from the application

894506 Newbie
Currently Being Moderated
Hi,

We have tried to create new user role from application, but got data base error "ROLESCRT: ERROR - CM90000-Database error." and after verified the weblogic manage server logs, we got the following exception while creating new user role.

####<Aug 31, 2012 4:29:51 PM IST> <Warning> <JDBC> <fss-pgs-app-01> <PGMS> <[ACTIVE] ExecuteThread: '78' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1346410791212> <BEA-001153> <Forcibly releasing inactive connection "weblogic.jdbc.wrapper.PoolConnection_oracle_jdbc_driver_T4CConnection@847f12" back into the connection pool "jdbcdb1", currently reserved by: java.lang.Exception
at weblogic.jdbc.common.internal.ConnectionEnv.setup(ConnectionEnv.java:308)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:314)
at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:292)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:425)
at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:316)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:93)
at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:106)
at weblogic.jdbc.pool.Driver.connect(Driver.java:149)
at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:642)
at weblogic.jdbc.jts.Driver.connect(Driver.java:124)
at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:338)
at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.newConnection(DBConnectionManager.java:181)
at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.getConnection(DBConnectionManager.java:101)
at com.aciworldwide.framework.database.DBConnectionManager.getConnection(DBConnectionManager.java:398)
at com.aciworldwide.framework.database.DBProcess.<init>(DBProcess.java:31)
at com.aciworldwide.application.UserSecurityAdmin.addRole(UserSecurityAdmin.java:51)
at jsp_servlet.__procrole._jspService(__procrole.java:1007)
at com.aciworldwide.framework.JSPPageBase.service(Unknown Source)
at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175)
at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3498)
at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
at weblogic.security.service.SecurityManager.runAs(Unknown Source)
at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
.>

Database: Oracle 11g

we are using weblogic 10.3.2 and the jdbc datasource configured in weblogic itself. We have below setting in our environment.

Initial capacity - 400
Maximum capacity - 400
Capacity Increment - 25
Test Connections On Reserve - true
Test Frequency - 120
Seconds to Trust an Idle Pool Connection - 10
Shrink Frequency - 900
Connection Creation Retry Frequency - 60
Inactive Connection Timeout - 250
Connection Reserve Timeout - 10
Ignore In-Use Connections - true
Remove Infected Connections Enabled - true


Please anyone help us to resolve the issue.

Thanks in advance


Thanks & Regards
Tamil
  • 1. Re: Getting error when tried to create new user role from the application
    Kalyan Pasupuleti-Oracle Expert
    Currently Being Moderated
    Hi,

    This Warning can be ignored.

    <BEA-001153> <Forcibly releasing inactive connection "weblogic.jdbc.wrapper.PoolConnection_oracle_jdbc_driver


    this warning only appears when you have Inactive Connection timeout enabled.

    To avoid such warning make it Inactive Connection timeout as zero.

    But for Performance factor and avoid Connection pool leak this parameter is required.


    Check more specific to following role set error with in log.

    Regards,
    Kal
  • 2. Re: Getting error when tried to create new user role from the application
    Joe Weinstein Expert
    Currently Being Moderated
    Please ignore Kalyan's post. He is incorrect.

    If you have a concern for pool connections being leaked, you need the 'inactive connection timeout'
    set so the pool does retract abandoned connections. However, the exception below is special,
    not because of the message, but because of the stack trace. This is a known bug and already fixed
    in a later version of WLS, where this exception is being thrown needlessly in a place where a connection
    has just been reserved and has not even yet been given to an application.

    So in general, this warning is an important sign that an application is either leaking connections
    or still has them but is not using them for longer than your configured limit.

    HTH,
    Joe
  • 3. Re: Getting error when tried to create new user role from the application
    894506 Newbie
    Currently Being Moderated
    Hi Joe,

    Thanks for your reply. Could you please let me know is it possible to resolve this warning by setting up any parameter in weblogic level.

    Waiting for your response.

    Thanks
    Tamil
  • 4. Re: Getting error when tried to create new user role from the application
    Joe Weinstein Expert
    Currently Being Moderated
    To get rid of that particular error, for that reason, you need to contact Oracle support and
    get a patch for that bug, or upgrade to the latest WLS version.
  • 5. Re: Getting error when tried to create new user role from the application
    894506 Newbie
    Currently Being Moderated
    Hi Joe,

    Once again thanks for your response. Could you please let me know how we can find if there is a connection leak, is there any parameter we can set in weblogic level to find the connection leak. If it is confirmed there is a connection leak, how we can resolve the same. Is there any possible settings we can do from weblogic level for further analysis.

    We assumed that we are getting this warnings because of we have not enabled the "Remove Infected Connections" in weblogic data source. If we enable this parameter, the infected connections supposed to be closed. Since the weblogic throws warning while forcefully releasing the inactive connections back into the connection pool, we think that application is leaking the connection. Please correct me if I am wrong and kindly provide your suggestions.


    Thanks,
    Tamil
  • 6. Re: Getting error when tried to create new user role from the application
    Joe Weinstein Expert
    Currently Being Moderated
    Actually, I must correct myself. This message is probably showing you specifically where the
    leak is. This code reserved the connection:

    at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.newConnection(DBConnectionManager.java:181)
    at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.getConnection(DBConnectionManager.java:101)
    at com.aciworldwide.framework.database.DBConnectionManager.getConnection(DBConnectionManager.java:398)
    at com.aciworldwide.framework.database.DBProcess.<init>(DBProcess.java:31)
    at com.aciworldwide.application.UserSecurityAdmin.addRole(UserSecurityAdmin.java:51)

    And is responsible for closing it when the connection is no longer needed. Instead, it did not close the
    connection, and had either lost the reference or simply kept it without using it for your inactive connection
    timeout setting.

    That code is what you need to debug. Look for non-thread-safety, such as the connection being a class/instance
    variable instead of a method-level variable. It should be a method-level variable, closed without fail by the end of a
    method.
  • 7. Re: Getting error when tried to create new user role from the application
    894506 Newbie
    Currently Being Moderated
    Hi Joe,

    Kindly suggest is there any settings to do in weblogic level to find the error message stating that there is a connection leak and can clearly say's where exactly it's leaking.

    Thanks,
    Tamil
  • 8. Re: Getting error when tried to create new user role from the application
    894506 Newbie
    Currently Being Moderated
    Hi Joe,

    Even couple of days back also they have tried the same thing from application. But this time trace log has been enabled by application and we are getting below exception.

    2012-09-04 17:38:24,940 trace.database.connection.com.aciworldwide.framework.database.DBProcess com.aciworldwide.application.UserSecurityAdmin.addRole(UserSecurityAdmin.java:154) [[ACTIVE] ExecuteThread: '51' for queue: 'weblogic.kernel.Default (self-tuning)'] - SQLException occurred: java.sql.SQLException: Connection has already been closed. SQLState: null ErrorCode: 0



    But from weblogic managed server logs we got below warning 2 minutes before when the above exception occured.


    ####<Sep 4, 2012 5:36:03 PM IST> <Warning> <JDBC> <fss-pgs-app-01> <PGMS> <[ACTIVE] ExecuteThread: '42' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1346760363923> <BEA-001153> <Forcibly releasing inactive connection "weblogic.jdbc.wrapper.PoolConnection_oracle_jdbc_driver_T4CConnection@a40de3" back into the connection pool "jdbcdb1", currently reserved by: java.lang.Exception
         at weblogic.jdbc.common.internal.ConnectionEnv.setup(ConnectionEnv.java:308)
         at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:314)
         at weblogic.common.resourcepool.ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:292)
         at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:425)
         at weblogic.jdbc.common.internal.ConnectionPool.reserve(ConnectionPool.java:316)
         at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:93)
         at weblogic.jdbc.common.internal.ConnectionPoolManager.reserve(ConnectionPoolManager.java:106)
         at weblogic.jdbc.pool.Driver.connect(Driver.java:149)
         at weblogic.jdbc.jts.Driver.getNonTxConnection(Driver.java:642)
         at weblogic.jdbc.jts.Driver.connect(Driver.java:124)
         at weblogic.jdbc.common.internal.RmiDataSource.getConnection(RmiDataSource.java:338)
         at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.newConnection(DBConnectionManager.java:181)
         at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.getConnection(DBConnectionManager.java:101)
         at com.aciworldwide.framework.database.DBConnectionManager.getConnection(DBConnectionManager.java:398)
         at com.aciworldwide.framework.database.DBProcess.<init>(DBProcess.java:31)
         at com.aciworldwide.application.UserSecurityAdmin.addRole(UserSecurityAdmin.java:51)
         at jsp_servlet.__procrole._jspService(__procrole.java:1007)
         at com.aciworldwide.framework.JSPPageBase.service(Unknown Source)
         at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
         at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
         at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292)
         at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:175)
         at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3498)
         at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
         at weblogic.security.service.SecurityManager.runAs(Unknown Source)
         at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2180)
         at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2086)
         at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1406)
         at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
         at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
    .>


    Not sure this time how the connection has been closed, at which point the connection has closed and where (whether it is closed by application or DB).


    Thanks,
    Tamil

    Edited by: user1583259 on Sep 6, 2012 7:07 AM

    Edited by: user1583259 on Sep 8, 2012 2:34 AM

    Edited by: user1583259 on Sep 8, 2012 2:35 AM
  • 9. Re: Getting error when tried to create new user role from the application
    Joe Weinstein Expert
    Currently Being Moderated
    that last exception is a separate issue. The connection was closed from within the driver, for internal failure reasons.
  • 10. Re: Getting error when tried to create new user role from the application
    Joe Weinstein Expert
    Currently Being Moderated
    And I cannot be more clear about the leak. I told you exactly where to look and what to do:

    This message is probably showing you specifically where the
    leak is. This code reserved the connection:

    at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.newConnection(DBConnectionManager.java:181)
    at com.aciworldwide.framework.database.DBConnectionManager$DBConnectionPool.getConnection(DBConnectionManager.java:101)
    at com.aciworldwide.framework.database.DBConnectionManager.getConnection(DBConnectionManager.java:398)
    at com.aciworldwide.framework.database.DBProcess.<init>(DBProcess.java:31)
    at com.aciworldwide.application.UserSecurityAdmin.addRole(UserSecurityAdmin.java:51)

    And is responsible for closing it when the connection is no longer needed. Instead, it did not close the
    connection, and had either lost the reference or simply kept it without using it for your inactive connection
    timeout setting.

    That code is what you need to debug. Look for non-thread-safety, such as the connection being a class/instance
    variable instead of a method-level variable. It should be a method-level variable, closed without fail by the end of a
    method.
  • 11. Re: Getting error when tried to create new user role from the application
    894506 Newbie
    Currently Being Moderated
    Hi Joe,

    I guess we can find where the connection leak is happening by enabling the parameter "Profile Connection Leak" on data source configuration.
    I have seen this parameter from the weblogic console Data sources-> "Name of the Data Source" -> Configuration -> Diagnostics

    This parameter's description was "Collect profile information about threads that have reserved a connection from the data source and the connection leaked (was not properly returned to the pool of connections)".

    Can you please suggest me whether we can enable this parameter to check and also what exactly this parameter will do. If we enable this parameter, what kind of output we can get.

    There will be any impact if we enable this parameter...


    Thanks,
    Tamil

    Edited by: user1583259 on Sep 8, 2012 2:33 AM
  • 12. Re: Getting error when tried to create new user role from the application
    Joe Weinstein Expert
    Currently Being Moderated
    Don't guess. I am telling you (and that stack trace is telling you) where the leak is. That code
    reserved a pool connection and that connection was never closed. You have to examine that
    application code to figure out why.
  • 13. Re: Getting error when tried to create new user role from the application
    795854 Newbie
    Currently Being Moderated
    Just mentioned as a known bug, could you advise the bug number?
  • 14. Re: Getting error when tried to create new user role from the application
    896779 Newbie
    Currently Being Moderated
    Please take a look at the Oracle Support site, ID 1329416.1 (WebLogic Server JDBC Connections Being Forcibly Released with Error BEA-001153: Forcibly releasing inactive connection )

    It has reference to BUG:9020369 and the solution. Good luck.

    Lawrence Manickam
    http://toyork.blogspot.ca

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points