5 Replies Latest reply: Jul 11, 2012 3:11 AM by gimbal2 RSS

    JPA error with nested exception for more details

    859967
      Hi All,

      A strange thing is that when i can deploy javaee app successfully. But some time, it is failed and i have to wait for 10, 15 minutes then i can deploy again. When i got error, below is the error message:

      I don't know if you can find a hint.
      INFO: EclipseLink, version: Eclipse Persistence Services - 2.3.2.v20111125-r10461
      SEVERE: Local Exception Stack: 
      Exception [EclipseLink-4019] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
      Exception Description: Error while obtaining information about the database. Refer to the nested exception for more details.
                      at org.eclipse.persistence.exceptions.DatabaseException.errorRetrieveDbMetadataThroughJDBCConnection(DatabaseException.java:361)
                      at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:602)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryProvider.login(EntityManagerFactoryProvider.java:206)
                      at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:488)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryDelegate.getDatabaseSession(EntityManagerFactoryDelegate.java:188)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryDelegate.createEntityManagerImpl(EntityManagerFactoryDelegate.java:277)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:294)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:272)
                      at org.glassfish.persistence.jpa.PersistenceUnitLoader.loadPU(PersistenceUnitLoader.java:211)
                      at org.glassfish.persistence.jpa.PersistenceUnitLoader.<init>(PersistenceUnitLoader.java:120)
                      at org.glassfish.persistence.jpa.JPADeployer$1.visitPUD(JPADeployer.java:224)
                      at org.glassfish.persistence.jpa.JPADeployer$PersistenceUnitDescriptorIterator.iteratePUDs(JPADeployer.java:495)
                      at org.glassfish.persistence.jpa.JPADeployer.createEMFs(JPADeployer.java:233)
                      at org.glassfish.persistence.jpa.JPADeployer.prepare(JPADeployer.java:168)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.prepareModule(ApplicationLifecycle.java:871)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:410)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:240)
                      at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:389)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:348)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:363)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1085)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:95)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1291)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1259)
                      at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:461)
                      at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:212)
                      at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:179)
                      at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
                      at com.sun.enterprise.v3.services.impl.ContainerMapper$Hk2DispatcherCallable.call(ContainerMapper.java:354)
                      at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)
                      at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:849)
                      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:746)
                      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045)
                      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228)
                      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
                      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
                      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
                      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
                      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
                      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.sql.SQLRecoverableException: Closed Connection
                      at oracle.jdbc.driver.PhysicalConnection.getMetaData(PhysicalConnection.java:4508)
                      at com.sun.gjc.spi.base.ConnectionHolder.getMetaData(ConnectionHolder.java:345)
                      at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:589)
                      ... 42 more
      
      SEVERE: Exception while invoking class org.glassfish.persistence.jpa.JPADeployer prepare method
      SEVERE: Exception while preparing the app
      SEVERE: Exception [EclipseLink-4019] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
      Exception Description: Error while obtaining information about the database. Refer to the nested exception for more details.
      javax.persistence.PersistenceException: Exception [EclipseLink-4019] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
      Exception Description: Error while obtaining information about the database. Refer to the nested exception for more details.
                      at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:517)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryDelegate.getDatabaseSession(EntityManagerFactoryDelegate.java:188)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryDelegate.createEntityManagerImpl(EntityManagerFactoryDelegate.java:277)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:294)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:272)
                      at org.glassfish.persistence.jpa.PersistenceUnitLoader.loadPU(PersistenceUnitLoader.java:211)
                      at org.glassfish.persistence.jpa.PersistenceUnitLoader.<init>(PersistenceUnitLoader.java:120)
                      at org.glassfish.persistence.jpa.JPADeployer$1.visitPUD(JPADeployer.java:224)
                      at org.glassfish.persistence.jpa.JPADeployer$PersistenceUnitDescriptorIterator.iteratePUDs(JPADeployer.java:495)
                      at org.glassfish.persistence.jpa.JPADeployer.createEMFs(JPADeployer.java:233)
                      at org.glassfish.persistence.jpa.JPADeployer.prepare(JPADeployer.java:168)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.prepareModule(ApplicationLifecycle.java:871)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:410)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:240)
                      at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:389)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:348)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:363)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1085)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:95)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1291)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1259)
                      at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:461)
                      at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:212)
                      at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:179)
                      at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
                      at com.sun.enterprise.v3.services.impl.ContainerMapper$Hk2DispatcherCallable.call(ContainerMapper.java:354)
                      at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)
                      at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:849)
                      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:746)
                      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045)
                      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228)
                      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
                      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
                      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
                      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
                      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
                      at java.lang.Thread.run(Thread.java:619)
      Caused by: Exception [EclipseLink-4019] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
      Exception Description: Error while obtaining information about the database. Refer to the nested exception for more details.
                      at org.eclipse.persistence.exceptions.DatabaseException.errorRetrieveDbMetadataThroughJDBCConnection(DatabaseException.java:361)
                      at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:602)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryProvider.login(EntityManagerFactoryProvider.java:206)
                      at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:488)
                      ... 40 more
      Caused by: java.sql.SQLRecoverableException: Closed Connection
                      at oracle.jdbc.driver.PhysicalConnection.getMetaData(PhysicalConnection.java:4508)
                      at com.sun.gjc.spi.base.ConnectionHolder.getMetaData(ConnectionHolder.java:345)
                      at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:589)
                      ... 42 more
      
      SEVERE: Exception while preparing the app : Exception [EclipseLink-4019] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
      Exception Description: Error while obtaining information about the database. Refer to the nested exception for more details.
      Local Exception Stack: 
      Exception [EclipseLink-4019] (Eclipse Persistence Services - 2.3.2.v20111125-r10461): org.eclipse.persistence.exceptions.DatabaseException
      Exception Description: Error while obtaining information about the database. Refer to the nested exception for more details.
                      at org.eclipse.persistence.exceptions.DatabaseException.errorRetrieveDbMetadataThroughJDBCConnection(DatabaseException.java:361)
                      at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:602)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryProvider.login(EntityManagerFactoryProvider.java:206)
                      at org.eclipse.persistence.internal.jpa.EntityManagerSetupImpl.deploy(EntityManagerSetupImpl.java:488)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryDelegate.getDatabaseSession(EntityManagerFactoryDelegate.java:188)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryDelegate.createEntityManagerImpl(EntityManagerFactoryDelegate.java:277)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManagerImpl(EntityManagerFactoryImpl.java:294)
                      at org.eclipse.persistence.internal.jpa.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:272)
                      at org.glassfish.persistence.jpa.PersistenceUnitLoader.loadPU(PersistenceUnitLoader.java:211)
                      at org.glassfish.persistence.jpa.PersistenceUnitLoader.<init>(PersistenceUnitLoader.java:120)
                      at org.glassfish.persistence.jpa.JPADeployer$1.visitPUD(JPADeployer.java:224)
                      at org.glassfish.persistence.jpa.JPADeployer$PersistenceUnitDescriptorIterator.iteratePUDs(JPADeployer.java:495)
                      at org.glassfish.persistence.jpa.JPADeployer.createEMFs(JPADeployer.java:233)
                      at org.glassfish.persistence.jpa.JPADeployer.prepare(JPADeployer.java:168)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.prepareModule(ApplicationLifecycle.java:871)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:410)
                      at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:240)
                      at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:389)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:348)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:363)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1085)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1200(CommandRunnerImpl.java:95)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1291)
                      at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1259)
                      at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:461)
                      at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:212)
                      at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:179)
                      at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:117)
                      at com.sun.enterprise.v3.services.impl.ContainerMapper$Hk2DispatcherCallable.call(ContainerMapper.java:354)
                      at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:195)
                      at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:849)
                      at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:746)
                      at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1045)
                      at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:228)
                      at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
                      at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
                      at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
                      at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
                      at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
                      at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
                      at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
                      at java.lang.Thread.run(Thread.java:619)
      Caused by: java.sql.SQLRecoverableException: Closed Connection
                      at oracle.jdbc.driver.PhysicalConnection.getMetaData(PhysicalConnection.java:4508)
                      at com.sun.gjc.spi.base.ConnectionHolder.getMetaData(ConnectionHolder.java:345)
                      at org.eclipse.persistence.internal.sessions.DatabaseSessionImpl.loginAndDetectDatasource(DatabaseSessionImpl.java:589)
                      ... 42 more
      thanks,

      -Hoang Long
        • 1. Re: JPA error with nested exception for more details
          gimbal2
          i have to wait for 10, 15 minutes then i can deploy again
          What do you mean by that? Deploy to a running instance or are you rebooting the container?

          The 10-15 minute thing looks an awful lot like some sort of connection pool timeout thing, but I can only guess.
          • 2. Re: JPA error with nested exception for more details
            r035198x
            Yep, the appserver got hold of a usable connection but the connection was closed before the appserver had finished/committed with it. It could be that the connection was closed by the database or some firewall timeout setting.
            A good start is to investigate options of configuring your appserver's connection eviction policies including forcing a validation SQL to be executed when a connection is acquired. Make sure that such a query doesn't impact on performance and have some performance and integration testing before rolling out.
            • 3. Re: JPA error with nested exception for more details
              859967
              r035198x wrote:
              Yep, the appserver got hold of a usable connection but the connection was closed before the appserver had finished/committed with it. It could be that the connection was closed by the database or some firewall timeout setting.
              A good start is to investigate options of configuring your appserver's connection eviction policies including forcing a validation SQL to be executed when a connection is acquired. Make sure that such a query doesn't impact on performance and have some performance and integration testing before rolling out.
              i found that if i close the computer which causes this issue, hence other dev computers can deploy normally.
              "deploy" here i mean deploy to a running instance on each dev's machine.

              From this, i would guess the common database server itself rejects connection from clients.

              I will try to investigate more if i am missing something.

              And here i found a link that possibly a first good attempt:

              [http://www.eclipse.org/forums/index.php/m/505809/]

              EclipseLink automatically recovers from communication failures, but there is little we can do to diagnose the failure. We basically execute a query and have to wait for the JDBC driver, or DataSource's response, if it hangs for 10minutes, then we are waiting. You could set the JDBC timeout hint on your query, but whether the JDBC driver honors this is up to the driver. You should see the SQLException in your logs, provided your log level is warning or lower.
              If you cannot fix your firewall, they you could change your connection pool settings to use a min number of connections of 0, so there will never be idle connections. But then you will be reconnecting a lot.
              You could also use a third party connection pool or DataSource that supports a timeout setting on the pooled connections, or spawn a thread in your application the clears the connection pool every 30minutes.
              regards,

              -HL
              • 4. Re: JPA error with nested exception for more details
                859967
                here i found a way to produce this issue:

                I am having my WAR project with database connection. When i run the WAR, after having things displayed on my browser, i disabled network connection then i enabled it again. Then i continued browsing my application. At this point, the application didn't respond any more and i had close connection exception.

                I think that the disconnection of network and re-enable it back should not affect to database connection like other web applications (in other programming languages).

                Have you known a better solution? Try to reconnect database on all database access? I think ejb3 have done this right after connection failure?

                regards,

                Edited by: CardM on Jul 10, 2012 6:21 PM
                • 5. Re: JPA error with nested exception for more details
                  gimbal2
                  Normally you have some sort of connection pool which takes care of stale or broken connections. But that's a Glassfish specific thing, I can't comment on that. Under Tomcat and JBoss things work just fine, that much I can say.

                  Btw: a programming language does not open connections.