This content has been marked as final. Show 17 replies
Sorry I wasn't 100% correct in my post. Yes, the method returns a int. What I meant was when hibernate goes through that array in BatchingBatcher one of the items in the array contains the -1 value.
Do you think Weblogic can have something to do with it since it wraps OraclePreparedStatement ?
Another thing I failed to mention is that whenever this error occurs the WebLogic connection pool gets screwed up and won't recover. We have to restart the entire App server.. restarting the connection pool does nothing.
Its quite hard to get a JDBC trace since this only happens in production and occurs maybe once a day on one of the nodes. Running in debug mode for that amount of time is not doable I think.
Unfortunately, we experience this in a very complex distributed application when we do web-service-based data loading into a cluster... not quite a simple application that can be wrapped up into a tarball...
We might try to replicate this later on with a nice standalone app, but for now we just have to try to resolve it with other workarounds (eliminating batch updates, etc).
Just a note-
We are using JBoss in our scenario, and we have found in our testing today that disabling the hibernate batching (passing a "-Dhibernate.jdbc.batch_size=0" param into the start command) seems to have solved those problems... those errors no longer manifest.
As well, we've actually seen some performance increase on quite a few of our queries.
Our one large row insert that would really benefit from batching is now taking longer, but at least it's working without errors.
We here have experienced the exact same situation here on both our Production system, and our Test System! Our environment is:
- Oracle 10.2.0.3.0 RAC database (2-node) on Linux AS4
- App Server is Linux AS4 running JBoss with Hibernate 3.2.2 using CMT
- Oracle JDBC driver 10.2.0.3.0 (with patch 5609480).
Whenever we turn on Hibernate Batch DML (set to 15 or 20, or anynumber > 0, after a period time and system load, the system has unrecoverable errors, and when any user tries to insert more than 3 records from anywhere in the system, it errors out. Basically, it goes into a state where we need to re-start JBoss and our applications. This works for a period of time until it breaks again. We have been able to successfully create a JMeter task to reproduce this error by loading the system with 50 concurrent users doing simple queries for a period of 20-30 minutes.
The Hibernate error we get is:
Date/Time of Error Thu Jan 10 13:35:29 EST 2008
Error Message RuntimeException; nested exception is: org.hibernate.StaleStateException: Batch update returned unexpected row count from update ; actual row count: -1; expected: 1
java.rmi.ServerException: RuntimeException; nested exception is:
org.hibernate.StaleStateException: Batch update returned unexpected row count from update ; actual row count: -1; expected: 1
Our testing has included a load of 50 concurrent users performing simple queries that take 15-20 secs. to complete.
The unit-of-work includes 4 items only:
1) get a sequence number
2) insert rec into audit table with query start time.
3) perform select query (sometimes this might be 2-3 select statements)
4) update rec in audit table with query end time.
We started with the Oracle JDBC 10.2.0.1.0 driver - this failed,
then tested 10.2.0.3.0 driver, - this failed,
then tested 10.2.0.3.0 driver with patch 5609480, - THIS FAILED TOO.
Since then, we tested our system with the latest DataDirect JDBC driver, and interestingly enough, we were able to successfully make Hibernate work with the the Batch DML setting ON (batch size set to 15 or 20 worked fine). We had no errors using this DataDirect Driver, with our replicatable system test. It is our belief that Oracle may still have a problem with their latest 10.2.0.3.0 driver, even with the latest patch.
Has anyone gotten this problem resolved yet, other than by turning off Batch DML ?? - Thanks, Vince Desborough DBA
To add another "me too" to this thread, we are also facing the same problem on our Production system. Unfortunately we are having issues recreating it on our development/test instances. Our production environment is:
- Oracle 10.2.0.2
- Suse Linux 64bit running Tomcat 5.5 with Hibernate 3.2.4sp1
- Oracle JDBC driver 10.2.0.1.0
We replaced the Oracle Driver with a Driver from Data Direct in an effort to isolate the problem... however it still happens.
Our problem occurs in multi-threaded batch code. When run in a single thread, the problem does not occur.
Given the variety of platforms documented here, this sounds like it might be a Hibernate problem.