we are working on CEP implementation in a Bank. One of the requeriments is of course the correlation of the events. The max time between two correlated events can be 50 days max, then the event dies. The volume of the event per day is about 15:000 and about 20% have to be correlated. The question is are:
-- make sense use cache (coherence) in order to mantain the events for 50 days?
-- when use range in CQL? what max time razonable in order use range clausole (second, hours or days)?
Let me explain our situation. We are working with follow stacks:
• Oracle CEP 11gR1 (18.104.22.168.0)
• S.O.: SUN Solaris
• JVM: Oracle JRockit 6 - R28.2.3 for Solaris
We are receiving the events associates to sell credit process in a Bank. Actually there three system that are sending events and these systems are using also for other scopes sending events that are not related with the sell credit process.
In CEP we are receiving the events via JMS and in order to handle the correlation we are using “select * from s[range 50 days]” statement . The problem that after 45 / 50 HHS the system crash because out of memory.
Using mission control of jrockit, we see that every time that the CEP use the statement “select * from s[range 50 days]” the heap decrease because the class oracle.cep.dataStructures.internal.memory.CharAttrVal is allocate and this instance live for 50 days (the garbage collector cannot free the memory). Monitoring the platform we see that the occupation of heap grow very fast due oracle.cep.dataStructures.internal.memory.CharAttrVal allocation and this instance is related with “select * from s[range 50 days]” statement.
We did also a test using “select * from s[NOW]” and the memory work find.
One solution could be use the Coherence in order to “persist” the event. What do you think? Is correct that the memory heap grow so fast?
Well the out of memory (OOM) was expected since your event type seems big. If we use range window, all the events will be stored in the heap, so we get OOM exception once the heap is full, for channel[now] it just deals with the latest event so no need of storing all the events so it works fine with the memory. I guess if you increase the max size of heap in jvm it may help for 3-4 days more but 50 days is very big range.
Using coherence cache sounds like a good workaround but it will be very slow since we have to store all the events onto disk (else we get same OOM exception for normal cache config which uses main memory).