I had the chance to test the Rules Engine Drools and even tough I like it I concluded that Drools did not perform as we had expected. The main reason to drop Drools where that the number of elements participating in the rules where not confined to a small number. My tests showed that Drools is fine when either the number of elements in the test is small or when the many of the possible combinations can be removed because of failing conditions. Again I did not conclude that Drools is not a good tool but rather that I tried to find the limitations of the tool.

Here I want to show a very, very simplistic test of drools performance just to show my point. I have three classes which does return the same object: a string called "Test".:

package com.madplanet.drools.test;
public class A {
    public Object getObject() {
        return "TEST";
    }
}

Then I created a Drools Rule Script which just tests each Class' getObject() value against one of the other type:

<?xml version="1.0"?>

<rule-set name="zip.code.rule"
    xmlns="http://drools.org/rules"
    xmlns:java="http://drools.org/semantics/java"
    xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
    xs:schemaLocation="http://drools.org/rules rules.xsd
                       http://drools.org/semantics/java java.xsd">

    <import>com.madplanet.drools.test.A</import>
    <import>com.madplanet.drools.test.B</import>
    <import>com.madplanet.drools.test.C</import>

    <rule name="Test Transitiveness" salience="10">
        <parameter identifier="a">
            <class>com.madplanet.drools.test.A</class>
        </parameter>
        <parameter identifier="b">
            <class>com.madplanet.drools.test.B</class>
        </parameter>
        <parameter identifier="c">
            <class>com.madplanet.drools.test.C</class>
        </parameter>
        <java:condition>
            a.getObject().equals( b.getObject() )
        </java:condition>
        <java:condition>
            b.getObject().equals( c.getObject() )
        </java:condition>
        <java:condition>
            c.getObject().equals( a.getObject() )
        </java:condition>
        <java:consequence>
        </java:consequence>
    </rule>
</rule-set>

Finally I created a jUnit Test Case that creates the same number of A, B and C instances, assign it to the working memory of Drools and finally fires the rule:

    public void testZipCodeExcludeWithinInclude()
            throws Exception {
        for( int i = 50; i < 150; i += 5 ) {
            WorkingMemory lWorkingMemory = mRuleBase.newWorkingMemory();
            long lStartTime = System.currentTimeMillis();
            System.out.println( "--- Start: " + i + ". Loop ---" );
            for( int j = 0; j < i; j++ ) {
                lWorkingMemory.assertObject( new A() );
            }
            printLog( "   ", "Added all As", i, lStartTime );
            for( int j = 0; j < i; j++ ) {
                lWorkingMemory.assertObject( new B() );
            }
            printLog( "   ", "Added all Bs", i, lStartTime );
            for( int j = 0; j < i; j++ ) {
                lWorkingMemory.assertObject( new C() );
            }
            printLog( "   ", "Added all Cs", i, lStartTime );
            lWorkingMemory.fireAllRules();
            printLog( "", "Finished (fired rules)", i, lStartTime );
        }
    }

Below is the output of the test above and as you can see the cross product of all the possible combinations defines the duration of the execution. Please keep in mind that in that test all conditions succeed and so it is the worst case scenario. If all condition would fail the test would go through in a blink of the eye but this is the other extreme. Also this test is so simple that it does not represent any actual business case.

--- Start: 50. Loop ---
   Added all As, loop: 50, duration: 0s
   Added all Bs, loop: 50, duration: 0s
   Added all Cs, loop: 50, duration: 0s
Finished (fired rules), loop: 50, duration: 1s
--- Start: 55. Loop ---
   Added all As, loop: 55, duration: 0s
   Added all Bs, loop: 55, duration: 0s
   Added all Cs, loop: 55, duration: 0s
Finished (fired rules), loop: 55, duration: 1s
--- Start: 60. Loop ---
   Added all As, loop: 60, duration: 0s
   Added all Bs, loop: 60, duration: 0s
   Added all Cs, loop: 60, duration: 1s
Finished (fired rules), loop: 60, duration: 2s
--- Start: 65. Loop ---
   Added all As, loop: 65, duration: 0s
   Added all Bs, loop: 65, duration: 0s
   Added all Cs, loop: 65, duration: 1s
Finished (fired rules), loop: 65, duration: 2s
--- Start: 70. Loop ---
   Added all As, loop: 70, duration: 0s
   Added all Bs, loop: 70, duration: 0s
   Added all Cs, loop: 70, duration: 1s
Finished (fired rules), loop: 70, duration: 3s
--- Start: 75. Loop ---
   Added all As, loop: 75, duration: 0s
   Added all Bs, loop: 75, duration: 0s
   Added all Cs, loop: 75, duration: 2s
Finished (fired rules), loop: 75, duration: 4s
--- Start: 80. Loop ---
   Added all As, loop: 80, duration: 0s
   Added all Bs, loop: 80, duration: 0s
   Added all Cs, loop: 80, duration: 2s
Finished (fired rules), loop: 80, duration: 5s
--- Start: 85. Loop ---
   Added all As, loop: 85, duration: 0s
   Added all Bs, loop: 85, duration: 0s
   Added all Cs, loop: 85, duration: 3s
Finished (fired rules), loop: 85, duration: 7s
--- Start: 90. Loop ---
   Added all As, loop: 90, duration: 0s
   Added all Bs, loop: 90, duration: 0s
   Added all Cs, loop: 90, duration: 4s
Finished (fired rules), loop: 90, duration: 8s
--- Start: 95. Loop ---
   Added all As, loop: 95, duration: 0s
   Added all Bs, loop: 95, duration: 0s
   Added all Cs, loop: 95, duration: 4s
Finished (fired rules), loop: 95, duration: 9s
--- Start: 100. Loop ---
   Added all As, loop: 100, duration: 0s
   Added all Bs, loop: 100, duration: 0s
   Added all Cs, loop: 100, duration: 6s
Finished (fired rules), loop: 100, duration: 11s
--- Start: 105. Loop ---
   Added all As, loop: 105, duration: 0s
   Added all Bs, loop: 105, duration: 0s
   Added all Cs, loop: 105, duration: 7s
Finished (fired rules), loop: 105, duration: 14s
--- Start: 110. Loop ---
   Added all As, loop: 110, duration: 0s
   Added all Bs, loop: 110, duration: 0s
   Added all Cs, loop: 110, duration: 7s
Finished (fired rules), loop: 110, duration: 15s
--- Start: 115. Loop ---
   Added all As, loop: 115, duration: 0s
   Added all Bs, loop: 115, duration: 0s
   Added all Cs, loop: 115, duration: 8s
Finished (fired rules), loop: 115, duration: 17s
--- Start: 120. Loop ---
   Added all As, loop: 120, duration: 0s
   Added all Bs, loop: 120, duration: 0s
   Added all Cs, loop: 120, duration: 11s
Finished (fired rules), loop: 120, duration: 21s
--- Start: 125. Loop ---
   Added all As, loop: 125, duration: 0s
   Added all Bs, loop: 125, duration: 0s
   Added all Cs, loop: 125, duration: 13s
Finished (fired rules), loop: 125, duration: 26s
--- Start: 130. Loop ---
   Added all As, loop: 130, duration: 0s
   Added all Bs, loop: 130, duration: 0s
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 180.885 sec
    [junit] [ERROR] TEST com.madplanet.drools.test.ObjectEqualsTest FAILED

The loop in the output means how many instances of A, B and C are created. The 'Added' line is printed out when all of the respective object were added to the Drools working memory. Finally he test case fails because the test runs out of memory (I used the Ant's default settings) and can be easily fixed by increasing the heap size.

My conclusion is that Drools works fine when the number of participating elements is small but Drools may run into a performance problem when the cross product of the participating objects is huge. In my test it was getting slow when the cross product run over half a million.