CONTEXT 

I want to share a problem that we had in our project. We were doing a real-time "Profits and Loss" server (P&L). The server sends stock updates to all the users subscribed to the stocks, basically as Google Finance or Yahoo Finance.

SIMPLE IMPLEMENTATION 

I will used a basic approach (no aggregation and no optimization) to explain the problem that we had with Serialization and Hessian.

The server always keep the last value on the stocks, because when a user ask for a quote we want to send back as soon as possible the last value (from the cache) that we had on that stock).

We kept the stock prices (BID and ASK) in a simple object :SymbolSerializable.

When a update is received for a stock, we update the values in his SymbolSerializable and send back the updated values to all client subscribed to this stock.

...

symbol.setBid(update.getBid());
symbol.setAsk(update.getAsk());

...

for(Client client : clientList){
    client.sendUpdate(symbol);
}

....

It can't be more simpler than that. The client will received each updates on his stocks. The problems that we had was surprising. The clients were receiving the always the same price on a stock !

WHY ? 

here an example :

stock : ABC

Updates :

1 : bid=10.25$, ask=10.50$
2 : bid=11.50$, ask=11.75$
3 : bid=12.00$, ask=12.15$

and the clients received :

1 : bid=10.25$, ask=10.50$
2 : bid=10.25$, ask=10.50$
3 : bid=10.25$, ask=10.50$

To debug our server we printed the values sent to the client, and we saw in the server's logs that the values were corrects.

value sent to client :
bid=10.25$, ask=10.50$

value sent to client :
bid=11.50$, ask=11.75$

value sent to client :
bid=12.00$, ask=12.15$

Everything look fine, why it doesn't work on the client side ?

INVESTIGATION/SOLUTION 

The only thing that could cause the problem were our Serialization in the server. I'll show you four different test cases. See the implementations below.

  • #1 : Reference implementation using Serializable (failed)
  • #2 : Changed Serializable for Externalizable interface (failed)
  • #3 : Serializable using new (passed)
  • #4 : Serializable using reset (passed)
               
#1 : Serializable : Reference#2 : Externalizable#3 : Serializable with new#4 : Serializable with reset
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • All items read
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • bid= 0.0 ask= 0.0
  • All items read
  • bid= 0.0 ask= 0.0
  • bid= 1.0 ask= 1.0
  • bid= 2.0 ask= 2.0
  • bid= 3.0 ask= 3.0
  • bid= 4.0 ask= 4.0
  • bid= 5.0 ask= 5.0
  • bid= 6.0 ask= 6.0
  • bid= 7.0 ask= 7.0
  • bid= 8.0 ask= 8.0
  • bid= 9.0 ask= 9.0
  • All items read
  • bid= 0.0 ask= 0.0
  • bid= 1.0 ask= 1.0
  • bid= 2.0 ask= 2.0
  • bid= 3.0 ask= 3.0
  • bid= 4.0 ask= 4.0
  • bid= 5.0 ask= 5.0
  • bid= 6.0 ask= 6.0
  • bid= 7.0 ask= 7.0
  • bid= 8.0 ask= 8.0
  • bid= 9.0 ask= 9.0
  • All items read
Here the Serializable pojo 
package ca.sebastiendionne.demo.model;

import java.io.Serializable;

public class SymbolSerializable implements Serializable {

    private static final long serialVersionUID = 7853892880704628717L;

    Double bid = null;
    Double ask = null;

    public SymbolSerializable(){
    }

    public Double getBid() {
        return bid;
    }
    public void setBid(Double bid) {
        this.bid = bid;
    }
    public Double getAsk() {
        return ask;
    }
    public void setAsk(Double ask) {
        this.ask = ask;
    }

    @Override
    public String toString() {
        return "bid = " + bid + "  ask=" + ask;
    }

}

Here the Externalizable pojo 
package ca.sebastiendionne.demo.model;

import java.io.Externalizable;
import java.io.IOException;
import java.io.ObjectInput;
import java.io.ObjectOutput;

public class SymbolExternalizable implements Externalizable {
    private static final long serialVersionUID = 1853892880704628717L;

    Double bid = null;
    Double ask = null;
    
    public SymbolExternalizable(){
        
    }
    
    public Double getBid() {
        return bid;
    }
    public void setBid(Double bid) {
        this.bid = bid;
    }
    public Double getAsk() {
        return ask;
    }
    public void setAsk(Double ask) {
        this.ask = ask;
    }
    
    @Override
    public String toString() {
        return "bid = " + bid + "  ask=" + ask;
    }

    @Override
    public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException {
        bid = (Double)in.readObject();
        ask = (Double)in.readObject();
    }

    @Override
    public void writeExternal(ObjectOutput out) throws IOException {
        out.writeObject(bid);
        out.writeObject(ask);
    }
    
}



 

For the test cases I use a for loop to create Symbol and serialized on the hard drive and unserialized them back.

Here the code for the test case #1 (used as reference)
package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolSerializable;

public class SerializationFailed {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolSerializable symbol = new SymbolSerializable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            oos.writeObject(symbol);
            oos.flush();
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolSerializable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

Here the code for the test case #2
package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolExternalizable;

public class SerializationFailed2 {

    
    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolExternalizable symbol = new SymbolExternalizable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            oos.writeObject(symbol);
            oos.flush();
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolExternalizable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

Here the code for the test case #3
package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolSerializable;

public class SerializationWithNew {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolSerializable symbol = new SymbolSerializable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            SymbolSerializable symbol2 = new SymbolSerializable();
            symbol2.setAsk(symbol.getAsk());
            symbol2.setBid(symbol.getBid());
            
            oos.writeObject(symbol2);
            oos.flush();
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolSerializable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

Here the code for the test case #4
package ca.sebastiendionne.demo;

import java.io.EOFException;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import ca.sebastiendionne.demo.model.SymbolSerializable;

public class SerializationWithReset {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        
        ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(new File("test.ser"))); 
        
        
        SymbolSerializable symbol = new SymbolSerializable();
        
        // dummy values
        symbol.setAsk(new Double(-1));
        symbol.setBid(new Double(-1));
        
        for(int i=0;i<10;i++){
            // update values
            symbol.setAsk(new Double(i));
            symbol.setBid(new Double(i));
            
            oos.writeObject(symbol);
            oos.flush();
            oos.reset(); // magic line
        }
        
        oos.flush();
        oos.close();
        
        // read
        
        ObjectInputStream ois = new ObjectInputStream(new FileInputStream(new File("test.ser")));

        try {
            Object obj = null;
            while((obj = ois.readObject())!=null){
                System.out.println((SymbolSerializable)obj);
            }
        } catch (EOFException e) {
            System.out.println("All items read");
        } catch(Exception e){
            e.printStackTrace();
        } finally {
            ois.close();
        }
        
        
    }

}

I can't say what is the performance impact if we use reset() on each updates sent. It is better to use new or reset ?


You can follow me on Twitter : http://twitter.com/survivant