This discussion is archived
1 2 Previous Next 20 Replies Latest reply: May 19, 2010 4:05 AM by EJP RSS

speed Issue with objectoutputstream and objectinputstream

843790 Newbie
Currently Being Moderated
Hi all,
I have serialized my resultset into a file on the harddisk and converted it back into resultset using below methods : -

public void writeRS(ResultSet pRs) {

try {
crs = new CachedRowSetImpl();
} catch (SQLException ex) {
Logger.getLogger(SerializeRecords.class.getName()).log(Level.SEVERE, null, ex);
}
try {
crs.populate(pRs);
} catch (SQLException ex) {
Logger.getLogger(SerializeRecords.class.getName()).log(Level.SEVERE, null, ex);
}

ObjectOutputStream outputStream = null;
try {
outputStream = new ObjectOutputStream(new FileOutputStream("c:/ABC.xml"));
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
try {
outputStream.writeObject(crs);
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
try {
outputStream.flush();
} catch (IOException ex) {
Logger.getLogger(SerializeRecords.class.getName()).log(Level.SEVERE, null, ex);
}
try {
outputStream.close();
} catch (IOException ex) {
Logger.getLogger(SerializeRecords.class.getName()).log(Level.SEVERE, null, ex);
}
}


public void ReadRS() {
ObjectInputStream ois = null;
try {
DataInputStream dis = null;
try {
dis = new DataInputStream(new BufferedInputStream(new FileInputStream(new File("c:/ABC.xml"))));
} catch (FileNotFoundException ex) {
Logger.getLogger(SerializeRecords.class.getName()).log(Level.SEVERE, null, ex);
}
ois = new ObjectInputStream(dis);
try {
ResultSet t_Rs = (ResultSet) ois.readObject();
try {
while (t_Rs.next()) {
System.out.println(t_Rs.getString("name"));
}
} catch (SQLException ex) {
Logger.getLogger(SerializeRecords.class.getName()).log(Level.SEVERE, null, ex);
}
} catch (ClassNotFoundException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
ois.close();
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
} finally {
try {
ois.close();
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
}

}

It Runs Fine with small resultset but when it became bigger likewise, suppose it is having 50,000 records then reading and writing taking too much time. Can i reduce it??? if yes, then please tell me how??
  • 1. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    1.) Please use code tags when posting code (simply select your code and push "CODE" just above the text area).
    2.) Why do you call your file ABC.xml, when you're not actually writing XML files?
    3.) What is "slow"? How much data do you write (concrete numbers, please), how long does it take (again: concrete numbers please) and how big is the resulting file?
  • 2. Re: speed Issue with objectoutputstream and objectinputstream
    EJP Guru
    Currently Being Moderated
    I have serialized my resultset into a file on the harddisk and converted it back into resultset
    Why? What exactly is the point? A RowSet represents the result of a query at a particular moment in time. One database update later it is obsolete. So what is to be gained by writing it to a file and reading it back? In all probability it would be more efficient to execute the query again instead.
    reading and writing taking too much time
    Compared to what?
  • 3. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    1) No. of rows in resultset = 50,000
    2) It took 40 seconds.
    3) Resulting file size = 15,500 KB
  • 4. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    user6654 wrote:
    2) It took 40 seconds.
    What is "it"? Writing and reading?

    Also, try iterating over the resultset, getting all values and not doing anything with them to check how long that takes. Maybe the serialization is not actually the slow part.
  • 5. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    Iteration with same data took 7 seconds
    Can we set the buffer size of outputstream and inputstream ??
  • 6. Re: speed Issue with objectoutputstream and objectinputstream
    EJP Guru
    Currently Being Moderated
    Iteration with same data took 7 seconds
    Is that iterating over the original ResultSet or the one you fetched from the file?

    And, if the former, again, what's the point? If fetching 50,000 records from the DB only takes 7 seconds why aren't you doing that? Why are you even contemplating writing them to a file and reading them back?

    And if the latter, it is your processing that is at fault, not serialization, or buffering.
    Can we set the buffer size of outputstream and inputstream ??
    Sure, put a BufferedOutputStream between the FileOutputStream and the ObjectOutputStream, and similarly with BufferedInputStream. You can control the buffer sizes of both.

    But I haven't yet heard what the actual point is.
  • 7. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    Sir,
    After setting the buffer size of objectoutputstream, it is saving the object in 8 seconds in place of 40 seconds.
    but again inspite of setting the buffer size of objectinputstream reading the object dosn,t become faster it is taking more than a minute which is not acceptable can you help me in that please.
  • 8. Re: speed Issue with objectoutputstream and objectinputstream
    800389 Newbie
    Currently Being Moderated
    user6654 wrote:
    After setting the buffer size of objectoutputstream, it is saving the object in 8 seconds in place of 40 seconds.
    but again inspite of setting the buffer size of objectinputstream reading the object dosn,t become faster it is taking more than a minute which is not acceptable can you help me in that please.
    You still didn't answer why you are serializing and deserilizing the result set. May be you are taking some wrong approach?
  • 9. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    Ok, forget about resultset, take it like i have written an object in to a file using objectoutputstream and the size of that file is 15,041 kb now i am reading that object from that file using objectinputstream than also it is taking more than a minute. why??

    Particular line which is taking time is Object obj = ois.readObject(); where ois is the object of ObjectInputStream.

    Now is there any way to read this quickly let say if we increase the buffer size of ois like as follows:-

    BufferedInputStream bis = new BufferedInputStream(new FileInputStream("c:/ABC.txt"), 8192);
    ObjectInputStream ois = new ObjectInputStream(bis);
    Object obj = ois.readObject();

    but ispite of this, time didn't get reduced.
    any idea now will be appriciable.
    Thanks
  • 10. Re: speed Issue with objectoutputstream and objectinputstream
    jtahlborn Expert
    Currently Being Moderated
    well, if you want to determine if the slow part is the serialization or the file access, i suggest performing your test entirely in memory (serialize to ByteArrayOutputStream and from ByteArrayInputStream). this will give a better idea of the pure serialization speed.
  • 11. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    No, Output is same it is taking around 70 seconds to deserialize the object. I used the following code

    // serialize
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    ObjectOutputStream oos = new ObjectOutputStream(out);
    oos.writeObject(myobj);
    oos.close();

    //deserialize
    byte[] pickled = out.toByteArray();
    InputStream in = new ByteArrayInputStream(pickled);
    ObjectInputStream ois = new ObjectInputStream(in);
    Object o = ois.readObject();

    now, i feel its serialization which is slow. now how can i resolve it.
  • 12. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    user6654 wrote:
    now, i feel its serialization which is slow. now how can i resolve it.
    You will need to grab a profiler and see why it is slow. I suggest JVisualVM. It's not the most powerful profiler out there, but it is rather easy to use and capable enough for many uses.
  • 13. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    when i set the buffer size like ByteArrayOutputStream out = new ByteArrayOutputStream(8192); in the previous code it stated performing ok.
    Now, it is taking only 8 to 10 seconds in both the operations but i want to serialize my object into a file and read it back from that later on now please suggest how to achieve it optimizingly. because now i got the point that file access is slow. so, i have to resolve it. please sugest me.
  • 14. Re: speed Issue with objectoutputstream and objectinputstream
    843790 Newbie
    Currently Being Moderated
    user6654 wrote:
    when i set the buffer size like ByteArrayOutputStream out = new ByteArrayOutputStream(8192); in the previous code it stated performing ok.
    Are you saying that the same code with "new ByteArrayOutputStream()" takes 40 seconds and with "new ByteArrayOutputStream(8192)" it takes 8-10 seconds? If find that highly unlikely, because using the default buffer size of the BAOS initially should only result in at most 8 more resize-operations, which are very cheap array copies of small-ish byte-arrays. Those should definitely not take 30 seconds longer.

    I'm suspecting that you're measuring is faulty somehow.
1 2 Previous Next