This discussion is archived
11 Replies Latest reply: Jul 12, 2010 3:52 PM by EJP RSS

XMLEncoder cutting off data??

843790 Newbie
Currently Being Moderated
This may not end up being the correct forum, but we have an applet that serializes data to a server via HTTP Put. We use XMLEncoder to write the java bean. However, when the server receives the encoded bean, it recieves only part of the bean. I suspect the full size of the XMLEncoded data is about 25,000 characters. I'm getting around 12000 of it.

Here is the code that writes the data:
     private int writeData(URL url, URLConnection connection, HashMap<String, Object> dataToSend, boolean xmlEncodeValues) throws IOException {
          if (dataToSend == null || dataToSend.size() == 0) return HttpURLConnection.HTTP_NO_CONTENT;
          
          try {
               if (connection == null) {
                    connection = url.openConnection();
                    connection.setUseCaches(false);
               }

               connection.setDoOutput(true);

               OutputStream out = connection.getOutputStream();
               PrintStream ps = new PrintStream(out);
               XMLEncoder encoder = null;
               
               if (xmlEncodeValues) encoder = new XMLEncoder(ps);

               int keyNum = 0;
               for (String key : dataToSend.keySet()) {
                    Object value = dataToSend.get(key);
                    
                    if (keyNum > 0) ps.print("&");
                    ps.print(key + "=");
                    if (xmlEncodeValues)
                         encoder.writeObject(value);
                    else
                         ps.print(value);
                    
                    keyNum++;
               }

               if (xmlEncodeValues) encoder.flush();
               ps.flush();
     
               return HttpURLConnection.HTTP_OK;
          }
          catch (IOException e) {
               throw e;
          }
     }
     
I'm completely open to a better way...
  • 1. Re: XMLEncoder cutting off data??
    67361 Newbie
    Currently Being Moderated
    I'm a bit confused by what this code ends up doing. It looks like you're trying to build a URL from the data (with the & and = ). Most servers are not going to handle a really long URL. But I would check in the server logs to see if there are any errors reported.

    But overall, what is it you're trying to do? XMLEncoder is used as a Java Object serializer. Are you trying to serialize Java Objects to the server? I guess that what you're doing would work but you'd have to split it up better. I might try to build a container XML document that has all of the objects in it. Then ship that over via a POST.

    Let us know a bit more about what it is you're trying to do.
  • 2. Re: XMLEncoder cutting off data??
    EJP Guru
    Currently Being Moderated
    The XMLEncoder doesn't finish the XML document until you close it (because it donesn't know when you've written the last object). This makes it difficult to use with Sockets - you have to interpose a custom output stream that ignores a close, close the XMLEncoder when you want the reader to start deserializing, and use another one next time you want to send.
  • 3. Re: XMLEncoder cutting off data??
    843790 Newbie
    Currently Being Moderated
    OK... I've fixed the closing issue. However, it is still cutting it off because the data has special characters ( &, CR, LF, etc ) within it. When I noticed that, I wrote a small encoder to URL Encode the data before XMLEncoder encodes it, and it still stops encoding (or decoding) at that exact same spot. I'm not sure if XMLEncoder is breaking, or if XMLDecoder is breaking. That is, I know XMLDecoder cannot handle the data, but I'm not sure what the stream looks like between the applet and the server.
  • 4. Re: XMLEncoder cutting off data??
    843790 Newbie
    Currently Being Moderated
    The HashMap contains String key and Java Object data pairs. The java objects must be serialized (xml encoded) before sending it across the wire. It is not about the sie of the data, and it is via POST. However, the cutoff is occurring on special characters, even if they have been translated into proper XML entities.
  • 5. Re: XMLEncoder cutting off data??
    843790 Newbie
    Currently Being Moderated
    I'm trying to fake what you mention... putting all the data into a "wrapper" XML by actually serializing java objects in a HashMap. Actually, for the call that is failing, there is only one entry in the incoming hash map. Then I'd figure I'd XMLEncode it and violet!, I'd have my XML. Howerver, looking at youranswer I may have the completely wrong idea about how to do this.

    Forget what my code is doing. Would you please show me a code snippet (or link/tutorial/etc.) about how you POST xml to an HTTP Server.
  • 6. Re: XMLEncoder cutting off data?? - How do I handle an ampersand
    843790 Newbie
    Currently Being Moderated
    OK.. I now know XMLEncoder is breaking. The data contains a java bean with a property, href, set to
    http://www.xxx.com/this/that?x=10&y=20
    . I have encoded that to be:
    "http://www.xxx.com/this/that?x=10&y=20"
    , but right after XMLEncoding, I have
    ...<void property="href"><string>http://www.xxx.com/this/that?x=10
    and that is the end of the stream! XMLEncoder doesn't seem to handle entities correctly, even when explicitly coded in the data.

    Does anyone know a work-around

    Edited by: Java-Joe-Morgan on Nov 25, 2009 2:32 PM

    Edited by: Java-Joe-Morgan on Nov 25, 2009 2:33 PM
  • 7. Re: XMLEncoder cutting off data?? - How do I handle an ampersand
    796085 Newbie
    Currently Being Moderated
    I would expect XMLEncoder to do any appropriate escaping. Have you tried not pre-encoding the data. i.e. just put ...x=10&y=20 in and see if the XMLEncoder does the right thing with it. It's not much of an XML encoder if it doesn't.
  • 8. Re: XMLEncoder cutting off data?? - How do I handle an ampersand
    843790 Newbie
    Currently Being Moderated
    That's how all this got started. I actually wrote a small test, creating a simple POJO and intentionally setting string data with & <, etc., and it did encode those properly. However, it doesn't explain why it doesn't like this data, which is still a relatively simple POJO, but it has an href property and the search string contains an ampersand. For whatever totally unexplainable reason, it doesn't encode that.

    So.. I'm now going to try to URL encode that string before sending the bean into the encoder. Failing that, I'm going to have to come up with another plan.
  • 9. Re: XMLEncoder cutting off data?? - How do I handle an ampersand
    EJP Guru
    Currently Being Moderated
    I've never had to bother with character encoding with the XMLEncoder, and I've been using it for years. I forgot to mention that w.r.t. the closing problem you have the opposite problem at the other end - the decoder will report an EOF where there isn't one, so you have to switch to another decoder.

    Donated to the world:
    /**
     * RMIXMLSocketOutputStream exists because {@link XMLEncoder} closes its output
     * stream, and RMI/XML clients need the underlying socket to remain open
     * so they can read the response. Therefore the {@link #close} method of this
     * class doesn't close the nested output stream.
     *
     * @author Esmond Pitt
     * @version $Revision: 5 $
     */
    class RMIXMLSocketOutputStream extends FilterOutputStream
    {
         private Logger     logger = Logger.getLogger(this.getClass().getName());
         private Socket     socket;
         
         /**
          * Creates a new instance of RMIXMLSocketOutputStream. 
          * @param socket The underlying Socket
          */
         RMIXMLSocketOutputStream(Socket socket) throws IOException
         {
              super(new BufferedOutputStream(socket.getOutputStream()));
              this.socket = socket;
         }
         
         public void     close() throws IOException
         {
              flush();
              // DO NOT CLOSE out or call super.close().
         }
    }
    
    /**
     * RMIXMLSocketInputStream exists because {@link XMLDecoder} closes its input stream
     * and we need to keep it open, e.g. for the RML/XML server to write the response.
     * Therefore the {@link #close} method of this class
     * doesn't cause a close of the nested stream.
     *
     * @author Esmond Pitt
     * @version $Revision: 5 $
     */
    class RMIXMLSocketInputStream extends FilterInputStream
    {
         private Logger     logger = Logger.getLogger(this.getClass().getName());
         private Socket     socket;
         private ByteArrayInputStream     bais;
         
         /**
          * Creates a new instance of RMIXMLSocketInputStream
          * @param socket The underlying Socket
          */
         public RMIXMLSocketInputStream(Socket socket) throws IOException
         {
              super(new BufferedInputStream(socket.getInputStream()));
              this.socket = socket;
         }
         
         public int     read() throws IOException
         {
              byte[]     buffer = new byte[1];
              int     count = read(buffer);
              if (count < 0)
                   return -1;
              return buffer[0];
         }
         
         public int     read(byte[] buffer, int offset, int count) throws IOException
         {
              // This kludge is here so the sender doesn't have to call Socket.shutdownOutput().
              if (this.bais == null)
              {
                   DataInputStream     in = new DataInputStream(super.in);
                   ByteArrayOutputStream     baos = new ByteArrayOutputStream();
                   DataOutputStream     out = new DataOutputStream(baos);
                   String     line;
    //               logger.info("entering readLine()");
                   boolean     eof = false;
                   while ((line = in.readLine()) != null)
                   {
    //                    logger.info("read "+line);
                        out.writeBytes(line+"\n");
                        if (line.indexOf("</java>") >= 0)
                             break;
                   }
    //               logger.info("read </java>: available="+in.available());
                   out.close();
                   if (line == null)
                        if (baos.size() == 0)
                   {
                        // Hard EOF
                        return -1;
                   }
                   this.bais = new ByteArrayInputStream(baos.toByteArray());
              }
              int     readCount = bais.read(buffer, offset, count);
              if (readCount <= 0)
                   this.bais = null;
              return readCount;
         }
         
         public void     close() throws IOException
         {
              // Does *not* close super.in or call super.close().
         }
         
    }
  • 10. Re: XMLEncoder cutting off data?? - How do I handle an ampersand
    843790 Newbie
    Currently Being Moderated
    Hi,

    Did you ever manage to fix this problem? I am having the same issue now where any special chaaracters such as & " etc cut off the xml.
  • 11. Re: XMLEncoder cutting off data?? - How do I handle an ampersand
    EJP Guru
    Currently Being Moderated
    Did you try not pre-encoding as I suggested? As I said above, I've been using this thing for years, never noticed a problem with &, and I certainly don't pre-encode anything.

    The other tip is to always attach an exception listener. This thing tends to eat exceptions otherwise so you don't know what's really going wrong.