Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 29, 2007 11:23 AM

Currently Being Moderated

Hi,

I am facing a problem with special charcter (� symbol) while parsing the XML. My application receives a request xml from another system which has some text notes having � symbol in it. The character encoding i am using at my end is utf-8. When i receive the xml at my end the � symbol gets converted to ? symbol and i can not see the � symbol in the request at my end.

After receiving the xml and doing some data validation i pass it to some other application for further processing. The problem is that when i pass the parsed xml with the ? symbol (which was earlier the � symbol and was converted to ? symbol) to other system it receives 3 ? symbol for each � symbol. This is creating problem at their end and hence throwing some corba exception to us.
I would appreciate if any one can help me in this regard by telling me that is this some problem specific to Java and how to receive the � symbol without conversion to ? symbol. The format used by all the systems end to end is UTF-8.

I am also not sure is it the display of � symbol as ? symbol or ? is shown to me because the charcter � is not recognized

My application is deployed on weblogic 8.1 sp4 and on solaris OS.

regards

Arvind

1. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 29, 2007 11:35 AM (in response to 807603)

Currently Being Moderated

Ok, so the problem is apparent when you receive the request xml. That means that either you get sent garbage, or that your app turns it into garbage upon receival. What is receiving the request, is it through servlet, rmi, corba or something else?
Report Abuse

Like (0)
2. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 29, 2007 11:46 AM (in response to 807603)

Currently Being Moderated

Hi Bryan,

Yes the problem is when i receive the xml. I am receiving this request via JMS as JMS message. I have asked the other team to send me the xml what they have forwarded and i can see that they have � symbol in their outbound xml.
I see that the it is turned into garbage or ? symbol at my end. How can i handle it.
Report Abuse

Like (0)
3. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 29, 2007 11:54 AM (in response to 807603)

Currently Being Moderated

So the problem is the jms forwarding. You can probably solve this if you ask them to send a byte array instead of a string message. I'm not familiar enough with jms on weblogic to say for sure, but it sounds like it's the culprit. And the name is Brynjar :)
Report Abuse

Like (0)
4. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 29, 2007 11:57 AM (in response to 807603)

Currently Being Moderated

Arvind_wmig wrote:
Hi Bryan,

Yes the problem is when i receive the xml.
No! You observe the symptom when your receive the xml.

I am receiving this request via JMS as JMS message. I have asked the other team to send me the xml what they have forwarded and i can see that they have � symbol in their outbound xml.
I see that the it is turned into garbage or ? symbol at my end. How can i handle it.
1) Make sure that when converting to bytes the 'other team' use the encoding specified in the header of the XML or utf-8 if not specified.
2) Make sure that you use the encoding specified in the header of the XML or utf-8 if not specified when converting the bytes to characters.
3) Make sure the font used to display the content has a glyph for the � sign,
Report Abuse

Like (0)
5. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 30, 2007 10:51 AM (in response to 807603)

Currently Being Moderated

I want to make a point clear that the application i am working on has a lot of interfaces managed by different vendors. I want to have a solution that i can put at my side rather than changing the end to end flow.

I will ask other teams for a change only when it seems that we do not have any other way to handle it.
Report Abuse

Like (0)
6. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

807603 Dec 30, 2007 11:03 AM (in response to 807603)

Currently Being Moderated

Arvind_wmig wrote:
I want to make a point clear that the application i am working on has a lot of interfaces managed by different vendors. I want to have a solution that i can put at my side rather than changing the end to end flow.

I will ask other teams for a change only when it seems that we do not have any other way to handle it.
Then you need to establish that everyone (including you) is handling the XML conversion to/from bytes properly and that YOU have a glyph for � in the font being used to display the �.
Report Abuse

Like (0)
7. Re: Encoding problem while parsing XML. � symbol converts to ? symbol

795426 Dec 31, 2007 8:49 AM (in response to 807603)

Currently Being Moderated

When random characters in your strings get turned into '?', that is a good sign that somewhere along the line, someone is using the platform's default encoding (rarely if ever UTF-8) to encode/decode a string instead of specifying a specific encoding to use.
Report Abuse

Like (0)

Go to original post