This discussion is archived
1 2 Previous Next 17 Replies Latest reply: Nov 1, 2012 2:57 AM by 938489 RSS

Character encoding issues.

938489 Newbie
Currently Being Moderated
We have a mysql table with the collation and fields all set as utf8_unicode_ci. We have a java application using bonecpconfig for connection pooling. We are trying as below to read the characters but is not showing up correctly. Even tried this useUnicode=true&characterEncoding=utf-8. Just to add on we have also tried without using bone cp just normal jdbc connection is the same. It can read normally english characters only when comes to like e.g. russian characters it has problem and shows up something like this ÐоÑÑавка гÑÑза but in db is this Доставка груза.
BoneCPConfig config = new BoneCPConfig();
config.setJdbcUrl("jdbc:mysql://localhost:3306/db1?characterEncoding=utf-8");
Edited by: 935486 on Oct 28, 2012 7:27 AM
  • 1. Re: Character encoding issues.
    831379 Newbie
    Currently Being Moderated
    This could be a mysql question. In other words, first you must determine that your data is truly using the character encoding you wish.

    How is the table in the database populated with data? Is it entered programmatically using a program with insert statements or did someone populate the table using the mysql client program and passing in a *.sql file? I ask because how the data table is structured won't matter unless the data is fed into the database using the proper encoding to begin with.

    This will populate the database with data from MyData using the Latin charset.
    mysql -u user -ppassword mydatabase < MyData.sql
    This will populate the database with data from MyData using the UTF8 charset.
    mysql -u user -ppassword --default-character-set=utf8 mydatabase < MyData.sql
    And in my experience it apparently doesn't matter what the setting on the table is. In contrast to mysqldump where the default value is utf8, the default value for the default character set for the mysql client is Latin.
  • 2. Re: Character encoding issues.
    938489 Newbie
    Currently Being Moderated
    Yes you are right about the table encoding and field encoding first we were using the standard latin ones.We are inserting from php and just left it as it is. Then we realise it was a problem for java to pick up. So then we change the collation, field type ,db character set to all utf8_unicode_ci. Then when we enter data from php in the db for e.g. the russian character are in their original form now. So we thought it should work now for java but is still the same yet. Just to add on we have tried from phpMyAdmin and did the same select statement it shows in the correct russian letters.

    Edited by: 935486 on Oct 28, 2012 11:27 AM
  • 3. Re: Character encoding issues.
    Kayaman Guru
    Currently Being Moderated
    935486 wrote:
    Then we realise it was a problem for java to pick up.
    It's not a problem for Java. It's a problem for developers if they don't know which encoding they're using.
    So we thought it should work now for java but is still the same yet.
    How are you displaying the text in Java? If you are printing it out to a windows terminal for example, it will most definitely show up as garbled (if it's UTF-8).
  • 4. Re: Character encoding issues.
    938489 Newbie
    Currently Being Moderated
    We are printing out on the linux terminal and it looks like this ÐоÑÑавка гÑÑза where as in the db is stored as Доставка груза. So what are the corrections need to be performed further?
  • 5. Re: Character encoding issues.
    gimbal2 Guru
    Currently Being Moderated
    935486 wrote:
    We are printing out on the linux terminal and it looks like this ÐоÑÑавка гÑÑза where as in the db is stored as Доставка груза. So what are the corrections need to be performed further?
    Perhaps the encoding used by the terminal is just wrong? Encoding problems come in two forms: problems in the data, problems in the displaying of that data. It seems like the data is correctly stored in the database, right?
  • 6. Re: Character encoding issues.
    938489 Newbie
    Currently Being Moderated
    Yes gimbal now the data is stored and retrieve properly via the php codes. Only issue now is in java. Actually what I need is to read the value and prepare as email message to be sent out. But unfortunately when this program run as a daemon via linux platform is generating this sort values.
  • 7. Re: Character encoding issues.
    gimbal2 Guru
    Currently Being Moderated
    So you already sent out a test (HTML) email and the text in the email was garbage?
  • 8. Re: Character encoding issues.
    938489 Newbie
    Currently Being Moderated
    Yes I am sending the email via the java application using this Message emailMessage = new MimeMessage() and the output is also the garbage characters too.
  • 9. Re: Character encoding issues.
    gimbal2 Guru
    Currently Being Moderated
    Then I'd try to see the MIME source of the email sent, or at least the email header. Perhaps it holds a clue what might be wrong. I'm sort of expecting there to be a wrong charset in there.
  • 10. Re: Character encoding issues.
    Tolls Journeyer
    Currently Being Moderated
    Crossposted at Java forums.
    http://www.java-forums.org/advanced-java/64420-reading-utf-8-mysql-not-working-certain-characters.html
  • 11. Re: Character encoding issues.
    gimbal2 Guru
    Currently Being Moderated
    Tolls wrote:
    Crossposted at Java forums.
    http://www.java-forums.org/advanced-java/64420-reading-utf-8-mysql-not-working-certain-characters.html
    Ugh, the responsibility I now feel!
  • 12. Re: Character encoding issues.
    Tolls Journeyer
    Currently Being Moderated
    gimbal2 wrote:
    Tolls wrote:
    Crossposted at Java forums.
    http://www.java-forums.org/advanced-java/64420-reading-utf-8-mysql-not-working-certain-characters.html
    Ugh, the responsibility I now feel!
    :)
    I only did that because you were further down the road I was expecting to travel.

    No pressure...
  • 13. Re: Character encoding issues.
    938489 Newbie
    Currently Being Moderated
    Dear Gimbal before I venture into the encoding of the email body I just tried to send an email and to my surprise it is sending the characters accordingly. Only thing on the terminal is not showing up properly.
  • 14. Re: Character encoding issues.
    gimbal2 Guru
    Currently Being Moderated
    935486 wrote:
    Dear Gimbal before I venture into the encoding of the email body I just tried to send an email and to my surprise it is sending the characters accordingly. Only thing on the terminal is not showing up properly.
    Uh so you not only did not read what I mentioned before, you were also lying in your reply to my question about sending out a test email. I wish you good luck when you pull stunts like that.
1 2 Previous Next

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points