4 Replies Latest reply on Feb 17, 2013 6:31 PM by Sergiusz Wolicki-Oracle

    €-Sign and WE8ISO8859P1 database used for Weblogic Web Application

      We are currently using WE8ISO8859P1 as database characterset on Linux x86-64 (RHEL5). We know that this is not ideal and are evaluating conversion to WE8MSWIN1252. The database is used for Weblogic Java Application. End Users are using Web Browser on Windows Desktops with codepage 1252.

      We are following MOS note for CS conversion between WE8ISO8859P1 to WE8MSWIN1252. During characterset scan, we realized that we have lots of hex80 (€) characters stored which are specified as "Lossy".

      The question we have now is:
      1) How is it possible that the end user can input a "€" sign in the web application forms and that this character is correctly stored as hex80 in WE8ISO8859P1 database? Weblogic Application is using JDBC or We thought that JDBC is translating characters between local application node locale UTF8 to WE8ISO8859P1 and thereby converting hex80 to hex BF (question mark). We would like to understand why the current setup with P1 characterset and weblogic web application is working for correctly for end users.

      2) Next question is: can we guarantee that end users can still input "€" sign in web application on windows desktop browser and that this is stored correctly (hex80) in the database after converting the database to WE8MSWIN1252?


      DB Server:
      OS: Linux x86-64 (RHEl5)
      OS locale: en_US.UTF8
      DB Characterset: WE8ISO8859P1

      Weblogic Application Server:
      OS: Linux x86-64 (RHEl5)
      OS locale: en_US.UTF8
      User Property: user.language=de

      We have already read everything on MOS regarding €, question mark, NLS, JDBC, etc. but fail to understand why the current "incorrect" setup is working for end users.

      Best regards,
        • 1. Re: €-Sign and WE8ISO8859P1 database used for Weblogic Web Application
          Zoltan Kecskemethy
          1. The answer is as simple as: It works because the application could save 1252 character set data to the database without any conversion.

          2. Yes. But you need a trick: convert the database to use WE8MSWIN1252 but do not convert any data in it.
          This can be achieved using FROMCHAR parameter of character set scanner.
          csscan FULL=Y FROMCHAR=WE8MSWIN1252 TOCHAR=WE8MSWIN1252 LOG=csscan_from_win1252_to_win1252 CAPTURE=Y ARRAY=1000000 PROCESS=2
          So the trick is that you set FROMCHAR to 1252 even if it is not because you know that your application saved data into the database using this character set. So this way you fix your database character set.

          HTH, Zoltan
          • 2. Re: €-Sign and WE8ISO8859P1 database used for Weblogic Web Application
            Hi Zoltan,

            thanks for your reply.

            Obviously, no conversion is taking place. But when we try the same with e.g. SQL Developer, which uses JDBC as well, we fail when trying to insert a € sign. It always ends up as hex BF (upside question mark). This also what we expect because jdbc is converting between 1252 and p1 and therefore fails. What is the difference between a weblogic web application that uses JDBC and SQL Developer?

            • 3. Re: €-Sign and WE8ISO8859P1 database used for Weblogic Web Application
              Zoltan Kecskemethy
              As far as I know SQL Developer always using UTF8 as client character set. That's why it fails when tried to convert from utf to p1.
              My assumption is that you did not setup NLS_LANG for your weblogic or could not setup the UTF8 for JDBC, so it defaults to the 7-bit ASCII character set (this default could be weblogic version dependent! but I know older versions used ASCII as default for sure - btw you did not specify weblogic version yet only JDBC). So the important part comes here: no conversion from ascii to p1 so the data goes into db in 1252. This is just an assumption. So I would try to dig in deeper in this area...
              • 4. Re: €-Sign and WE8ISO8859P1 database used for Weblogic Web Application
                Sergiusz Wolicki-Oracle

                JDBC does certain optimizations for selected character sets and this optimization may depend on the particular JDBC version. It is possible that this optimized conversion passes the codes through. This also assumes that the Web side of things does a similar pass-through. Therefore, I am not surprised that you may have some 0x80 codes in the database. But in no case you should rely this. Use the method described by Zoltan (csscan FROMCHAR=WE8MSWIN1252 TOCHAR=WE8MSWIN1252) followed by ?/rdbms/admin/csalter.plb to repair the character set declaration of your database. After fixing the database character set declaration, make sure that Weblogic pages are generated as UTF-8. If they are generated as iso-8859-1, the application may stop accepting Windows-specific characters.

                -- Sergiusz