1 2 Previous Next 24 Replies Latest reply: Feb 22, 2013 2:08 AM by Martin1 RSS

    WebDAV encoding problem

    Martin1
      Hi,

      i have XML files which are saved in a XDB-WebDAV directory. When i now read these files the encoding hase changed - why that?

      Encoding before saving the file in WebDAV:
      <?xml version="1.0" encoding="UTF-8" standalone="no" ?>
      Encoding after saving the file in WebDAV:
      <?xml version="1.0" encoding="ISO-8859-15" standalone='no'?>
      The problem what i have when i want to read and save it in the database is that special (french, german, ...) characters ar not saved correctly in the database.
      The database characterset is WE8ISO8859P15.

      Has anybody an idea to solve this problem?

      Best Regards,
      Martin
        • 1. Re: WebDAV encoding problem
          odie_63
          Martin1 wrote:
          i have XML files which are saved in a XDB-WebDAV directory. When i now read these files the encoding hase changed - why that?
          How are you reading the files?
          The encoding you'll see (both in the prolog and the content) depends on the method used to view the files.

          Are you using Object-Relational storage?
          The database can only store data in its own character set (as far as XML DB is concerned), that's why a conversion occurs from the input charset to the db's.
          The problem what i have when i want to read and save it in the database is that special (french, german, ...) characters ar not saved correctly in the database.
          You said the input files have a prolog mentioning UTF-8 but are they really encoded using UTF-8 ?
          • 2. Re: WebDAV encoding problem
            Martin1
            Hi odie,

            thanks for your reply.
            I'm reading the files by
            INSERT INTO fp_epass_pass_antraege VALUES (xdburitype('/home/FP/ePassFL/84239536904120131828493266953553299522.xml').getXML());
            The table fp_epass_pass_antraege was created by dbms_xmlschema.registerSchema and the insert works fine.
            The XML file is really encoded in UTF-8. The problem occurs as i described at the process of saving in the WebDAV directory. At the time of the insert the XML file is already encoded in ISO 8859-15. How can i influence the encoding when saving the file in the WebDAV directory?

            Best Regards,
            Martin
            • 3. Re: WebDAV encoding problem
              Marco Gralike
              :-)

              and insert is not a select (view)... as Odie mentioned "*how* do you see this difference"

              Your "wrong encoding" remark is done via SQL*Plus or Explorer or a Text editor or...???

              Regarding encoding in SQL*Plus have a read at the good example in de FAQ of this XMLDB forum (see link on the first page).

              Regarding encoding rules and possible "conversions" in the XDB repository (access via FTP/HTML/WebDAV client software) see: http://docs.oracle.com/cd/E11882_01/appdev.112/e23094/xdb03usg.htm#ADXDB4065

              Edited by: Marco Gralike on Feb 19, 2013 3:14 PM
              • 4. Re: WebDAV encoding problem
                Martin1
                Hi Marco,

                the difference i can see when i compare the original XML file and that saved in the WebDAV directory.

                When i open the XML file just saved before in the WebDAV directory by a text editor i can see regarding encoding what i described before.
                In the doc you mentioned i can't find a method to influence the encoding at the time of saving in the WebDAV directory.

                Regards,
                Martin
                • 5. Re: WebDAV encoding problem
                  odie_63
                  The order in which (and how) you do things is not entirely clear.

                  Do you have the defautTable mechanism in place when you create the resource via WebDAV, i.e. is the XML file automatically inserted in the XMLType table or are you doing this manually afterwards?

                  Please post the output of :
                  select t.*
                       , dbms_xdb.getContentXMLRef(r.any_path) as xref
                  from resource_view r
                     , xmltable(
                         xmlnamespaces(default 'http://xmlns.oracle.com/xdb/XDBResource.xsd')
                       , '/Resource'
                         passing r.res
                         columns res_name    varchar2(260) path 'DisplayName'
                               , res_charset varchar2(15)  path 'CharacterSet'
                               , res_type    varchar2(30)  path 'ContentType'
                       ) t
                  where equals_path(res, '/home/FP/ePassFL/84239536904120131828493266953553299522.xml') = 1 ;
                  Edited by: odie_63 on 19 févr. 2013 16:00
                  • 6. Re: WebDAV encoding problem
                    Martin1
                    Hi Odie,

                    the following steps i did:
                    1. Creating the WebDAV directory in XDB.
                    2. Copy the XML files via Windows File Explorer into the WebDAV directory.
                    3. dbms_xdb.createResource (for the schema file)
                    4. dbms_xmlschema.registerSchema (object types and table are created)
                    5. insert as i described before (the XML file for the insert is located in the WebDAV directory)

                    The output of your query:
                    RES_NAME,RES_CHARSET,RES_TYPE,XREF
                    84239536904120131828493266953553299522.xml,ISO-8859-15,text/xml,(REFERENCE)

                    Regards,
                    Martin
                    • 7. Re: WebDAV encoding problem
                      odie_63
                      Ok, so apparently you're not using the defaultTable annotation.
                      When i open the XML file just saved before in the WebDAV directory by a text editor i can see regarding encoding what i described before.
                      Do you just double-click on the file directly from the Windows WebDAV explorer?

                      Which characters are problematic, could you give an example of some, with byte codes from the original file and byte codes once in the DB?
                      • 8. Re: WebDAV encoding problem
                        Martin1
                        Hi Odie,

                        yes i open the XML file in WebDAV via UltraEdit.
                        For unexplained reasons the original file and the file in the WebDAV directory are now the same :-)
                        But when i do a query via XMLTYPE ... PATH i get the mutilate text. Problematic are characters i.e. umlaut-äöü (in german) and french characters.

                        Example:
                        Original file (encoding="UTF-8"):
                        Text: Keine Fingerabdrücke
                        Hex: 4B 65 69 6E 65 20 46 69 6E 67 65 72 61 62 64 72 C3 BC 63 6B 65

                        WebDAV file (encoding="ISO-8859-15"):
                        Text: Keine Fingerabdrücke
                        Hex: 4B 65 69 6E 65 20 46 69 6E 67 65 72 61 62 64 72 C3 BC 63 6B 65

                        XMLTYPE PATH:
                        Text: Keine FingerabdrÃŒcke
                        Hex: 4B 65 69 6E 65 20 46 69 6E 67 65 72 61 62 64 72 C3 8C 63 6B 65

                        I hope i gave you enough information.

                        Regards,
                        Martin
                        • 9. Re: WebDAV encoding problem
                          odie_63
                          Martin1 wrote:
                          WebDAV file (encoding="ISO-8859-15"):
                          Text: Keine Fingerabdrücke
                          Hex: 4B 65 69 6E 65 20 46 69 6E 67 65 72 61 62 64 72 <span style="color:red">C3 BC</span> 63 6B 65
                          Are you sure?

                          C3BC is the byte sequence of "ü" in UTF-8, ISO-8859-15 uses a single byte (FC).
                          If that's not a copy/paste mistake, that means the content has been converted back to UTF-8 at some point.

                          XMLTYPE PATH:
                          Text: Keine FingerabdrÃŒcke
                          Hex: 4B 65 69 6E 65 20 46 69 6E 67 65 72 61 62 64 72 C3 8C 63 6B 65
                          Not sure what you mean by "XMLTYPE PATH".
                          Could you post the whole query? Thanks.
                          • 10. Re: WebDAV encoding problem
                            Marco Gralike
                            Ok, so apparently you're not using the defaultTable annotation.
                            The fast check would also have been "is the size of the file in explorer 0 bytes (after a refresh)" (if not = resource, if it is = shredded)
                            Not sure what you mean by "XMLTYPE PATH".
                            He did a query, via Toad or some other means. In short he still didn't look-up the FAQ...otherwise he would have know about "...and my NLS Client setting was..."

                            ;-)

                            Apparently it's time for me to go to bed 8)
                            • 11. Re: WebDAV encoding problem
                              Martin1
                              Here the query
                              SELECT pe.name, pe.bes_kennz
                              FROM fp_epass_pass_antraege pa
                                 , XMLTABLE('/PassAntrag'
                                   PASSING pa.sys_nc_rowinfo$
                                   COLUMNS person     XMLTYPE PATH '/PassAntrag/Person') paan
                                 , XMLTABLE('/'
                                   PASSING paan.person
                                   COLUMNS name       VARCHAR2(512) PATH 'Name'
                                         , bes_kennz  VARCHAR2(200) PATH 'BesKennz') pe;
                              Edited by: Martin1 on Feb 20, 2013 7:40 AM
                              • 12. Re: WebDAV encoding problem
                                Martin1
                                Hi Odie,

                                i used also the select via bfilename. There the code conversion is also not done :-(
                                SELECT XMLTYPE(bfilename('EXTDATA', '84239536904120131828493266953553299522.xml'), nls_charset_id('UTF8'))
                                  FROM dual;
                                Regards,
                                Martin
                                • 13. Re: WebDAV encoding problem
                                  odie_63
                                  You didn't reply about this :
                                  odie_63 wrote:
                                  >
                                  WebDAV file (encoding="ISO-8859-15"):
                                  Text: Keine Fingerabdrücke
                                  Hex: 4B 65 69 6E 65 20 46 69 6E 67 65 72 61 62 64 72 <span style="color:red">C3 BC</span> 63 6B 65
                                  Are you sure?

                                  C3BC is the byte sequence of "ü" in UTF-8, ISO-8859-15 uses a single byte (FC).
                                  If that's not a copy/paste mistake, that means the content has been converted back to UTF-8 at some point.

                                  What tool (which version) are you using to run your queries?

                                  In SQL*Plus, if you're not serializing the XML document explicitly, it relies on the client NLS_LANG variable for the output encoding.
                                  • 14. Re: WebDAV encoding problem
                                    Martin1
                                    1. It's no copy/paste mistake.
                                    2. I use the TOAD (Vers. 11.6).

                                    Regards,
                                    Martin
                                    1 2 Previous Next