8 Replies Latest reply: May 1, 2013 5:45 PM by Anuj Dwivedi-Oracle RSS

    B2B 11.1.1.6  -unable to parse special character

    user390678
      We are trying to process Inbound EDI X12 850 PO file in Oracle SOA 11g version 11.1.1.6. The EDI file has a special character � (0xfffd) as segment delimiter and due to this the B2B component is unable to parse the edi file to xml. The xml received in SOA composite is incomplete.

      If we change delimiter to ~, then B2B process successfully.
      This same segment delimiter is processed fine in 11.1.1.3 but doesn’t get parsed in 11.1.1.6.

      We tried setting the B2B properties in the SOA server with encoding as UTF-16 but it did not help (oracle.tip.adapter.b2b.encoding = UTF-16).



      Please advise.
        • 1. Re: B2B 11.1.1.6  -unable to parse special character
          987937
          Under the Document Protocol setup in B2B did you try changing the delimiter to "~" and pass in the original file? Does that work?
          • 2. Re: B2B 11.1.1.6  -unable to parse special character
            user390678
            It works fine for "~". The problem happens for � (0xfffd).
            • 3. Re: B2B 11.1.1.6  -unable to parse special character
              user390678
              This segment delimiter ‘0xfffd’ processed fine in 11.1.1.3 but doesn’t completely get parsed in 11.1.1.6.

              Please see translated XML file. It ends at </Segment-ST>

              <?xml version="1.0" encoding="UTF-8"?><Transaction-850 xmlns="http://www.edifecs.com/xdata/100" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" XDataVersion="2.0" Standard="X12" Version="V4010" CreatedDate="2013-04-18T20:16:07" CreatedBy="XEngine_2956" GUID="{53944088-A886-11E2-8A38-005056B10C3C}"><Internal-Properties><Data-Structure Name="Interchange"><Lookup Name="InterchangeControlVersion">00400</Lookup><Lookup Name="InterchangeReceiverID">2035735845 </Lookup><Lookup Name="InterchangeReceiverQual">12</Lookup><Lookup Name="InterchangeSenderID">9254291001 </Lookup><Lookup Name="InterchangeSenderQual">08</Lookup><Lookup Name="InterchangeUsageIndicator">P</Lookup><Lookup Name="Standard">X12</Lookup><Property Name="ElementDelimiter">0x2a</Property><Property Name="InterchangeAckRequested">0</Property><Property Name="InterchangeAuthorizationInfo"> </Property><Property Name="InterchangeAuthorizationInfoQual">00</Property><Property Name="InterchangeChildCount">1</Property><Property Name="InterchangeComponentElementSep">0x3e</Property><Property Name="InterchangeControlNumber">000005105</Property><Property Name="InterchangeControlStandard_RepeatingSeparator">U</Property><Property Name="InterchangeControlVersion">00400</Property><Property Name="InterchangeDate">130129</Property><Property Name="InterchangeReceiverID">2035735845 </Property><Property Name="InterchangeReceiverQual">12</Property><Property Name="InterchangeSecurityInfo"> </Property><Property Name="InterchangeSecurityInfoQual">00</Property><Property Name="InterchangeSenderID">9254291001 </Property><Property Name="InterchangeSenderQual">08</Property><Property Name="InterchangeTime">1445</Property><Property Name="InterchangeTrailerControlNumber">000005105</Property><Property Name="InterchangeUsageIndicator">P</Property><Property Name="SegmentDelimiter">0x2026</Property><Property Name="Standard">X12</Property><Property Name="SubelementDelimiter">0x3e</Property><Property Name="TPName">KMARTHARDLINES</Property><Data-Structure Name="Group"><Lookup Name="GroupReceiverID">2035735845</Lookup><Lookup Name="GroupSenderID">9254291001</Lookup><Lookup Name="GroupVersionNumber">004010</Lookup><Property Name="GroupAgencyCode">X</Property><Property Name="GroupChildCount">200</Property><Property Name="GroupControlNumber">5105</Property><Property Name="GroupDate">20130129</Property><Property Name="GroupID">PO</Property><Property Name="GroupReceiverID">2035735845</Property><Property Name="GroupSenderID">9254291001</Property><Property Name="GroupTime">1445</Property><Property Name="GroupTrailerControlNumber">5105</Property><Property Name="GroupVersionNumber">004010</Property><Property Name="TPName">KMARTHARDLINES</Property><Data-Structure Name="Transaction"><Lookup Name="TransactionID">850</Lookup><Property Name="TransactionChildCount">17</Property><Property Name="TransactionControlNumber">0007</Property><Property Name="TransactionID">850</Property><Property Name="TransactionImplementationReference"></Property><Property Name="TransactionTrailerControlNumber">0007</Property></Data-Structure></Data-Structure></Data-Structure></Internal-Properties><Segment-ST><Element-143>850</Element-143><Element-329>0007�BEG</Element-329></Segment-ST></Transaction-850>
              • 4. Re: B2B 11.1.1.6  -unable to parse special character
                Anuj Dwivedi-Oracle
                First of all, unicode character 0xfffd is a placeholder (known as Replacement Character) and it is used to indicate problems with the data. Whenever a character cannot be displayed with the current system/application character encoding then it is displayed as 0XFFD. You may like to refer -

                http://www.unicodemap.org/details/0xFFFD/index.html
                http://en.wikipedia.org/wiki/Specials_%28Unicode_block%29#Replacement_character

                Now coming back to your question -
                The EDI file has a special character � (0xfffd) as segment delimiter
                Though it is nowhere mentioned specifically but as a best practice, EDI delimiters should be single byte characters (with CRLF as exception). If you are choosing any multibyte character as EDI delimiter (including CRLF) then you are definitely going to end up in a tricky situation one day because EDI files are flat files and transferring them from one system to another may corrupt the file itself as different operating systems (with different encoding and character set conversion rule) may interpret multibyte characters differently.

                So I strongly recommend to use only single byte characters as EDI delimiter.
                If we change delimiter to ~, then B2B process successfully.
                As expected because tilde (~) is a single byte character (unicode 0X007E).
                This same segment delimiter is processed fine in 11.1.1.3 but doesn’t get parsed in 11.1.1.6.
                I don't think that Oracle B2B ever supported any multibyte character as EDI delimiter and specially this character must not have been supported. If it ever worked with B2B then it might have been a bug (which is the least possible) or you are overlooking some details. If you can forward your PS2 (11.1.1.3) agreement export along with the sample payload which worked in your environment, to my id then I may quickly verify. You may also consider logging a SR with support if you need formal confirmation from Oracle on this.

                Regards,
                Anuj
                • 5. Re: B2B 11.1.1.6  -unable to parse special character
                  user390678
                  Hi Anuj,

                  We are getting these files from customers so I can't change the separator values at wish. These files are successfully processed by 11GR1 PS2 SOA 11.1.1.3.0 which is our existing production server.

                  Now we are trying to create another server 11GR6 PS5 SOA 11.1.1.6.5 and these files are not getting processed in this version.
                  SR is also open on this issue and Bug is logged with the development team. I am also sending you 11.1.1.3.0 agreement export and raw data via email.

                  Please advise.


                  Thanks.
                  • 6. Re: B2B 11.1.1.6  -unable to parse special character
                    Anuj Dwivedi-Oracle
                    Thanks for sending the artifacts. Will check them and let you know my comments.

                    Regards,
                    Anuj
                    • 7. Re: B2B 11.1.1.6  -unable to parse special character
                      user390678
                      hi Anuj,

                      The segment separator for which this issue is happening is "0x2026" (a.k.a. "…")
                      http://www.unicodemap.org/details/0x2026/index.html


                      B2B successfully translates ISA, GS and ST segments but after these segments B2B fails. Please see application message payload.

                      <?xml version="1.0" encoding="UTF-8"?>
                      <Transaction-850 xmlns="http://www.edifecs.com/xdata/100" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" XDataVersion="1.0" Standard="X12" Version="V5010" CreatedDate="2013-04-29T14:54:39" CreatedBy="XEngine_2956" GUID="{3C001FAC-B0FE-11E2-8436-005056B10C3C}">
                      <Internal-Properties>
                      <Data-Structure Name="Interchange">
                      <Lookup Name="InterchangeControlVersion">00501</Lookup>
                      <Lookup Name="InterchangeReceiverID">00116520M </Lookup>
                      <Lookup Name="InterchangeReceiverQual">ZZ</Lookup>
                      <Lookup Name="InterchangeSenderID">5032337598 </Lookup>
                      <Lookup Name="InterchangeSenderQual">12</Lookup>
                      <Lookup Name="InterchangeUsageIndicator">P</Lookup>
                      <Lookup Name="Standard">X12</Lookup>
                      <Property Name="ElementDelimiter">0x2a</Property>
                      <Property Name="InterchangeAckRequested">0</Property>
                      <Property Name="InterchangeAuthorizationInfo"> </Property>
                      <Property Name="InterchangeAuthorizationInfoQual">00</Property>
                      <Property Name="InterchangeChildCount">1</Property>
                      <Property Name="InterchangeComponentElementSep">0x3e</Property>
                      <Property Name="InterchangeControlNumber">000006132</Property>
                      <Property Name="InterchangeControlStandard_RepeatingSeparator">^</Property>
                      <Property Name="InterchangeControlVersion">00501</Property>
                      <Property Name="InterchangeDate">121015</Property>
                      <Property Name="InterchangeReceiverID">00116520M </Property>
                      <Property Name="InterchangeReceiverQual">ZZ</Property>
                      <Property Name="InterchangeSecurityInfo"> </Property>
                      <Property Name="InterchangeSecurityInfoQual">00</Property>
                      <Property Name="InterchangeSenderID">5032337598 </Property>
                      <Property Name="InterchangeSenderQual">12</Property>
                      <Property Name="InterchangeTime">1436</Property>
                      <Property Name="InterchangeTrailerControlNumber">000006132</Property>
                      <Property Name="InterchangeUsageIndicator">P</Property>
                      <Property Name="RepeatingSeparator">0x5e</Property>
                      <Property Name="SegmentDelimiter">0x2026</Property>
                      <Property Name="Standard">X12</Property>
                      <Property Name="SubelementDelimiter">0x3e</Property>
                      <Property Name="TPName">FREDMEYER</Property>
                      <Data-Structure Name="Group">
                      <Lookup Name="GroupReceiverID">00116520M</Lookup>
                      <Lookup Name="GroupSenderID">5032337598</Lookup>
                      <Lookup Name="GroupVersionNumber">005010VICS</Lookup>
                      <Property Name="GroupAgencyCode">X</Property>
                      <Property Name="GroupChildCount">2</Property>
                      <Property Name="GroupControlNumber">1453</Property>
                      <Property Name="GroupDate">20121015</Property>
                      <Property Name="GroupID">PO</Property>
                      <Property Name="GroupReceiverID">00116520M</Property>
                      <Property Name="GroupSenderID">5032337598</Property>
                      <Property Name="GroupTime">1436</Property>
                      <Property Name="GroupTrailerControlNumber">1453</Property>
                      <Property Name="GroupVersionNumber">005010VICS</Property>
                      <Property Name="TPName">FREDMEYER</Property>
                      <Data-Structure Name="Transaction">
                      <Lookup Name="TransactionID">850</Lookup>
                      <Property Name="TransactionChildCount">469</Property>
                      <Property Name="TransactionControlNumber">000002231</Property>
                      <Property Name="TransactionID">850</Property>
                      <Property Name="TransactionImplementationReference">
                      </Property>
                      <Property Name="TransactionTrailerControlNumber">000002231</Property>
                      </Data-Structure>
                      </Data-Structure>
                      </Data-Structure>
                      </Internal-Properties>
                      <Segment-ST>
                      <Element-143>850</Element-143>
                      <Element-329>000002231…BEG</Element-329>
                      <Element-1705>00</Element-1705>
                      </Segment-ST>
                      </Transaction-850>
                      • 8. Re: B2B 11.1.1.6  -unable to parse special character
                        Anuj Dwivedi-Oracle
                        Yes, now you are correctly pointing that it is horizontal ellipsis (...) which is being used as Segment Delimiter. All my initial tests and investigations prove that it is a bug in B2B so please continue to work with support. Meanwhile I will run few more tests to confirm that it is actually a bug.

                        Regards,
                        Anuj