Forum Stats

  • 3,837,399 Users
  • 2,262,255 Discussions
  • 7,900,269 Comments

Discussions

How to read special characters in JTextField

User_KZASH
User_KZASH Member Posts: 39 Blue Ribbon
edited Nov 16, 2018 2:35PM in Java Programming

have a scanner that can read 2D GS1-DataMatrix codes.

I can scan codes in Notepad++ and I can see that the special character used by GS1 codes(FNC1) are transmited(the GS before 2210, 1D in HEX - first image)

Now I'm trying to read the same GS1 code from Java but isn't working, the FNC1 is not seen by Java. In Java I only see "01095011010209171719050810ABCD12342110".

I transformed the string to HEX hoping that there is an invisible character but the result is the same, FNC1 is not in HEX either(second image).

This is the test code:

import java.awt.Font;import java.io.UnsupportedEncodingException;import java.math.BigInteger;import java.util.logging.Level;import java.util.logging.Logger;import javax.swing.JButton;import javax.swing.JFrame;import javax.swing.JTextArea;import javax.swing.event.DocumentEvent;import javax.swing.event.DocumentListener;import javax.swing.text.BadLocationException;import javax.swing.text.Document;public class GS1DataMatrix {    public static void main(String[] args) {        JFrame f=new JFrame();//creating instance of JFrame          Font font = new Font("Courier New", Font.PLAIN, 16);        JTextArea jtf2 = new JTextArea(); // used to hold the HEX data        jtf2.setBounds(10,250,900, 200);        jtf2.setFont( font.deriveFont( 24.0f) );        jtf2.setLineWrap(true);        f.add(jtf2);//adding button in JFrame          JTextArea jtf1 = new JTextArea(); // scan area for the DataMatrix scanner        jtf1.setBounds(10,10,900, 200);        jtf1.setFont( font.deriveFont( 24.0f) );        jtf1.getDocument().addDocumentListener(new DocumentListener() {            @Override            public void insertUpdate(DocumentEvent e) {                update(e);            }            @Override            public void removeUpdate(DocumentEvent e) {                update(e);            }            @Override            public void changedUpdate(DocumentEvent e) {                update(e);            }            public void update(DocumentEvent e) {                try {                    Document doc = (Document)e.getDocument();                    String hex = String.format("%040x", new BigInteger(1, doc.getText(0, doc.getLength()).getBytes("UTF8"))); // transform to HEX                    jtf2.setText(java.util.Arrays.toString(hex.split("(?<=\\G..)"))); // split hex data by 2 characters                    jtf1.selectAll();                } catch (Exception ex) {                    Logger.getLogger(GS1DataMatrix.class.getName()).log(Level.SEVERE, null, ex);                }            }        });        f.add(jtf1);//adding button in JFrame          f.setSize(1000,500);        f.setLayout(null);        f.setVisible(true);    }}

First image: this is how Notepad++ reads FNC1(GS special character on black background):

n++

Secnond image, Java result (HEX value is down):

enter image description here

Third image: notepad++ hex dump showing FNC1 as 1D in HEX at every scan:

pastedImage_2.png

I also tried to read directly from System.in but the result is the same.

Tests were conducted on a Win10 with locale English(US), using Java 11 and Java 8

I tried a lot of different options like "file.encoding" and "client.encoding.overide" but none worked.

Thank you.

Answers

  • Unknown
    edited Nov 15, 2018 12:44PM

    That third image is NOT the same file as what the first image shows.

    Notice that it shows LOWER CASE abcd while the first image shows UPPER CASE.

    So while I'm sure you have an issue I won't spend anymore time looking at this until I know for sure I am really looking at the SAME file for all images.

    I transformed the string to HEX hoping that there is an invisible character but the result is the same,

    But the 1D group separator character is a control character and NOT a text character so it gets filtered out.

    1. Use a GUI debugger like NetBeans

    2. step through the code line by line

    3. examine the variables and expressions for each line

    You should be able to see where the character is getting filtered.

  • User_KZASH
    User_KZASH Member Posts: 39 Blue Ribbon
    edited Nov 15, 2018 2:52PM

    You are correct, I replaced that image, I have no idea how that lowercase text got there, probably I pressed some shortcut key to make text lowercare in Notepad++.

    All tests are made with the same image code, this one(that I have it from the GS1 original guideline https://www.gs1.org/docs/barcodes/GS1_DataMatrix_Guideline.pdf ):

    pastedImage_0.png

    I also tried a much simpler variant like this:

            System.err.println("Default char set: "+Charset.defaultCharset().toString());                    try {            while(true) {                int i = System.in.read();                String hex=Integer.toHexString(i);                System.out.println(" hex "+hex + " char "+(char)i+" int "+i);            }        } catch (IOException ex) {            Logger.getLogger(GS1DataMatrix.class.getName()).log(Level.SEVERE, null, ex);        }        }

    The result was the same, this is an example output:

    Default chs: UTF-801095011010209171719050810ABCD12342110 hex 30 char 0 int 48 hex 31 char 1 int 49 hex 30 char 0 int 48 hex 39 char 9 int 57 hex 35 char 5 int 53 hex 30 char 0 int 48 hex 31 char 1 int 49 hex 31 char 1 int 49 hex 30 char 0 int 48 hex 31 char 1 int 49 hex 30 char 0 int 48 hex 32 char 2 int 50 hex 30 char 0 int 48 hex 39 char 9 int 57 hex 31 char 1 int 49 hex 37 char 7 int 55 hex 31 char 1 int 49 hex 37 char 7 int 55 hex 31 char 1 int 49 hex 39 char 9 int 57 hex 30 char 0 int 48 hex 35 char 5 int 53 hex 30 char 0 int 48 hex 38 char 8 int 56 hex 31 char 1 int 49 hex 30 char 0 int 48 hex 41 char A int 65 hex 42 char B int 66 hex 43 char C int 67 hex 44 char D int 68 hex 31 char 1 int 49 hex 32 char 2 int 50 hex 33 char 3 int 51 hex 34 char 4 int 52 hex 32 char 2 int 50 hex 31 char 1 int 49 hex 31 char 1 int 49 hex 30 char 0 int 48 hex a char  int 10

    I tried debugging with F7, but there is nothing where FNC1 should be.

    But if this group separator gets filtered, how can I make Java to stop filtering it?

    I didn't mention this in the original post but this can be relevant: sometimes, in my Java tests I get a special character in the place where FNC1 should be.

    Example output  scanning the same code 8 times with Java using System.in:

    01095011010209171719050810ABCD12342110 - no FNC1

    01095011010209171719050810ABCD12342110 - no FNC1

    01095011010209171719050810ABCD12342110 - no FNC1

    01095011010209171719050810ABCD12342110 - no FNC1

    01095011010209171719050810ABCD1234†2110 - I got a character coded in HEX as fffd

    01095011010209171719050810ABCD1234�2110 - I got a character coded in HEX as 3f

    01095011010209171719050810ABCD12342110 - no FNC1

    01095011010209171719050810ABCD12342110 - no FNC1

    This is very weird because the image is the same but the result is changing.

    This is not heappening in Notepad++, here I scanned 50 consecutive times and every time FNC1 is 1D in HEX.

    Thank you.

  • Unknown
    edited Nov 15, 2018 4:23PM
    I also tried a much simpler variant like this:

    That uses System.in which is an input stream - you won't get any binary/control characters using that.

       JTextArea jtf2 = new JTextArea(); // used to hold the HEX data  

    A text area is NOT going to hold 'HEX data' - it holds text.

    The character you mention is NOT in the text field to begin with so you won't be able to get it out to display it.

    Why would you use a text area for an image anyway?

    I suggest you start over and explain WHAT PROBLEM you are really trying to solve - the code you posted doesn't seem to be realistic.

  • User_KZASH
    User_KZASH Member Posts: 39 Blue Ribbon
    edited Nov 16, 2018 6:21AM
    That uses System.in which is an input stream - you won't get any binary/control characters using that.Why would you use a text area for an image anyway?

    I don't parse a image, I use a hand held scanner like the ones in the supermarket, not like the scanners that scan paper and send it to the computer as jpg.

    The image is hardware processed in the scanner and I receive text with some special characters.

    It's more like a keyboard that like a scanner, this is why I tried System.in and JTextArea.

    I don't know what else to use, what can I use to get bynary control from such a device  ?

  • Unknown
    edited Nov 16, 2018 11:19AM
    I don't know what else to use, what can I use to get bynary control from such a device ?

    You need to use a process that handles binary if you use control characters that have no 'inherent text meaning (my term.

    For example a TAB character is a single control character but it has an 'inherent text' that can result in either shifting the next character position to the right (in some implementations) or inserting one or more space characters to effect the same thing.

    The GS character isn't generally used by any text processors that I am aware of.

    The barcode pdf file you linked to SPECIFICALLY designates those codes as having special meaning for barcode scanners to enable them to process/parse the barcode properly. In that sense I would not expect those codes to even be passed on after parsing but only the barcode groups themselves.

    I suggest you talk to the scanner developer/manufacturer and/or review their documentation to understand why they are passing those control characters beyond the parser.

    That doc seems to support what I said above about those characters not having meaning for 'text' but only for the barcode format itself.

    So if you need the raw barcode data you will need to get the BINARY text representation.

  • User_KZASH
    User_KZASH Member Posts: 39 Blue Ribbon
    edited Nov 16, 2018 2:35PM

    The first FNC1 is used to recognize GS1 codes and is not send to Java, the rest of the FNC1's are used as group separator and must get to the Java application, otherwise I won't know how to separate my keys and values.

    I downloaded javahexeditor - the binary version and here FNC1'a are fine.

    I will download the source code and try to reproduce what they are doing.

    Thank you.