This discussion is archived
7 Replies Latest reply: Apr 4, 2012 8:54 AM by fgrieu RSS

How do I report bugs in the JCDK?

fgrieu Newbie
Currently Being Moderated
I have found a clear bug in the Classic JCDK.

How do I report that? I have tried jc3-ri-feedback@sun.com over <strike>three</strike> five weeks ago, and got no answer.
Update: I also made a bug report at [url http://bugreport.sun.com/bugreport] but got no sign that it was processed after two weeks. See my message of Apr 4, 2012 later in the thread for a copy of the automated reply, including a complete report.


Specifically, the problem is with Converter (3.0.3 and former). It is supposed to implement the requirement spelled in the Virtual Machine Specification, section 2.2.3.1: +"The result of an arithmetic expression produced by a Java Card virtual machine must be equal to the result produced by a Java virtual machine, regardless of the input values. A Java Card virtual machine that does not support the int data type must reject expressions that could produce a different result"+. It mostly works: Converter invoked without the -i option issues +"unsupported int type of intermediate value, must cast intermediate value to type short or byte"+ when it encounters an expression that could require runtime support for int in order to meed that requirement. However there are cases where Converter fails to perform these duties. E.g:
boolean int_supported() {
  short x = 1<<14;
  return 0 > ~(x+x); // converter reports no error (wrong)
}
Update: The above function returns true in Java, and false on a Java Card (Classic) platform (without or with int support). Converter (regardless of usage of the -i option) translates the Java bytecode to nonfunctional Java Card (Classic) bytecode, and does not report an error. See my next post for the Java Card (Classic) bytecode.

Edited by: fgrieu on Apr 6, 2012 11:17 AM
  • 1. Re: How do I report bugs in the JCDK?
    safarmer Expert
    Currently Being Moderated
    Hi,

    Are you sure that is incorrect? My interpretation of that clause in the spec is not that the bytecode must be the same, but the resulting arithmetic operation gives the same result. If the result would have been an int without sign extension for instance with int support and a negative short/int without then the error shall be shown.

    Even if you add the intermediate casts, the bytecode from my Eclipse compiler would not run on a smart card without int support, but if you run the code on both a JCVM and a JVM the result is the same with one outer cast added. I don't know if this would constitute a difference in result though since the Eclipse compiler is using i* opcodes rather than s* for shorts.
    boolean int_supported() {
      short x = 1<<14;
      return 0 > (short)(~((short)(x+x)));
    }
    The bytecode for this is:
     0 sipush 16384
     3 istore_1
     4 iload_1
     5 iload_1
     6 iadd
     7 i2s // added cast
     8 iconst_m1
     9 ixor
    10 i2s // added cast
    11 ifge 16 (+5)
    14 iconst_1
    15 ireturn
    16 iconst_0
    17 ireturn
    The Java Card bytecode (not Sun converter) for the exact same code is:
    P:0000   11 40 00         sspush           16384 (0x4000)
    P:0003   30               sstore_1         unknown local variable
    P:0004   1d               sload_1          x (S)
    P:0005   1d               sload_1          x (S)
    P:0006   41               sadd             
    P:0007   02               sconst_m1        
    P:0008   57               sxor             
    P:0009   63 04            ifge             +4 (P:000d)
    P:000b   04               sconst_1         
    P:000c   78               sreturn          
    P:000d   03               sconst_0         
    P:000e   78               sreturn          
    Cheers,
    Shane
  • 2. Re: How do I report bugs in the JCDK?
    fgrieu Newbie
    Currently Being Moderated
    Hi, thanks for your attention.

    I think that I have the same interpretation as you of the requirement +"The result of an arithmetic expression produced by a Java Card virtual machine must be equal to the result produced by a Java virtual machine, regardless of the input values. A Java Card virtual machine that does not support the int data type must reject expressions that could produce a different result"+. I ask that the lack of int support either does not change the result of an expression, or that the expression is rejected. I'm not asking that it does not change the Java Card (Classic) bytecode generated.

    I have verified that my function compiles / converts (without -i option, giving no error or warning) / loads / executes (without runtime exception), and returns false, on a physical Java Card 2.2.1 platform (which happens to be without int support); while it returns true under Java (as it should, per the Java Language Specification, which prescribes that a short is extended to int before addition, and the result is an int).

    The requirement is broken. The Java bytecode for my function produce true when run, but the Java Card (Classic) bytecode produce false. That's a bug of the component which performed this conversion from one to the other (without error or warning), which is Converter.

    That requirement is normally implemented by Converter (issued without the -i option) by aborting with +"unsupported int type of intermediate value, must cast intermediate value to type short or byte"+, when it fails to perform Java bytecode to Java Card (Classic) bytecode conversion demonstrably producing the result mandated by the Java Language Specification (while using only instructions supported regardless of int support). Converter issues that message most often when it should do so (e.g. if the ~ is removed), but here (and in a few other corner cases) it just fails to abort and issue that error message.

    Therefore this lack of abortion and error message is a bug of Converter. Or is there a mistake in my reasoning?

    What happens is that Converter translated Java bytecode that evaluates (x+x) as a 32-bit signed value, into Java Card (Classic) bytecode doing that as a 16-bit signed value. Instead of the expected 32768, the result is -32768. That shows (including after negation) in the outcome of the function. The expression should be rejected (or correctly translated, but it is so complex that Converter understandably does not attempt that).


    Yes, adding casts to short as you did makes the code return false on every platform. That's the expected result per the Java Language Specification. The condition +"could produce a different result"+ in the requirement is no longer met. Converter exhibits no bug with that different input.


    BTW, I'm interested in how you compiled and converted; something at your place performs optimizations that do not occur at mine. I used javac from jdk1.6.0_31 with options -g -target 1.5, and converter 3.0.3, to get (from my original code)
              .method  int_supported()Z 128 {
                   .stack 3;
                   .locals 1;
    
                        L0:     sspush 16384;
                             sstore_1;
                             sconst_0;
                             sload_1;
                             sload_1;
                             sadd;
                             sconst_m1;
                             sxor;
                             if_scmple L2;
                        L1:     sconst_1;
                             goto L3;
                        L2:     sconst_0;
                        L3:     sreturn;
              }
    Notice goto L3 could be replaced by sreturn. And sconst_0 could be removed by replacing if_scmple with ifge (or something on that tune). Both optimizations occur at your place.

    Edited by: fgrieu on Mar 15, 2012 8:57 PM
  • 3. Re: How do I report bugs in the JCDK?
    Sebastien_Lorquet Journeyer
    Currently Being Moderated
    Hi Safarmer,

    anything interesting to add about this? the topic was becoming quite technical and interesting.

    Sebastien
  • 4. Re: How do I report bugs in the JCDK?
    safarmer Expert
    Currently Being Moderated
    Sebastien_Lorquet wrote:
    Hi Safarmer,

    anything interesting to add about this? the topic was becoming quite technical and interesting.

    Sebastien
    I do, but I would like to re-read something first.

    Shane
  • 5. Re: How do I report bugs in the JCDK?
    safarmer Expert
    Currently Being Moderated
    Hi,

    From my understanding, the JCVM spec (specifically the clause you mention) is not referring to the source code that is fed into the converter, but the bytecode that will be executed. It seems that if the converter cannot create JC bytecode that would produce the same result on a JVM as a JCVM then it will fail. On the other hand, if the converter can generate bytecode that is semantically correct and that will produce the same result on both types of VM it will. There are also cases where this theory falls down as the sadd opcode should leave a short on the stack but a cast is required there. I think the confusion comes down to the fact that the s* opcodes do not exist in a JVM and in this case I am not comparing apples to apples, in which case this may be an issue with the converter. Thinking a little more about this, I could go either way. Maybe the spec needs a little more clarification.

    If this is indeed a bug in the converter, then several versions of the JCDK have it as well as converters from other vendors.
    BTW, I'm interested in how you compiled and converted; something at your place performs optimizations that do not occur at mine. I used javac from jdk1.6.0_31 with options -g -target 1.5, and converter 3.0.3, to get (from my original code)
    I used the Eclipse compiler which may do some optimisation that the Sun compiler does not. I used the default settings for Java 6. The CAP file converter is the one that comes with NXP's JCOP Tools.

    Cheers,
    Shane
  • 6. Re: How do I report bugs in the JCDK?
    fgrieu Newbie
    Currently Being Moderated
    Hi Shane,

    indeed, the JCVM specification does not spell what is supposed to implement the requirement in section 2.2.3.1. My reading is that reject in practice is implemented by Converter (issued without the -i option) aborting when it finds Java bytecode in its input that it fails to convert into equivalent Java Card bytecode. The requirement can also be understood as a requirement for the JCVM interpreter, applicable when it encounters an int opcode (implementable by muting the card, among other possibilities). Or at the other end of the spectrum, the requirement could be implemented by an hypothetical compiler fed Java source and producing Java Card bytecode, without intermediate Java bytecode.

    In any case, I maintain there is a clear bug in Converter (issued without the -i option): in my original example it is fed in with Java bytecode that unambiguously produce true, and outputs Java Card bytecode that unambiguously produce false (regardless of int support in the JCVM), without issuing the error message that it is designed to issue precisely when facing that risk (and issues appropriately most of the time).

    The problem can really harm. For example consider this function
    short mix(short x) {
      return (short)(((x<<8)^x)>>>4);
    }
    If "^x" was changed to "+x", or removed, Converter (without -i option) would rightly refuse to convert the code. But as is, this compiles and converts without error or warning, giving the following Java Card bytecode, which runs without a runtime exception
              .method  mix(S)S 128 {
                   .stack 2;
                   .locals 0;
    
                        L0:     sload_1;
                             bspush 8;
                             sshl;
                             sload_1;
                             sxor;
                             sconst_4;
                             sushr;
                             sreturn;
              }
    When fed 0x0100, the function is supposed to output 0x1010. The Java bytecode does that. But the Java Card bytecode produce 0x0010 instead. A developer using regression tests performed on the basis of the Java bytecode (rather than the Java Card bytecode) will likely miss the problem. That could occur if tests are performed using a JCOP Java Card simulator, which was based on a JVM running the Java bytecode last time I looked at it (that's when it was an IBM product).


    I used the Eclipse compiler which may do some optimisation that the Sun compiler does not.
    Yes, that seems to be the case. Even if the question in the title remains unanswered, and the bug in Converter unfixed, I'm glad that I learnt that valuable information.

    Regards,

    Edited by: fgrieu on Mar 17, 2012 8:41 AM
  • 7. Re: How do I report bugs in the JCDK?
    fgrieu Newbie
    Currently Being Moderated
    I have made an expanded, formal bug report at [url http://bugreport.sun.com/bugreport] but got no answer so far. Here is a copy.
    Date: Thu, 22 Mar 2012 15:13:45 -0700 (MST)
    From: "IncidentDaemon@sun.com" <IncidentDaemon@sun.com>
    Message-ID: <16868914.01332454425769.JavaMail.daemon@localhost>
    Subject: Your Report (Review ID: 2211712) - Converter may generate invalid Java Card bytecode for expressions with ^ & | ~
    X-IM-Review-ID: 2211712
    
    (..)
    Your report has been assigned an internal review ID of 2211712, which is NOT visible on the Sun Developer Network (SDN).
    
    Please be aware that the large volume of reports we receive sometimes prevents us from responding individually to each message.
    (..)
    
    ---------------------------------------------------------------
    
    
    Date Created: Thu Mar 22 15:13:44 MST 2012
    Type:        bug
    Customer Name:   Francois Grieu
    Customer Email:  (..)
    SDN ID:       
    status:      Waiting
    Category:    javacard
    Subcategory: tools
    Company:     (..)
    release:     jc22
    hardware:    x86
    OSversion:   win_xp
    priority:    4
    Synopsis:    Converter may generate invalid Java Card bytecode for expressions with ^ & | ~
    Description:
     FULL PRODUCT VERSION :
    Any JCDK 2 or Classic 3, up to current 3.0.3
      Bug is with converter 3.0.3 and earlier.
    JDK/JRE 1.7.0_03 [irrelevant]
    
    ADDITIONAL OS VERSION INFORMATION :
    Windows XP Pro SP3 [irrelevant]
    
    A DESCRIPTION OF THE PROBLEM :
    For some expressions, converter outputs (without error or warning) Java Card bytecode that, when run, gives a result different
    from the Java bytecode that it is given as input.
    
    This violates the JCVM requirement 2.2.3.1 (from JCVMspecCLASSIC-3_0_1-RR.pdf or JavaCard222VMspec.pdf): "The result of an
    arithmetic expression produced by a Java Card virtual machine must be equal to the result produced by a Java virtual machine,
    regardless of the input values. A Java Card virtual machine that does not support the int data type must reject expressions
    that could produce a different result."
    
    This occurs in this example
    
        // when input is 0x0100 this outputs 0x1010 under Java
        // but outputs 0x0010 on a Java Card (Classic)
        short mix(short x) {
          return (short)(((x<<8)^x)>>>4);
        }
    
    The javac compiler correctly translates this into Java bytecode which evaluates the <<, ^, and >>> operators over 32 bits (as
    unambiguously specified by the Java Language specification). That Java Bytecode, when run on a JVM, produces the expected
    result.
    
    Converter translates that Java bytecode to Java Card (Classic) bytecode that evaluates the  <<, ^, and >>> operators over 16
    bits. Converter issues no error or warning. That Java Card Bytecode, when run on a (Classic) JCVM, produces invalid result.
    
    This occurs regardless of if the -i option of converter (controlling support of int) is used or not; and regardless of if the
    Java Card Virtual Machine has int support or not.
    
    The bug is 100% reproducible, but in practice is rarely triggered. I have seen it only in expressions involving an operator
    among ^ | & ~ and at least 2 other operators, one (evaluated earlier) among << + - *  and the other (evaluated later)
    / % >>> >> ! == != < > <= >=.
    
    The bug is a serious nuisance, worse than a compiler bug: regression tests of a Java library can not detect the issue. It can
    even be missed when testing the final Applet using a Java Card simulator, if this is one based on the Java bytecode (rather
    than on the Java Card bytecode), as is the case for the JCOP Eclipse plugin 3.1.x from IBM.
    
    STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
    Add the following code fragment to a working Java Card (Classic) applet, and build it as usual (compilation with javac,
    conversion with converter).
    
        // simplified version of code that first exhibited the problem
        // when input is 0x0100 this should output 0x1010
        // but output 0x0010 when run on a Java Card (Classic)
        short mix(short x) {
          return (short)(((x<<8)^x)>>>4);
        }
    
    Examine the Java Card bytecode produced for this function, by using the -out JCA option of converter and searching "short mix"
    in the .jca output file. See below for expected and actual result.
    
    Note: We proceed by examination of the Java Card bytecode generated. For the incredulous, it can be run on a physical Java
    Card 2(.x.x), or 3 Classic, or a simulation thereof using a (Classic) JCVM. A complete example applet is provided.
    
    EXPECTED VERSUS ACTUAL BEHAVIOR :
    EXPECTED -
    If Converter is issued without the -i option, it should issue the error "unsupported int type of intermediate value, must cast
    intermediate value to type short or byte". This error message correctly occurs if ^ is replaced by + in the code fragment.
    
    If Converter is issued with the -i option (indicating that the target JCVM supports int), it should generate Java Card
    bytecode using the "ishl", "ixor" and "iushr" opcodes.
    
    Note: Some other behaviors of converter would be acceptable, but are not observed.
    ACTUAL -
    Converter gives no error or warning, and produce the following Java Card bytecode, using the "sshl", "sxor" and "sushr"
    opcodes. This is independent of passing the -i option to Converter, or not.
    
    .method  mix(S)S 128 {
                .stack 2;
                .locals 0;
     
                 L0:    sload_1;
                        bspush 8;
                        sshl;
                        sload_1;
                        sxor;
                        sconst_4;
                        sushr;
                        sreturn;
            }
    
    When run on an actual JCVM, that Java Card (Classic) bytecode produce an invalid result for input 0x0100: the value produced
    by sshl for 0x0100<<8 is 0x0000 instead of 0x00010000, this propagates and the output is 0x0010 instead of 0x1010. This is
    regardless of int support on the JCVM.
    
    
    ERROR MESSAGES/STACK TRACES THAT OCCUR :
    There is no error message. Here is an actual build of the example code:
    
    ]C:\PROGRA~1\Java\jdk1.7.0_03\bin\javac.exe -g -source 1.4 -target 1.4 -bootclasspath C:\java_fgr\JCDK3.0.3_ClassicEdition_RR\lib\api_classic.jar my\test\test.java
    
    ]C:\java_fgr\JCDK3.0.3_ClassicEdition_RR\bin\converter.bat -out CAP EXP JCA -exportpath C:\java_fgr\JCDK3.0.3_ClassicEdition_RR\api_export_files -applet 0xF0:0xF0:0xF0:0xF0:0xF0:0x01 test my.test 0xF0:0xF0:0xF0:0xF0:0xF0 1.0
    [ INFO: ] Converter [v3.0.3]
    [ INFO: ]     Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
    
    
    [ INFO: ] conversion completed with 0 errors and 0 warnings.
    
    REPRODUCIBILITY :
    This bug can be reproduced always.
    
    ---------- BEGIN SOURCE ----------
    // A complete applet showing the bug.
    // After installation and selection, it is supposed
    // to give SW = 0x6F30 on any APDU.
    // Instead it gives SW = 0x6F31.
    package my.test;
    
    import javacard.framework.APDU;
    import javacard.framework.Applet;
    import javacard.framework.ISOException;
    import javacard.framework.JCSystem;
    
    public class test extends Applet {
    
        // when input is 0x0100 this outputs 0x1010 under Java
        // but outputs 0x0010 on a Java Card (Classic)
        short mix(short x) {
          return (short)(((x<<8)^x)>>>4);
        }
    
        // when input is 1052 this outputs 2719 under Java
        // but outputs 1308 on a Java Card (Classic)
        short bad(short x) {
          return (short)(((x*997)|x)%9973);
        }
    
        // this outputs false under Java
        // but outputs true on a Java Card (Classic)
        boolean bug() {
          short x = 1<<14;
          return 0 < ~(x+x);
        }
    
        // the rest is a minimal testbed for the above two functions
        
        public test(byte[] bArray, short bOffset, byte bLength) {
        }
    
        public static void install(byte[] bArray, short bOffset, byte bLength) {
            new test(bArray, bOffset, bLength).register();
        }
    
        public boolean select() {
            return true;
        }
    
        public void deselect() {
        }
    
        public void process(APDU inApdu) throws ISOException {
            if ( selectingApplet() )
                ISOException.throwIt( (short)0x9000 );
            if ( mix( (short)0x0100 ) != (short)0x1010 )
                ISOException.throwIt( (short)0x6F31 );
            if ( bad( (short)1052 ) != (short)2719 )
                ISOException.throwIt( (short)0x6F32 );
            if ( bug() )
                ISOException.throwIt( (short)0x6F33 );
            ISOException.throwIt( (short)0x6F30 );
        }
    }
    
    ---------- END SOURCE ----------
    
    CUSTOMER SUBMITTED WORKAROUND :
    Perform detailed unit tests on the basis of the actual Java Card bytecode.
    
    Perform line-by-line code review to locate potentially offending expressions, and try to rewrite them in an a manner that
    allows computation using 16-bit signed values; that might be (not tested !):
        short mix2(short x) {
          return (short)((x<<4)^(x>>4));
        }
    which perhaps works correctly, including without int support. When that's not possible, perform an addition of the constant 0
    on the result of operator ^ & | ~; that might be:
        short bad2(short x) {
          return (short)((((x*997)|x)+0)%9973);
        }
    which works correctly with int support, and correctly cause converter (issued without -i) to abort with error. That workaround
    is not safe: it might depend on the java compiler used.
    
    SUPPORT :
    YES
    workaround:  
    comments:    (..)

Legend

  • Correct Answers - 10 points
  • Helpful Answers - 5 points