1 2 Previous Next 27 Replies Latest reply: Jun 18, 2010 8:26 AM by 843811 Go to original post RSS
      • 15. Re: AES CTR Decryption
        843811
        I see the problem! The comment in the Javascript
         // use AES itself to encrypt password to get cipher key (using plain password as source for key
          // expansion) - gives us well encrypted key
        indicates that the bytes of the password are not simply the bytes of the AES key!

        Edit: Solved! For 128 bit encryption, the bytes of the key to be used in the CTR encryption are the encryption of the password using AES/ECB/NoPadding using the password as key! I haven't done the analysis for 192 and 256 bits - I leave that as an exercise for anyone that needs it!

        The Nonce used by Java is the 8 byte nonce extracted from the first 8 bytes of the Javascript ciphertext padded with zeros to 16 bytes.

        Edited by: sabre150 on Jun 9, 2010 12:34 PM
        • 16. Re: AES CTR Decryption
          843811
          hi sabre could you give me some example code of your decryption routine.
          i not fully understand your last post.

          thx a lot
          • 17. Re: AES CTR Decryption
            843811
            mroth wrote:
            hi sabre could you give me some example code of your decryption routine.
            i not fully understand your last post.
            Sorry but I enjoy solving the problem and will explain how the problem is solved but these days I will not provide code unless it is very general code. I don't get paid for the solution, you do. There are two parts to solving this.

            1) The Nonce used by Java is taken as the first 8 bytes (after base64 decoding) of the ciphertext expanded to 16 bytes by padding with zeros. This is just 1 line of code using Arrays.copyOf() twice.
            2) The key bytes to be used in CTR mode are the AES/ECB/NoPadding encryption of the password using itself as key - 3 lines of code.

            My Java code for the decryption is just 11 active lines.

            If you get stuck, post your code.

            P.S. I'm not keen on using a 16 character password as key here even though it is incestuously processed by itself before active use. Using PBKDF2 or PKCS12 would be the recommended approaches.
            • 18. Re: AES CTR Decryption
              843811
              hi sabre,

              my solution works NOW!!!!!

              absolutely amazing problem analysis of your side.

              thx a lot.

              great forum!
              • 19. Re: AES CTR Decryption
                843811
                mroth wrote:
                my solution works NOW!!!!!
                Well done.
                • 20. Re: AES CTR Decryption
                  843811
                  I think there is a fundamental flaw in the [Javascript AES CTR code|http://www.movable-type.co.uk/scripts/aes.html].

                  The encrypt() method in the AesCtr object assumes that both the cleartext and the key are strings that have to be converted to pseudo bytes using a crude (but probably effective) utf-8 converter.

                  In common with Java, Javascript strings hold utf-16 characters and also in common with Java the utf-16 chars do not need to represent valid UNICODE characters. Nothing enforces the content of a Java or Javascript string to be valid UNICODE. This means that when converting a string to utf-8 bytes one cannot guarantee that when converting back again one gets the same string content one started with.

                  AES keys are binary data. Cleartext may or may not be binary data but it is always safe to assume it is binary data. By converting to and from utf-8, the Javascript always assume that both are not binary and are valid UNICODE.

                  To handle this binary data in Javascript, one has to ignore one of the Java/Javascript tenets and place binary data in a String. Since it just maps one-one, in Java one can use iso-8859-1 encoding to convert an array of arbitrary bytes to characters. In Javascript one makes each character
                  the equivalent of a byte by only using characters in the range 0 to 0xff inclusive.

                  Using this approach, to fix the Javascript AES CTR one just removes all the code that performs utf-8 encoding or decoding and one makes sure that all characters have a value in the range 0 to 0xff. It then works as I expect it to work.

                  I now await the "Strings are not valid containers for binary data" responses.

                  Edited by: sabre150 on Jun 13, 2010 10:08 PM
                  • 21. Re: AES CTR Decryption
                    843811
                    I hit the 7,500 char limit on above response so

                    Crude Java test code (edit to use your favourite hex and base 64 encoders) is
                    import java.io.InputStreamReader;
                    import java.io.Reader;
                    import java.security.SecureRandom;
                    import java.util.Arrays;
                    import java.util.Random;
                    import javax.crypto.Cipher;
                    import javax.crypto.SecretKey;
                    import javax.crypto.spec.IvParameterSpec;
                    import javax.crypto.spec.SecretKeySpec;
                    import me.grm.library.codec.Base64Decoder;
                    import me.grm.library.codec.Base64Encoder;
                    import me.grm.library.codec.HexEncoder;
                    import org.mozilla.javascript.Context;
                    import org.mozilla.javascript.Script;
                    import org.mozilla.javascript.Scriptable;
                    import org.mozilla.javascript.ScriptableObject;
                    
                    public class CTRTest
                    {
                        static Base64Decoder base64Decoder = new Base64Decoder();
                        static Base64Encoder base64Encoder = new Base64Encoder();
                        static HexEncoder hexEncoder = new HexEncoder();
                        static int errorCount = 0;
                    
                        static void process(byte[] cleartext, byte[] keyBytes, boolean hexEncodeOutput) throws Exception
                        {
                            System.out.printf("Cleartext (Hex)  : %s\n", hexEncoder.encodeAsString(cleartext));
                            Context cx = Context.enter();
                            Scriptable scope = cx.initStandardObjects();
                            {
                                Reader reader = new InputStreamReader(CTRTest.class.getResourceAsStream("/aes.js"));
                                Script script = cx.compileReader(reader, "rijndael", 0, null);
                                script.exec(cx, scope);
                                reader.close();
                            }
                            // Encrypte JS, decrypt Java
                            if (true)
                            {
                                ScriptableObject AesCtr = (ScriptableObject) scope.get("AesCtr", scope);
                                Object[] encryptArgs =
                                {
                                    new String(cleartext, "iso-8859-1"), new String(keyBytes, "iso-8859-1"), 128
                                };
                                String base64OfCiphertext = (String) ScriptableObject.callMethod(cx, AesCtr, "encrypt", encryptArgs);
                                byte[] ciphertextBytes = base64Decoder.decode(base64OfCiphertext);
                                System.out.printf("Encrypted using JS   : %s\n", base64OfCiphertext);
                    
                                Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
                                SecretKey secretKey = generateSecretKey(keyBytes);
                                byte[] nonceBytes = Arrays.copyOf(Arrays.copyOf(ciphertextBytes, 8), 16);
                                IvParameterSpec nonce = new IvParameterSpec(nonceBytes);
                                cipher.init(Cipher.ENCRYPT_MODE, secretKey, nonce);
                    
                                byte[] recoveredCleartext = cipher.doFinal(ciphertextBytes, 8, ciphertextBytes.length - 8);
                                if (hexEncodeOutput)
                                    System.out.printf("Decrypted using Java : %s\n", hexEncoder.encodeAsString(recoveredCleartext));
                                else
                                    System.out.printf("Decrypted using Java : %s\n", new String(recoveredCleartext));
                                if (!Arrays.equals(cleartext, recoveredCleartext))
                                    errorCount++;
                            }
                            // Encrypt Java, decrypt JS
                            if (true)
                            {
                                SecretKey secretKey = generateSecretKey(keyBytes);
                    
                                SecureRandom random = new SecureRandom();
                                byte[] nonceBytes = new byte[8];
                                random.nextBytes(nonceBytes);
                                IvParameterSpec nonce = new IvParameterSpec(Arrays.copyOf(nonceBytes, 16));
                                Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");
                                cipher.init(Cipher.ENCRYPT_MODE, secretKey, nonce);
                                byte[] ciphertextWithoutNonce = cipher.doFinal(cleartext);
                                byte[] ciphertext = Arrays.copyOf(nonceBytes, nonceBytes.length + ciphertextWithoutNonce.length);
                                for (int i = 0; i < ciphertextWithoutNonce.length; i++)
                                    ciphertext[i + 8] = ciphertextWithoutNonce;
                    String base64OfCiphertext = base64Encoder.encodeAsString(ciphertext);
                    System.out.printf("Encrypted using Java : %s\n", base64OfCiphertext);

                    ScriptableObject AesCtr = (ScriptableObject) scope.get("AesCtr", scope);
                    Object[] decryptArgs =
                    {
                    base64OfCiphertext, new String(keyBytes, "iso-8859-1"), 128
                    };
                    String recoveredCleartextString = (String) ScriptableObject.callMethod(cx, AesCtr, "decrypt", decryptArgs);
                    byte[] recoveredCleartext = recoveredCleartextString.getBytes("iso-8859-1");
                    if (hexEncodeOutput)
                    System.out.printf("Decrypted using JS : %s\n", hexEncoder.encodeAsString(recoveredCleartext));
                    else
                    System.out.printf("Decrypted using JS : %s\n", recoveredCleartext);
                    if (!Arrays.equals(cleartext, recoveredCleartext))
                    errorCount++;
                    }
                    }

                    private static SecretKey generateSecretKey(byte[] keyBytes) throws Exception
                    {
                    // var key = Aes.Cipher(pwBytes, Aes.KeyExpansion(pwBytes)); // gives us 16-byte key
                    // key = key.concat(key.slice(0, nBytes-16)); // expand key to 16/24/32 bytes long
                    SecretKey secretKey = new SecretKeySpec(keyBytes, "AES");
                    Cipher cipher = Cipher.getInstance("AES/ECB/NoPadding");
                    cipher.init(Cipher.ENCRYPT_MODE, secretKey);
                    keyBytes = cipher.doFinal(keyBytes);
                    return new SecretKeySpec(keyBytes, "AES");
                    }

                    static Random random = new Random();

                    public static void main(String[] args) throws Exception
                    {
                    String keyString = "0123456789ABCDEF";
                    String defautlCleartestString = "The quick brown fox jumps over the lazy dog.";
                    process(defautlCleartestString.getBytes("utf-8"), keyString.getBytes("iso-8859-1"), false);
                    process(defautlCleartestString.getBytes("utf-8"), keyString.getBytes("iso-8859-1"), true);

                    for (int i = 0; i < 1000; i++)
                    {
                    System.out.println("====== " + i + " =======");
                    byte[] keyBytes = new byte[16];
                    random.nextBytes(keyBytes);
                    byte[] cleartext = new byte[13 + random.nextInt(31)];
                    random.nextBytes(cleartext);
                    process(cleartext, keyBytes, true);
                    }

                    System.out.println("Error count = " + errorCount);
                    }
                    }
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            
                    • 22. Re: AES CTR Decryption
                      843811
                      Plus ...

                      To get this code to work one should edit the Javacript utf-8 methods to do nothing i.e.
                      /**
                       * Encode multi-byte Unicode string into utf-8 multiple single-byte characters
                       * (BMP / basic multilingual plane only)
                       *
                       * Chars in range U+0080 - U+07FF are encoded in 2 chars, U+0800 - U+FFFF in 3 chars
                       *
                       * @param {String} strUni Unicode string to be encoded as UTF-8
                       * @returns {String} encoded string
                       */
                      Utf8.encode = function(strUni) {
                          if (true)
                              return strUni;
                        // use regular expressions & String.replace callback function for better efficiency
                        // than procedural approaches
                        var strUtf = strUni.replace(
                            /[\u0080-\u07ff]/g,  // U+0080 - U+07FF => 2 bytes 110yyyyy, 10zzzzzz
                            function(c) {
                              var cc = c.charCodeAt(0);
                              return String.fromCharCode(0xc0 | cc>>6, 0x80 | cc&0x3f); }
                          );
                        strUtf = strUtf.replace(
                            /[\u0800-\uffff]/g,  // U+0800 - U+FFFF => 3 bytes 1110xxxx, 10yyyyyy, 10zzzzzz
                            function(c) {
                              var cc = c.charCodeAt(0);
                              return String.fromCharCode(0xe0 | cc>>12, 0x80 | cc>>6&0x3F, 0x80 | cc&0x3f); }
                          );
                        return strUtf;
                      }
                      
                      /**
                       * Decode utf-8 encoded string back into multi-byte Unicode characters
                       *
                       * @param {String} strUtf UTF-8 string to be decoded back to Unicode
                       * @returns {String} decoded string
                       */
                      Utf8.decode = function(strUtf) {
                          if (true)
                              return strUtf;
                       var strUni = strUtf.replace(
                            /[\u00c0-\u00df][\u0080-\u00bf]/g,                 // 2-byte chars
                            function(c) {  // (note parentheses for precence)
                              var cc = (c.charCodeAt(0)&0x1f)<<6 | c.charCodeAt(1)&0x3f;
                              return String.fromCharCode(cc); }
                          );
                        strUni = strUni.replace(
                            /[\u00e0-\u00ef][\u0080-\u00bf][\u0080-\u00bf]/g,  // 3-byte chars
                            function(c) {  // (note parentheses for precence)
                              var cc = ((c.charCodeAt(0)&0x0f)<<12) | ((c.charCodeAt(1)&0x3f)<<6) | ( c.charCodeAt(2)&0x3f);
                              return String.fromCharCode(cc); }
                          );
                        return strUni;
                      }
                      • 23. Re: AES CTR Decryption
                        843811
                        hi all,

                        we are currently using the pidcrypt javascript lib which includes the aes-ctr implementation from movable type scripts (posted before).

                        now our problem:

                        if we generate our aes key randomly (not 'AAAAAAAAAAAAAAAA'), e.g. some char code values go beyond 127 the decryption on java side doesnt deliver the expected plaintext.

                        does this problem relates to encoding or the fact that a byte on java side can only hold values from -128 127?

                        thx a lot
                        • 24. Re: AES CTR Decryption
                          843811
                          mroth wrote:
                          hi all,

                          we are currently using the pidcrypt javascript lib which includes the aes-ctr implementation from movable type scripts (posted before).

                          now our problem:

                          if we generate our aes key randomly (not 'AAAAAAAAAAAAAAAA'), e.g. some char code values go beyond 127 the decryption on java side doesnt deliver the expected plaintext.

                          does this problem relates to encoding or the fact that a byte on java side can only hold values from -128 127?
                          No!

                          Why did I bother with my last three responses to this thread? I give up!
                          • 25. Re: AES CTR Decryption
                            843811
                            hi sabre,

                            i now got i working with your explanations.

                            thx a lot for that.

                            but i got one question left:

                            is it obligatory to have a specific encoding on java side (except UTF-8) or is it negligible?

                            thx for your effort.
                            • 26. Re: AES CTR Decryption
                              843811
                              mroth wrote:

                              is it obligatory to have a specific encoding on java side (except UTF-8) or is it negligible?
                              What do you think? What do you think is safest? What do you think would happen if one side had default encoding of EBCIDIC and the other side had UTF-8?
                              • 27. Re: AES CTR Decryption
                                843811
                                i think it would not match correctly.

                                i will test some encodings on java side and decide the best for our purposes for the specific case.

                                thx a lot anyway
                                1 2 Previous Next