6 Replies Latest reply: Dec 21, 2011 2:05 AM by sabre150 RSS

    Cipher transformation that outputs non-padded plaintext final partial block

    809786
      I have written some Java code to read from and write to an existing data format that encrypts its data using a known symmetric key. It appears to be using the "AES/ECB/NoPadding" transformation / algorithm for all blocks, except for the final block, if the final block is a partial block. Any partial final block is just written out plaintext. When I use an "AES/ECB/NoPadding" Cipher in a CipherInputStream or a CipherOutputStream to read from or write to such a byte stream, respectively, everything works fine, except that any partial final block is omitted, as is expected from what I know of the behavior of "AES/ECB/NoPadding".

      (FYI, all of these questions relate to the Oracle JDK 7u2. For classes whose source is not included with this JDK, e.g., com.sun.crypto.provider.AESCipher, I looked at the source from the OpenJDK 7u2. I only looked at the default crypto provider included in the JDK, which I assume is the SunJCE, but I may be wrong.)

      1) Is there any other transformation that will properly decrypt / encrypt all the full blocks, but read / write, respectively, a partial final block as plaintext? (I assume not, but I also assume that it won't hurt to ask)

      2) Is there any way to obtain the buffered partial final block's data from Cipher? I can get the length of the leftover data from getOutputSize(0), but I haven't found a way to get the content. (I assume that this is done intentionally, to keep the Cipher-related classes as secure as possible)

      If I knew the length of the input, I could just determine the index of the end of the last full block, but I'd like this code to work with any arbitrary InputStream, and there's no way to know the length of an InputStream (without reading until you receive a -1, of course).

      I will probably wind up using a wrapping InputStream / OutputStream that always buffers the last partial block read from / written to it, respectively, until it's received a -1 from a read, or a flush call, respectively, but I wanted to avoid this if I could possibly use the crypto API more effectively.

      As an aside, I investigated exposing a partial final block's data by creating a wrapper class for one or more of the crypto classes. This appears difficult since many of these classes are final. Some other non-final crypto classes have final methods that use non-exposed private members, so those might be difficult, if not impossible, to wrap properly. The best plan that I devised is to create a CipherSpi subclass that wraps around a Cipher, and then use a Cipher subclass that wraps around both the wrapping CipherSpi subclass and the original Cipher. This appears convoluted, so I haven't yet thoroughly investigated its feasibility.

      I don't think that I can plug in a new mode or padding implementation to the existing SunJCE classes, since com.sun.crypto.provider.CipherCore seems to limit the potential implementations to those of which it is already aware.

      Please let me know if I've overlooked or misunderstood anything (this is the first time that I've used the crypto API).

      Thanks.
        • 1. Re: Cipher transformation that outputs non-padded plaintext final partial block
          EJP
          Any partial final block is just written out plaintext.
          In other words there is a bug in the source of this data. A major* security bug that compromises the entire system. Fix that, don't try to compensate for bugs.
          • 2. Re: Cipher transformation that outputs non-padded plaintext final partial block
            sabre150
            ECB mode is insecure in that it allows forging of ciphertext by splicing together block of ciphertext. One should use one of the feedback modes such as CBC. CBC is most secure using a random IV which must be passed with the ciphertext so there is an inflation of 16 bytes.

            AES is a block cipher which in it's basic form can only encrypt multiples of 1 block (16 bytes). One therefore needs to pad the cleartext to a multiple of 16 bytes in a reversible manner. There are many such padding schemes so you can take your pick. I use PKCS5 unless a client suggest a different one. PKCS5 in common with most reversible padding systems inflates the ciphertext by between 1 and 16 bytes.

            By using AES/CBC/PKCS5Padding to define your Cipher and a random IV sent with the ciphertext you will eliminate the problems you are having and one you didn't realise you had.
            • 3. Re: Cipher transformation that outputs non-padded plaintext final partial block
              809786
              It's an existing data format from a huge company. I can't change it, so I cannot switch to CBC or any other mode that is not compatible with ECB for all but a partial final block. I'm just trying to read and write from it. I don't care about security, since the data is on my computer, and it's not sensitive data (at all; it's data about music files). I will never transmit the data anywhere, and, even if someone got hold of the data, I wouldn't care in the slightest.

              You're definitely right about it being a major security bug, though, so thanks for the recommendation. I just want a nice programmatic interface to read and write the data. I understand why the crypto API would want to make it difficult to obtain partial final block info, but was just wondering if there is any easy way to obtain it anyway. I just wrote the cyclic buffer & associated input & output streams, so I'll probably just use those since I probably won't be able to extricate the partial final block from the crypto classes.

              Edited by: Ross on Dec 20, 2011 3:07 PM
              • 4. Re: Cipher transformation that outputs non-padded plaintext final partial block
                sabre150
                Ross wrote:
                It's an existing data format from a huge company. I can't change it, so I cannot switch to CBC or any other mode that is not compatible with ECB for all but a partial final block.
                If the existing data format is not binary then you have another problem. Ciphertext is binary and storing binary in a 'char' or 'varchar' column is likely to corrupt the ciphertext and in order to reversibly convert it to ASCII or one of the other character encodings then you are going to have to encode it as Base64 (approx 33% inflation), Hex (100% inflation) or ASCII85 (approx 25% inflation).
                I'm just trying to read and write from it. I don't care about security, since the data is on my computer, and it's not sensitive data (at all; it's data about music files).
                I don't understand this. If it's not sensitive then why are you considering encrypting it? Seems to me to be a pointless requirement!
                I will never transmit the data anywhere, and, even if someone got hold of the data, I wouldn't care in the slightest.
                Again, why encrypt it then?

                >
                You're definitely right about it being a major security bug, though, so thanks for the recommendation. I just want a nice programmatic interface to read and write the data. I understand why the crypto API would want to make it difficult to obtain partial final block info, but was just wondering if there is any easy way to obtain it anyway.
                You could use one of the stream ciphers or one of the techniques that turns a block cipher into a stream cipher resulting in one byte per byte. To make it secure you will still need to use a random IV (or something similar) so there will still be an inflation. You can normally get away with just 8 bytes for the IV. The output is still binary bytes and will need to be encoded (Base64, Hex or ASCII85) if you are going to try to store it in a 'char' or 'varchar' which will result in further inflation.
                I just wrote the cyclic buffer & associated input & output streams, so I'll probably just use those since I probably won't be able to extricate the partial final block from the crypto classes.
                I don't understand this.

                Note - pretty much every time one encrypts data one ends up with ciphertext longer than the cleartext. This is fundamental to encryption and is nothing particularly to do with Java. One can often use compression of the original cleartext prior to encryption but this does not guarantee to result in smaller ciphertext than cleartext.

                Note 1 - it seems to me that all you are trying to do is obfuscate the data so why not just use a simple insecure substitution cipher? This way you end up with ciphertext of the same length as the cleartext and using the same character set so nothing in the database structure has to change.
                • 5. Re: Cipher transformation that outputs non-padded plaintext final partial block
                  809786
                  It's an existing data format that is used by an existing application, over which I have no control. I have no desire to encrypt the data myself; I just want to be able to read it and fix errors in the underlying data, then write it back out so the existing application can use the corrected data. (the underlying data is binary, by the way)

                  The data must be encrypted in the slightly insecure format that I've described. I have no choice in the matter. Again, I have no desire to make this more secure, or even secure at all. I just want a programmatically clean way of achieving what I originally described.

                  Eveyone's comments about how to improve the security of the encryption are correct, but do not answer my original question. I think that there will probably be no cleaner way to get the leftover partial final block data than to use the buffering output stream that I wrote, so I'll just use that.

                  Thanks for the suggestions, though.
                  • 6. Re: Cipher transformation that outputs non-padded plaintext final partial block
                    sabre150
                    Sorry - having re-read the OP twice more it looks like I was missing the point; please ignore everything I have posted.