This discussion is archived
0 Replies Latest reply: Feb 22, 2010 9:01 AM by 843811 RSS

Invalid padding SSLHandshakeException while using SSL_RSA_WITH_3DES_EDE_CBC

843811 Newbie
Currently Being Moderated
Hello,

I'm using Tomcat server with PKCS11 provider which use NSS in FIPS mode.
The configured cipher is SSL_RSA_WITH_3DES_EDE_CBC_SHA since it is the only RSA FIPS cipher which is supported by IE7/8 (other FIPS ciphers use DSA type certificates which don't suite our application)

The enclosed exception is thrown during the handshake process.
Is this is a known bug in JSSE P11RSACipher?


Thanks,
albarak

*** ClientHello, TLSv1
RandomCookie: GMT: 1250003748 bytes = { 114, 210, 185, 55, 179, 83, 63, 27, 77, 196, 250, 192, 223, 219, 183, 69, 210, 223, 243, 38, 55, 205, 179, 221, 191, 17, 250, 108 }
Session ID: {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_RSA_EXPORT1024_WITH_RC4_56_SHA, SSL_RSA_EXPORT1024_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_RC2_CBC_40_MD5, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_DHE_DSS_EXPORT1024_WITH_DES_CBC_SHA]
Compression Methods: { 0 }
***
%% Created: [Session-33, SSL_RSA_WITH_3DES_EDE_CBC_SHA]
*** ServerHello, TLSv1
RandomCookie: GMT: 1250061902 bytes = { 206, 225, 88, 30, 81, 139, 132, 138, 111, 122, 178, 88, 34, 34, 53, 189, 31, 239, 83, 164, 131, 174, 34, 145, 15, 64, 235, 152 }
Session ID: {75, 130, 110, 78, 98, 92, 55, 164, 113, 98, 210, 45, 226, 10, 180, 24, 104, 54, 34, 239, 191, 69, 148, 218, 221, 150, 249, 99, 56, 249, 66, 185}
Cipher Suite: SSL_RSA_WITH_3DES_EDE_CBC_SHA
Compression Method: 0
***
Cipher suite: SSL_RSA_WITH_3DES_EDE_CBC_SHA
*** Certificate chain
chain [0] = [
[
Version: V3
Subject: CN=cd-acs5-13-161, OU=Unknown, O=Unknown
Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

Key: Sun RSA public key, 1024 bits
modulus: 158427005298490965271477092915445128357797319827240679543144590660409928120989190083655453887334930977101196338337282315517236067568427591704344852062196532686412089379088564076119232493108256067719066736084066297991638269770994069594668587426501049144412749797268827642032993786854761039362591476537118302201
public exponent: 65537
Validity: [From: Mon Feb 22 11:13:24 UTC 2010,
               To: Sat May 22 11:13:24 UTC 2010]
Issuer: CN=cd-acs5-13-161, OU=Unknown, O=Unknown
SerialNumber: [    9005d0b1]

]
Algorithm: [SHA1withRSA]
Signature:
0000: CF 90 76 D1 98 6E A1 3E 29 41 A4 9B 74 1F CF F5 ..v..n.>)A..t...
0010: 93 89 1A 24 F8 E3 FC 6D 95 44 AD 57 E0 3C D7 8D ...$...m.D.W.<..
0020: 03 3B 51 CE 4D F5 05 2F C3 8E 11 15 09 1A 61 6F .;Q.M../......ao
0030: A3 52 BE 84 AE 0D C4 58 C3 4B 89 C6 76 EE C0 63 .R.....X.K..v..c
0040: F2 F7 7A A2 93 27 9F 2E 6B 32 72 53 F5 5F 94 88 ..z..'..k2rS._..
0050: 47 65 5B 62 75 E1 9E 4E 26 3B 3B EA 58 E8 E2 23 Ge[bu..N&;;.X..#
0060: AA B8 EF A8 3D F0 1D B1 28 52 D1 CA 61 59 0D 47 ....=...(R..aY.G
0070: 37 90 60 CA B2 17 1B C7 09 83 35 06 43 4E 6A F5 7.`.......5.CNj.

]
***
*** ServerHelloDone
http-443-1, WRITE: TLSv1 Handshake, length = 590
http-443-1, READ: TLSv1 Handshake, length = 134
Error decrypting premaster secret:
java.security.InvalidKeyException: wrap() failed
at sun.security.pkcs11.P11RSACipher.engineUnwrap(P11RSACipher.java:421)
at javax.crypto.Cipher.unwrap(DashoA13*..)
at com.sun.net.ssl.internal.ssl.RSAClientKeyExchange.<init>(Unknown Source)
at com.sun.net.ssl.internal.ssl.ServerHandshaker.processMessage(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.startHandshake(Unknown Source)
at org.apache.tomcat.util.net.jsse.JSSESocketFactory.handshake(JSSESocketFactory.java:156)
at org.apache.tomcat.util.net.JIoEndpoint.setSocketOptions(JIoEndpoint.java:632)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
at java.lang.Thread.run(Unknown Source)
Caused by: sun.security.pkcs11.wrapper.PKCS11Exception: CKR_ATTRIBUTE_VALUE_INVALID
at sun.security.pkcs11.wrapper.PKCS11.C_UnwrapKey(Native Method)
at sun.security.pkcs11.P11RSACipher.engineUnwrap(P11RSACipher.java:417)
... 13 more
Generating random secret
*** ClientKeyExchange, RSA PreMasterSecret, TLSv1
SESSION KEYGEN:
PreMaster Secret:
0000: 03 01 83 82 9B DC 0D DE C9 4C 27 B6 15 D5 CF 87 .........L'.....
0010: E7 24 79 82 44 C9 44 98 44 1E 02 95 0E B8 59 3F .$y.D.D.D.....Y?
0020: 19 3B CC 13 AD 6D 5D 1C E3 1C EE 46 F5 F9 3B 08 .;...m]....F..;.
CONNECTION KEYGEN:
Client Nonce:
0000: 4B 82 8B 24 72 D2 B9 37 B3 53 3F 1B 4D C4 FA C0 K..$r..7.S?.M...
0010: DF DB B7 45 D2 DF F3 26 37 CD B3 DD BF 11 FA 6C ...E...&7......l
Server Nonce:
0000: 4B 82 6E 4E CE E1 58 1E 51 8B 84 8A 6F 7A B2 58 K.nN..X.Q...oz.X
0010: 22 22 35 BD 1F EF 53 A4 83 AE 22 91 0F 40 EB 98 ""5...S..."..@..
Master Secret:
0000: A0 A3 20 A5 B5 89 41 5C CC 51 2B 39 1D DB AF 25 .. ...A\.Q+9...%
0010: F1 09 83 C8 08 3A C9 65 D2 D8 41 AA 8A 8A E2 57 .....:.e..A....W
0020: CA 46 28 00 AA 89 3D 4C 47 AB 5A 2E 98 37 72 B9 .F(...=LG.Z..7r.
Client MAC write Secret:
0000: DF 78 B0 C0 EA 0C 85 C5 30 7C 5A DA ED 96 31 45 .x......0.Z...1E
0010: 1A 15 35 64 ..5d
Server MAC write Secret:
0000: E5 89 CA F8 04 38 87 90 25 F5 13 90 F7 A0 8A 01 .....8..%.......
0010: 40 BB 6C AF @.l.
Client write key:
0000: 32 C2 2C 5A 8A 1B FF AD 67 2C D2 62 7F CF 00 D7 2.,Z....g,.b....
0010: D5 09 0E 21 FA 20 02 68 ...!. .h
Server write key:
0000: CE 48 6C BB 0A BC 50 6C 1E 5C 36 59 32 E1 0D 0C .Hl...Pl.\6Y2...
0010: E7 26 0B B6 21 C9 2C E5 .&..!.,.
Client write IV:
0000: 6D F4 87 05 C0 DE 63 14 m.....c.
Server write IV:
0000: A6 49 D9 CC 9C 34 BB B0 .I...4..
http-443-1, READ: TLSv1 Change Cipher Spec, length = 1
http-443-1, READ: TLSv1 Handshake, length = 40
http-443-1, SEND TLSv1 ALERT: fatal, description = handshake_failure
http-443-1, WRITE: TLSv1 Alert, length = 2
http-443-1, called closeSocket()
http-443-1, handling exception: javax.net.ssl.SSLHandshakeException: Invalid padding
http-443-1, called close()
http-443-1, called closeInternal(true)
http-443-2, setSoTimeout(60000) called
http-443-2, READ: SSLv3 Handshake, length = 65