[SC-191] Credential config parsing of PublicKeys is broken in AbstractBasicCredentialBeanDefinitionParser Created: 16/Apr/14 Updated: 20/Jan/16 Resolved: 20/Jan/16 Status: Project: Component/s: Affects Version/s: Fix Version/s: Closed Shibboleth Common - Java None 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4 Type: Reporter: Resolution: Remaining Estimate: Time Spent: Original Estimate: Bug Brent Putman Fixed 0 minutes 1.4.5 Priority: Assignee: Major Scott Cantor 1 hour, 30 minutes Not Specified Description In looking at at a question asked by Rod re: porting the parsers to v3, I noticed that the code that purports to pars credential public key for basic (non-X509) credentials is broken. The code (really the subclasses of this class) is inline element content or filesytem and passing as-is to the SecurityHelper decodePublicKey(byte[]) method. Th presumed to be Base64-encoded DER. The SecurityHelper method requires the raw DER encoded form. The parser(s) need to Base64-decode the data before passing it to the decodePublicKey method. Currently a BasicInline credential element errors out as follows: 16:14:44.729 - DEBUG [edu.internet2.middleware.shibboleth.common.config.security.AbstractBasicCredentialBeanDef - Parsing credential public key 16:14:44.730 - ERROR [edu.internet2.middleware.shibboleth.common.config.BaseService:188] not loaded for shibboleth.RelyingPartyConfigurationManager service, error creating compone cause of this error was: java.security.KeyException: Unsupported key type. Comments Comment by Brent Putman [ 16/Apr/14 ] As I noted to Rod, we've obviously gone years without anyone noticing this error, as everyone presumably uses X509Credentials and so there's no PublicKey element to parse. Comment by Rod Widdowson [ 17/Apr/14 ] Just for the record, the motivation for getting this fixed in V3 is not just to get it fixed, but also to allow us to extend the Signature metadata filter to allow us to specify the public key inline. I looked and couldn't see any documentation for this for V2, which is probably another topic for friday. Comment by Scott Cantor [ 04/Aug/14 ] I added base64-decoding to both the public and private key parsing functions in the hierarchy, and added a unit test for them. As far as I can tell, the SecurityHelper class's routine for private keys claims to handle PEM, but doesn't, or I wouldn't have to decode the private key also. I did try without, same error as the public key case. Comment by Scott Cantor [ 04/Aug/14 ] The private key code seems to handle PEM but only with the headers included. With the headers in place, the code in the unit test doesn't decode the private key that the V3 unit test for this same code does handle, but I don't want to change any behavior or create new bugs, so I just left the fix at handling a PublicKey element alone. Comment by Scott Cantor [ 01/Mar/15 ] Regression was introduced handling file system formats. Comment by Scott Cantor [ 01/Mar/15 ] Tom Poage noted a regression on this because I put the base64 decode in the wrong spot. It's decoding unconditionally no matter how the bytes are extracted, so if the key is in a file, the file has to be base64-encoded DER, not plain DER. It's all a little muddy what we meant to support here, and PEM has issues regardless because the underlying APIs don't seem to support what we thought they did, but we can at least support DER by moving the base64 decode into the "inline" case explicitly and not in the decoding method in the base class. Don't know if we'll have another patch or not, but noting this so that we can fix it if there is one. Comment by Brent Putman [ 20/Jan/16 ] Regression fixed in r1142. Moved the Base64 decoding into Inline- parser. Filesytem- behavior now reverts back to expecting DER-encoded file, as it originally did. Comment by Brent Putman [ 20/Jan/16 ] Just for the record, to untangle the madness: The public key support here is the oddball. In the underlying utility classes, like SecurityHelper and X509Util, for private keys and certs, notyet-commons-ssl classes are used and they support both raw DER and PEM (with headers). For the public keys, there is no not-yet-commons support, since they are cert-oriented, and we rolled our own decode using Java X509EncodedKeySpec and KeyFactory. It supports raw DER only. We can easily layer "naked" Base64-encoded DER (no PEM headers) on that, like now doing for the inline parser. But there is no support for public key PEM with headers. At the time this sort of made sense by itself since that's what we see and expect with XML KeyInfo structures - base64, no PEM headers. But side by side with the others, it's clearly not like the other kids. Had we actually tested this back before 2.0.0, we probably would have noticed the asymmetry in the config parser support and reconciled. Oh well. V3 is better and consistent via cryptacular. Comment by Brent Putman [ 20/Jan/16 ] Also, the private key parsing that was failing I think was probably due to the actual format and the header used. It had I think BEGIN RSA PRIVATE KEY. Changing to BEGIN PRIVATE KEY, it worked. (Or maybe it was the other way around, doesn't matter). I didn't really remember any of this, but apparently there's a big difference re those headers, in terms of what the base64 encoded data is assumed to represent (the ASN.1 structure): http://stackoverflow.com/questions/20065304/what-is-the-differences-between-begin-rsaprivate-key-and-begin-private-key Basically it's PKCS#1 vs PKCS#8. The one Scott pulled must have been the wrong type for the header used. I just generated a fresh keypair with openssl and that one worked, so checked that in and reenabled the inline public+private test. Generated at Tue Feb 09 13:40:31 EST 2016 using JIRA 7.0.10#70120sha1:37e3d7a6fc4d580639533e7f7c232c925e554a6a.