It was time to renew the Web Auth certificate on our WLC again and it never fails, there is usually something that causes this process to go awry. Having said that, each time I need to troubleshoot certificates my understanding grows and my arsenal of troubleshooting skills gets a little larger making me more effective at tackling the problem. I would like to show you a couple tools I used this year to figure out exactly what was going on.
This year I was contacted to assist in figuring out why a certificate was failing to upload into the WLC. I was sent two different .pem files and the private key file. What exactly are pem files? A PEM file is the output of a Binary64 encoded DER certificate based on the ASN.1 data structure. It stands for Privacy Enhanced Mail and is an old failed format created for the transmission of secure emails but the container format lives on and used for other purposes like certificates. It is one of several formats an X.509 certificate can be in. It is basically a Base64-encoded text file. Certificates in the PEM format can hold the entire chain of certificates in one file. The WLC requires certificates to be in the pem format. Since I received the files already in pem format I thought it would be an easy task by simply combining the three files into one. This can simply be done by opening Notepad and copying and pasting the contents of each file into one and naming it something like “wildcard_cert.pem”. Here is the order in which they are to be in:
This is all fine and dandy and should work just fine however when I tried to upload the PEM file into the WLC, I received the error: “Failed to install certificate”. The logs did not provide any deeper insight as well.
What to do? This is where the debug commands come in handy.
I went to the CLI of the WLC and issued the following two commands:
“debug transfer all enable“
“debug pm pki enable“
The first command was issued to troubleshoot any file transfers and the second command issued provides insight into the certificate problem. I then tried to upload the PEM certificate once again and the following output appeared:
The debug line stating “unable to get issuer certificate” was the hint I needed in order for me to second guess the entirety of the certificate chain. I should have checked this at the beginning but I just overlooked it. The entire chain has to be present! I quickly looked at an old certificate and realized I was missing the root certificate. I then went to Digicert.com and downloaded their public root certificate and pasted the contents into the pem file and boom!…the certificate could be successfully uploaded!
Another way to get to the core of the problem is to utilize OpenSSL and verify the certificate itself. If there is ever a question whether or not the entire chain is in the certificate, issue the following commands:
You can clearly see in the picture above the root cert is missing altogether. You can tell when you are at the end or the bottom of the certificate chain (root) when the subject of each certificate matches the issuer.
Here is a picture of the entire chain:
Let me know what certificate problems you have faced with the WLC and thanks for reading!
RF Blogger 