Troubleshooting 401 Unauthorized with x509 (OpenSSL or Self Signed Certificate)

[nageshcaparthy]

I am writing this blog to explain some of the issues that we had during the X509 setup self-signed by OpenSSL. Many a times we have documents from SAP to configure them but we still end up with few issues, in my case I had few issues such as

• - 401 unauthorized (figure 2) while testing the X509 certificate with oData URL on browser without SMP
• - Unable to ping backend system from SMP
• - Unable to test the certificate from REST Client, error such as
  1. Certificate is not getting called
  2. No response on REST Client
  3. 500 Internal Error

Please refer to this Guide as this blog is in continuation to fix the issue in case you have errors.

http://scn.sap.com/docs/DOC-65095

  http://wiki.scn.sap.com/wiki/download/attachments/425200590/How%20to%20configure%20mutual%20authenti...

401 Unauthorized Error

In our case, we had configured pretty much as described in the document, but we still ended up with 401 Authorization Error. It was difficult for us to understand fix the issue, but once we got the trace results it was a quick fix in 5 mins.

The error screen is as below:

Fig 1

Fig 2

In order to fix this issue, we had to run few trace analysis to check where the issue was. We followed the procedure below:

1. Go to SE38
2. Run the Report SEC_TRACE_ANALYZER
3. Click on Reset Trace Files – This will help to clear the old trace
4. Select the ICF Service according to your oData URL as show in the picture below fig 3
5. Select Logon Trace(got HTTP 401)
6. Change Level to 2
7. Select Record and Set ICMAN Trace Level

    8. Now In parallel open the service is the browser but do not select the certificate

Fig 3

    9. Go back to SAP Screen and click on Activate User Trace

  10. Now select the Certificate in the browser and click on Ok

Once you get the error 401 Unauthorized as Fig 2, click on Show User Trace

Fig 4

Click on Enter and expand the trace results

Fig 5

In our case we found the issue with Certificate External ID Mapping.

Fig 6

In my case what we had missed is the SP value. The certificate subject showed us only S=Telangana (as shown in fig 7) and hence we had an authorization issue.

Fig 7

The tracing report helped us to fix the issue quickly in no delay. Hope this blog helps to solve your issues as well. Looking for your feedback and questions if any.

Unable to ping backend system from SMP

At this stage, it is very important to setup a Trust between SMP and Gateway System. In case you are unable to ping the Backend system using SSOTECHNICAL, you may have issue in 3 places.

1. Certificates are installed in Gateway correctly
2. External User ID Mapping may have issues with the values
3. Certificates in the SMP Keystore may not have been refreshed on SMP Server

You may have to recheck everything according to the guide and also validate the RootCA is installed correctly in GW and SMP Server and setup the ExternalID Mapping in GW. Specifically with SMP after installing the RootCA and SSOTECHNICAL Certificate you have to restart the system. Incase you are still unable to ping the backend system, please use “go.bat –clean” from command prompt to refresh the certificates form keystore.

You may have to navigate to Drive:\SAP\MobilePlatform3\Server in cmd prompt to run this command. Ensure the server is stopped during this activity.

Alternatively you may also debug the JVM using props.ini.  I am not going to discuss more about JVM Debugging as this was not necessary in my scenario.

Once the clean is done and server is started, you should be able to ping. If not, please continue to check the certificates and other settings.

Screens from SMP Keystore and App:

Fig 8

Fig 9

Fig 10

Fig 11

Fig 12

Testing from Chrome Browser:

I am using Advanced REST Client for testing the X509 Certificates, here I have installed the OpenSSL Root Certificate in “Trusted Root Certificate Authorities” and SSO Demo Certificate into “Personal”. The following screen illustrates on how to register to an SMP Server:

Fig 13

Click on Send and the browser will ask for X509 Certificates installed, select ssodemo and click on OK:

Fig 14

Application was successfully registered with X509 Certificates:

Fig 15

In case you are running on SMP lower version such as SP06 or Java 7, you may see an error “Server has a weak ephemeral Diffie-Hellman public key”. Please refer to SAP Note 2217055 to fix the issue.

If you see any 500 internal error during registration or if the certificates is not getting picked up, if there are any certificate errors which may cause errors, you may also do the testing by disabling the certificate errors.  Go to command prompt and the run command as shown below:

Chrome --ignore-certificate-errors

Fig 16

Testing from Kapsel Mobile Apps on Android Device:

To confirm the solution we wanted to test it, so we opted for Kapsel Logon AuthProxy and we started to integrate it. Here are the screen shots of the working application:

Application deployed on to Android Device:

Fig 17

Registrations page

Fig 18

Certificate popup during the logon procedure.

Fig 19

Successful Registration Alert.

Fig 20

Get query function and the data from SAP Backend System.

Fig 21

I hope this blog will be useful and looking for your feedback and comments.

Regards,

Nagesh