SSO fails in 3.1 SP3. Works in SP2.

[Former Member]

Hi, everybody.

I have experienced this problem in two separate installations of BOXI SP3.

Using the exact same SSO set up (using the document by Tim Ziemba, "Configuring Vintela SSO in Distributed Environments"), SSO works fine in SP2, but fails with this error in SP3:

HTTP Status 500 - com.wedgetail.idm.sso.ProtocolException: com.wedgetail.idm.spnego.server.SpnegoException:
com.dstc.security.util.asn1.Asn1Exception: Bad tag encountered: 78

--------------------------------------------------------------------------------

type Status report

message com.wedgetail.idm.sso.ProtocolException: com.wedgetail.idm.spnego.server.SpnegoException:
com.dstc.security.util.asn1.Asn1Exception: Bad tag encountered: 78

description The server encountered an internal error (com.wedgetail.idm.sso.ProtocolException:
com.wedgetail.idm.spnego.server.SpnegoException:
com.dstc.security.util.asn1.Asn1Exception: Bad tag encountered: 78) that prevented it from fulfilling this request.

The SP3 installations are new installations on win2008 servers and the Tomcat server is running on port 80 (on SP2 we are running win2003 and port 8080), but apart from this the SSO setup is identical. We use the same keytab file, krb5.ini and bscLogin.conf, with the one exception that I added default_tkt_enctypes = rc4-hmac and default_tgs_enctypes = rc4-hmac to the krb5.ini on the 2008 server.

Does anyone have any idea about what the cause of this error is?

Thanks!

[Former Member · Accepted solution]

Hi, Björne.

The reason it failed for me was a slight oversight when setting up server.xml on Tomcat.

I had increased the maxHttpHeaderSize for SSL instead of non-SSL, which is what we use for our setup.

After increasing maxHttpHeaderSize for non-SSL, everything works fine

Thanks for all your help, Tim!

[BasicTek]

There is a new white paper available for SP3 that is more 2008 friendly (search kerberos sso SP3), although your error is not documented and shouldn't occur with the existing white paper. I don't know what would cause it without further investigation on a case. It's possible you may run into the same issue with the new docs as well but I'd give it a shot as several things have changed.

Regards,

Tim

[Former Member]

Thanks, Tim!

I'll start from scratch following the new guide

And for anyone trying to find this documentation for yourselves:

Open SAP Note 1483762 from the SAP Support Portal.

[Former Member]

Hi Jan Terje

We have recently upgraded our environment to XI3.1 SP3, and encounters the same problem as you regarding SSO. It seems like SP3 needs to be configured differently!!

Hope that you have solved your issue, and can share your solution??

Regards

Björn

[BasicTek]

SSO does not need to be configured differently, the new libraries are compatible with the old configuration we have set this up in house and on several customers with no issues to date. I'm not sure what's going wrong and throwing that error but a support incident with boj-bip-aut should be created so the issue can be investigated. If you wish a new white paper (referred to above) has been written for SP3 SSO which simplifies the configuration. Possibly you will have better luck with this doc.

Regards,

Tim