Disable URL-Mangling in BSP Runtime

[Former Member]

Hi Experts,

we decided to provide some web applications für mobile devices. Currently (and maybe never) WD ABAP does not support mobile devices. Therefore we are looking for alternatives.

There are a lot of threads in the net where other people discussed the same topic and several decided to use BSP. We wanted to use BSP and jQueryMobile but we noticed that there are some problems because of URL-Mangling / rewriting.

Do you have any idea if it is possible to disable URL-Mangling without modification? We already thought about an own request handler in TA SICF but in this case we also need an own CL_BSP_RUNTIME (to disable the coding in method on_check_rewrite). However this is difficult because some standard BSP runtime classes (RUNTIME, NAVIGATION...) are marked as final, so it isn't possible to inherit from them and we don't want to copy / redevelop all relevant classes.

Furthermore there are other disadvantages like:

- many redirects (problematical with low-bandwidth connections)

- client cookie handling is more difficult

Do you have any idea?

Kind regards

Danny

[danilo_schembri]

Hi Danny,

It's not a clean solution, but you can try using 4 equals in URL instead of URL mangling code:

http://sapserver.company.com:1080/sap(====)/bc/bsp/zapp/

In this way, URL mangling is disabled, but you have to keep 4 equals in URL, ever.

Could this help you?

Danilo

GPG key: 56828871

[rdiger_plantiko2]

OK, this works since it is a programmed exception in CL_BSP_RUNTIME->ON_CHECK_REWRITE (I first thought you are relying on a non-handled error in the base64 decodiing process, since "====" is no valid base64 string - which would be a very unsafe way).

* special case 2: if "(====)" is in path, don't redirect!
   l_help = c_request->get_header_field( '~path' ).          "#EC NOTEXT
   if l_help cs '(====)'.
     return.  " no rewrite in case of (====) in path !!!
   endif.

[rdiger_plantiko2]

In fact, if you use the method CL_BSP_RUNTIME=>CONSTRUCT_BSP_URL to construct your request URL, and you set the method parameter IN_SUPPRESS_REWRITE = 1, exactly this trick will be used: The URL gets the mangling string '(====)'.

[JaySchwendemann]

You sir, are my personal hero 🙂 We were trying to access an BSP which provides a binary for download via cloud connector and CPI and ultimately from C4C. We struggled big times on the 302 redirect within cloud connector.

Thanks again. Cheers

Jens

[Former Member]

Hello!

I've done quite a lot of BSP development together with and without javascript frameworks like jQuery. It's possible to create a scenario without URL mangling if you set the system architecture in a specific way. I could speak a lot about this setup as it works really well! Here is a quick guide to get started:

Step 1:

The BSP runtime isn't really used at all. Create a BSP application but only store MIME objects in it. The MIME objects isn't URL mangled! The drawback is of course you can't do server side ABAP in your HTML documents. On the positive side, any Web Developer can create the HTML and javascript without ABAP programming skills. SAP functions as a static web server with all the benefits of the ICM cache and transport system.

Step 2:

Add an external alias in SICF for the BSP application. Like pointing "/myapp" to "/sap/bsp/sap/zmyapp_v001". The external alias is configured with a service account (without any authorizations) with authentication set to "Required with Logon Data". All fields should be specified (Client, User and Language). Now, any anonymous user can request the HTML files without interfering with logon data.

You don't need an external alias, but they come in handy to have nice-looking URLs; the possibility to change their target when you have multiple versions at the same time; and you can create multiple alias for the same application with different logon data. Like "/myapp/english" if you want. Or "/myapp/client800". Or "/myapp/latest", "/myapp/version1" etc.

Step 3:

Create a custom HTTP handler for your business data and assign it to a node in SICF. For example "/customers/orders/get" or whatever you find appropriate for your app. Since you now have your own handler, the BSP runtime isn't involved and no URL mangling exists.

The handler returns data in a suitable format for Web Developers (read: JSON).

Protect this node with security that matches the flexibility required from the frontend. I usually set another service account for this service and handle authorization myself; returning HTTP status codes like 403, 200 to the Web Developer (but not 401 as it's quite difficult to handle in AJAX scenarios). In it's most simple form you can provide username and password as form fields and perform a user switch in the handler for every request, effectively changing SY-UNAME in the handler ensuring that the correct user context is used.

Best regards,

Mikael Gurenius

SAP Solution Architect (Mobility)

PS: MIME objects are of course served with URL Mangling if referenced relative to the BSP application. You can avoid this (for example for the Apple icons) by always linking your MIME objects from the root. Get the application path using code or type manually. Like

  <link rel="apple-touch-icon-precomposed"

sizes="57x57"

href="<%= lv_path %>/icon_57x57.png" /> where lv_path is "/myapp/images" or "/sap/bc/bsp/myapp/images". If you do this, you can have URL Mangling (and do ABAP development) in your BSP pages but still have "clean" URLs for MIME objects.

[Former Member]

Yes, this is one way - it's not BSP any more - quite attractive for modern HTML5/JSON stuff. You do have the challenge of generating your own HTML and have to come up with a templating system (e.g. XSLT) which does what the BSP <%=...%> syntax does.

However I am interested in how you do a "user switch". We investigated this once and it seems you can only switch from S (service) users to A (dialog) users but not to C (communication) with SUSR_INTERNET_USERSWITCH. Did you ever manage to switch to C?

[rdiger_plantiko2]

Hi Mikael,

thanks for the detailled guide to alternatives (for those people who absolutely can't live with URL mangling).

The first option, however - developing an application with statical MIME's alone, is not feasible for most projects. In the majority of web application projects, you will need the server-side logic of the SAP system: for applications which don't, there would be no reason why to use an SAP system at all (and not, say, simply a Linux server with XAMPP).

The other two options - external alias and custom request handler - are viable and useful, as I can confirm. After all, BSP is just a request handler itself, so why not write your own handler if this one doesn't fit your needs! Request handlers come particularly handy if they only send data to the client, not HTML views, as I blogged here.

You could add the fourth option, mentioned in this blog by another author: Just mangling the fixed string '(====)' will prevent the BSP runtime from doing the unwanted redirect as well.

The standard option, of course, would be: Don't bother about the mangling, and just give the BSP runtime what it wants. The redirect can be avoided by passing the appropriate client and langauge from the beginning in the request URL. This is the best option in my eyes.

It should be noted that URL mangling is not only used in  the BSP runtime. It is part of the whole Internet Communication Framework. Any HTTP request served by the SAP system can work with the mangling syntax - it's not restricted to BSP. When you use a custom request handler, you can mangle the URL with additional information like client, language, session-id and more. The information will be evaluated and used by the ICF.

The difference is only that there is no automatic redirect if the mangling information is considered incomplete. It's only this redirect which is peculiar to the BSP runtime.

Regards,

Rüdiger

[Former Member]

I think he meant those as steps, not options. You would use the BSP/Mime repository in combination with HTTP handlers.

Following in that idea you could actually use something like NGINX to serving the static files from any file server and combine SAP BSP or HTTP handlers in one domain. You could even use NGINX to rewrite the mangled URLs into something pretty - or hide completely since the client/language is part of the session and not needed in stateful requests.

[rdiger_plantiko2]

If he meant them as steps, then I herewith propose to use them as alternative options. They make sense isolated from each other.

You can use many cool things, sure. But when using options like XAMPP or NGINX, the question arises: Why not kick SAP out of the landscape at all?

Regards,

Rüdiger

[Former Member]

If he meant them as steps, then I herewith propose to use them as alternative options. They make sense isolated from each other.

A typical SAP web application needs data from SAP and static resources (css, images, js) so you need both. They are isolated in a sense but the application needs both at the same time.

You can use many cool things, sure. But when using options like XAMPP or NGINX, the question arises: Why not kick SAP out of the landscape at all?

Because you need SAP data and BSP/HTTP Handlers are the most tightly integrated - simpler than reading the data from PHP/ASP via some API. Remember this post is about avoiding mangled URLs, not how to live with them 😉

[Former Member]

Challenge or not depends on your team members. I've had many projects where (non-SAP) frontend web developers create beautiful web applications in pure HTML/Javascript (HTML5 and AJAX). They don't do any server-side programming at all. This approach is so common now it's not a challenge anymore.

For the user switch, you are correct. I've never had a case where I would need to switch to a C (communication) user. Do you? You configure the service in SICF to have a S (service) user. If my end users are standard SAP (dialog) users, they can be switched to. If they are not, they probably should switch to some common/shared dialog user anyway or continue anonymously.

[Former Member]

Rüdiger Plantiko wrote:
The first option, however - developing an application with statical MIME's alone, is not feasible for most projects. In the majority of web application projects, you will need the server-side logic of the SAP system: for applications which don't, there would be no reason why to use an SAP system at all (and not, say, simply a Linux server with XAMPP).

Of course, you need server-side logic. Written by an ABAP developer - not a Web Developer IMHO. It makes sense for me to put this logic in standard ABAP Classes or Function Modules. The connection to HTTP can then be done in a (generic) custom-build HTTP handler to isolate them from HTTP completely.

This way, I can staff my projects with dedicated, skilled Web Developers specialized in front-end development. For the backend logic, I don't want to rely on finding an ABAP developer that truly understands Web. They are hard to come by. But when I do, the BSP environment is an excellent choice.

A 3rd party web server? Sure, that's one way. But there is also a strong argument for having all your code in SAP. Often, I let my web developers develop wherever they want, often involving Github. But once they have something to release, it goes straight into SE80 as MIME objects.

[Former Member]

Marc C. wrote:
I think he meant those as steps, not options. You would use the BSP/Mime repository in combination with HTTP handlers.
Following in that idea you could actually use something like NGINX to serving the static files from any file server and combine SAP BSP or HTTP handlers in one domain. You could even use NGINX to rewrite the mangled URLs into something pretty - or hide completely since the client/language is part of the session and not needed in stateful requests.

Excatly. As steps. You combine MIME objects with a custom HTTP handler for the backend logic. Voila - you have your application!

The SAP system does an excellent job serving static files. The ICM cache is powerful and works really well. If you want to rewrite, there is no need for an external product. The modification rules in Web Dispatcher will do it: http://help.sap.com/saphelp_nw73/helpdata/en/48/9266ffaa6b17cee10000000a421937/content.htm

[rdiger_plantiko2]

Mikael,

OK - I got it. 🙂 You really meant your three steps as parts to be combined, the SAP logic being mixed in using HTTP request handlers.

For the backend logic, I don't want to rely on finding an ABAP developer that truly understands Web. They are hard to come by. But when I do, the BSP environment is an excellent choice.

Maybe we are lucky: We have precisely this kind of personnel! Therefore, using "stateless" BSP (i.e.: no session reserved in the memory, but small amounts of session data managed in table SSCOOKIE), with MVC and tag libraries is our standard choice.

Sometimes, when not the complete session context is needed and the task is isolated (for example value helps), we use request handlers called from the BSP.

The actual choice will depend on how separated the "web skills" and "SAP/ABAP skills" are in the heads of the same or of different people.

Regards,

Rüdiger

[Former Member]

Hi Experts,

thanks for your answers and please excuse my very late reply (worked on other projects the last months). If anybody knows a solution in the meantime, please let me know...

Today I recognized that it even is not possible to use the tag apple-touch-icon if the URL is mangled. I am not sure if this problem only appears at android smartphones.

By adding a bookmark you only see the bookmark text but there is no icon. I copied the page to another web server to call it without URL mangling and everything works fine.

I also tried to use (====) - without success...

Regards

Danny

[Former Member]

In my tests (using Fiddler to manipulate the requests) it is unproblematic to remove the (...==) insertion (which turns out to be the client and language encoded as base64). I am considering modifying CL_BSP_RUNTIME->ON_CHECK_REWRITE to remove this.

Does anyone know why SAP implemented this apparent nonsense or what the risk is of removing it?

[rdiger_plantiko2]

Does anyone know why SAP implemented this apparent nonsense or what the risk is of removing it?

Marc - be careful with the "nonsense" accusation!

If you omit the mangled part in the URL and skip the redirect that usually is sent in this case (for example by sending '(====)' ), the system will work with the system-wide default client and the system-wide default language.

This is normally not desired, for example fo web applications with multilangual support. In short, the SAP system needs more login data than user and password: Client and Language have to be provided, too. If you don't want to rely on the client cookies and don't want to pollute the URL with GET parameters, mangling the URL to force these data is a reasonable approach.

So what speaks against sending client and language as part of the URL in your case? If language and client are fixed in your case - even better: Just enrich the URL with this fixed mangling string.

Regards,

Rüdiger

[Former Member]

Hi Rüdiger,

I asked about "apparent nonsense" because there is no apparent sense. SAP has the query string parameters for client and language (e.g. ?sap-language=EN) so why force add them in encoded form to the path? In SICF the service has both client and language settings so it seems unnecessary except in special cases. In particular the language will be known when the user logs in and client can be part of the login form. For reasons why this encoded insertion is problematic in URLs see this thread.

Allow me to re-phrase the question: what is the disadvantage of removing this Zusatz in a single-client system?

Cheers

Marc

[rdiger_plantiko2]

Hi Marc,

sure, SICF services provide the fields for language and client. But that's not the solution. I am living in Switzerland. French-speaking users want to access service in French, Italian-speaking in Italian, German-speaking in German. So the language can't be set for all users - it has to be sent with the request instead. You have three options:

• cookies
• GET parameter
• put ("mangle") data into the URL path

All of these options are common, including URL mangling: How many URL have you seen with an .../EN/... inside the path?

Cookies could be switched off and are usually restricted to certain paths. Also, they don't distinguish between different tabs or windows of a single browser session which played a role for the mangling information "THEME" (see Thomas Jung/Brian McKellar's text below). Form fields get lost in navigation, when you provide links from your BSP to other URLS (like the MIME's of the MIME repository). I would find it extremely ugly to provide in each href that I offer the fixed suffix

../other_bsp/main.do?<%=ZCL_GET=>CLIENT_AND_LANGUAGE( )%>&further_parm1=val1

NO! The query part is owned by the application developer! (And if used, the query part should be as short and as expressive as possible.)

With the missing data mangled in the top node of the path, I can navigate through the whole tree below this node (e.g. using URL's relative to the current URL), without having to care about sending these data. I find this quite comfortable.

99% of the BSP applications I am producing are stateless. This means that a signon is performed with each dialog step. "Stateless" is a great feature of BSP, I don't want to miss it. (I store all the data that have to persist the single dialog step in a sever side session cookie - and this works great even for quite complex business applications).

With the mangling mechanism, the basic information of the current session are available in all requests that branch away from the current application / page.

For the records: Brian McKellar/Thomas Jung on URL mangling in "Advanced BSP programming", p.84. They argue with the "THEME" which is also mangled (although rarely used in practice). But the argument holds for other data as well: The navigation problem will be the same.

Why did SAP introduce this mangling? During Web AS 6.10 development, a theme concept was devised whereby specific objects in the MIME repository can be overwritten into different themes. For each BSP application, a cookie is set with the relevant information. However, this cookie is set exactly to matching the URL of the BSP application and is not sent by the browser when the MIME repository is accessed ( /sap/something_else/... )
The logic behind this concept is that you could have more than one active BSP application at the same time, and each application must convey its own configuration information over different stateless HTTP GET requests to the MIME repository. As the MIME repository is accessed, it is not possible to set different cookies in different windows of the browser locked onto the same path. The only reliable solution was to encode information in the URL.

Kind regards,

Rüdiger

[Former Member]

Hi Rüdiger,

I also live and work in Switzerland and develop apps for up to 15 languages so I know about the concerns of i18n. I've just never seen a framework which forces you to have such an ugly solution shoved into the URL. It reminds me of the SAP ITS session id in the URL - a novel idea until users start bookmarking...

I'll admit that a lot of the issue here is the ugliness of the URLs but there are technical barriers which Danny raised and also e.g. with having brackets in the URL (see Issue 4849 in jQuery Mobile). It's just wacky and unnecessary, /xx/ would even make more sense than what we see here but a general web framework shouldn't force developers to have something in their URL they do not want/need.

Even SAP ITS and Web Dynpro do not do this with language/client. Consider the 2 cases:

1) Custom Authentication: Application runs with a service user (set in SICF) and is either anonymous or offers some other authentication

2) SAP Authentication: The user must authenticate as a SAP (SU01) user.

In SAP ITS and Web Dynpro you can use ?sap-language=xx to switch the language and the similarly for the client. If nothing is set in the query string it is taken from the user in case 1 from SICF and in case 2 from the user defaults. This works great and gives you as a developer full power and flexibility. If newer technologies went this way I would argue SAP noticed that (...==) was a mistake.

If you are writing stateless HTTP endpoints then it doesn't matter because your URLs are probably invisible and called via AJAX etc. For users and some devices in modern apps these things matter.

I'll certainly grant that stateless apps (or apps with no cookies) need to have session information in the URL for you to be able to run them in parallel. HTTP is missing a window-based session/cookie strategy for sure!

Thanks for your feedback though!

Marc

[rdiger_plantiko2]

Marc,

thanks for your reply.

If you are writing stateless HTTP endpoints then it doesn't matter because your URLs are probably invisible and called via AJAX etc. For users and some devices in modern apps these things matter.

No, the URL's are visible in the browser's address bar.

As for your "modernity" argument: I personally hold the BSP framework much more usable for modern web applications than Web Dynpro. Simply because the framework is so loose that it leaves me free to do whatever I want on the HTML/CSS/JavaScript level.

You are right, the brackets are one of the points that I am forced to accept in this framework, so they are an exception from the general rule "BSP is freedom" (Brian McKellar)!

Form fields sap-client and sap-language are accepted by BSP as by any other SICF request handler, because they are handled on the SICF level not on BSP level. The only BSP-specific point is the forced redirect if the mangling term is missing. Until now, I had no problems accepting this.

For Web Dynpro, WebGUI and ITS, this would be no problem. The reason is that these frameworks force me to be "stateful" (and within this server-side state, they can of course keep whatever data they like). That's the price of these frameworks: Each browser session binds a complete session in the SAP system memory. I don't like this concept.

In your fiddle: Have you tested that navigation with relative URL to other locations served by the same SAP system works well too? If yes, then you may really dare to modify away the forced redirect in CL_BSP_RUNTIME. Let me know if it works!

Regards,

Rüdiger

[rdiger_plantiko2]

Usually it helps to provide the mangled part from the beginning. What speaks against it in your case?

If you send a request to http://sap-system.de/sap(adsd312==)/bc/bsp... instead of http://sap-system.de/sap/bc/bsp... the server will not send the redirect response, but will directly send the expected response.

The term inside the brackets usually only requires the base64 encoded values for client (c) and language (l).

If your desire goes to have nice, readable URL's on the client, you may consider putting a proxy before your SAP system (which isn't a bad idea for internet applications anyway).

Using an Apache server, you may use the following set of directives (which work fine in our case):

...

  ProxyPass /sap/ http://migzmcx2.migros.ch:1080/sap(bD1kZSZjPTU2NQ==)/

  ProxyPassReverse /sap/ /sap(bD1kZSZjPTU2NQ==)/

  ProxyPassReverseCookiePath /sap(bD1kZSZjPTU2NQ==)/ /sap/

...

[darren_bambrick2]

Hi Danny

it is not possible to disable mangling, however you can hide it.

Please search for "attempting to hide the url mangling" on the net and you will find some articles that use framesets.

Regards

Darren