From time to time the error message GEN_BRANCHOFFSET_LIMIT_REACHED pops up. It is a very complex problem right from the heart of the ABAP VM. Let us look quickly at why it happens and some ideas to work around the problem. No, there is no patch for this problem.

Background Information

Any (most?) computer languages have typical limitations that play a role in code generation. The ABAP VM is not exception in this case. One such an limitation is that one IF-statement can only jump 32KB of byte code. So effectively, the source code between the IF- and ENDIF-statements, when compiled, must not be more than 32KB of byte code.

And this plays a big role when the Layout of a BSP page is compiled. (The same rules apply for BSP views as well.) For each BSP page, an ABAP class is generated (in the temporary package $TMP). The name is very criptic, as it constructed from with a GUID (global unique identifier). For example: CL_O2D7QDB5488MACT6YI2S549GQVB.

In the generated class, for the complete layout, source code isgenerated into one method. The event handlers (OnCreate, OnInitialization, etc) are also placed into separate methods on the same class. In this way, each BSP page is translated into a self contained ABAP class, that is generated and instantiated in one step.

So what the problem effectively states, is that somewhere within a method, there is an IF-statement that has a body that exceeds the jump limitations of the ABAP VM. To understand the problem better, we have to look a little bit at how source code generation is done in BSP.

BSP Source Code Generation

Let us assume a simple BSP page with the following source code:

This program shows one technique how an IF-statement can appear within the generated source code. The complete source code (in slightly abstract format) will be:

]]>
<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb" prefix="htmlb"%>
<!code>  <htmlb:content design="design2003">
<!code>    <htmlb:page title = "Test Page">
<!code>      <htmlb:form>
<!code>        <htmlb:button text    = "myButton"
<!code>                      onClick = "HitMe!" />
<!code>      </htmlb:form>
<!code>    </htmlb:page>
<!code>  



This simplest of simple examples, already generate rather complex code. The reason for this is in the specification of the BSP elements themselves. To allow a flexible architecture, BSP elements can dynamically decide a number of aspects. First, let us look at the generated source for the example (extremely stripped!):



<!code>  METHOD _onlayout.
<!code>  * generated by BSP converter version: 200502181048
<!code>  * generated by BSP compiler version:  1.60
<!code>   
<!code>    DATA: %_elem_rc TYPE I.
<!code>   
<!code>  * <htmlb:content>
<!code>    DATA: %_bsp_elem_0 TYPE REF TO CL_HTMLB_CONTENT.
<!code>    CREATE OBJECT %_bsp_elem_0.
<!code>    %_bsp_elem_0->design = 'DESIGN2003'.
<!code>    %_elem_rc = %_bsp_elem_0->DO_AT_BEGINNING( ).
<!code>    IF %_elem_rc = IF_BSP_ELEMENT=>CO_ELEMENT_CONTINUE.
<!code>   
<!code>  *   <htmlb:page>
<!code>      DATA: %_bsp_elem_1 TYPE REF TO CL_HTMLB_PAGE.
<!code>      CREATE OBJECT %_bsp_elem_1.
<!code>      %_bsp_elem_1->title = 'Test Page'.
<!code>      %_elem_rc = %_bsp_elem_1->DO_AT_BEGINNING( ).
<!code>      IF %_elem_rc = IF_BSP_ELEMENT=>CO_ELEMENT_CONTINUE.
<!code>   
<!code>  *     <htmlb:form>
<!code>        DATA: %_bsp_elem_2 TYPE REF TO CL_HTMLB_FORM.
<!code>        CREATE OBJECT %_bsp_elem_2.
<!code>        %_elem_rc = %_bsp_elem_2->DO_AT_BEGINNING( ).
<!code>        IF %_elem_rc = IF_BSP_ELEMENT=>CO_ELEMENT_CONTINUE.
<!code>   
<!code>  *       <htmlb:button/>
<!code>          DATA: %_bsp_elem_3 TYPE REF TO CL_HTMLB_BUTTON.
<!code>          CREATE OBJECT %_bsp_elem_3.
<!code>          %_bsp_elem_3->text = 'myButton'.
<!code>          %_bsp_elem_3->onClick = 'HitMe!'.
<!code>          %_bsp_elem_3->DO_AT_BEGINNING( ).
<!code>          %_bsp_elem_3->DO_AT_END( ).
<!code>   
<!code>  *     </htmlb:form>
<!code>        ENDIF.
<!code>        %_bsp_elem_2->DO_AT_END( ).
<!code>   
<!code>  *   </htmlb:page>
<!code>      ENDIF.
<!code>      %_bsp_elem_1->DO_AT_END( ).
<!code>   
<!code>  * </htmlb:content>
<!code>    ENDIF.
<!code>    %_bsp_elem_0->DO_AT_END( ).
<!code>   
<!code>  ENDMETHOD.
<!code>   


The complexity of this source code comes from the fact that the BSP compiler does not know before hand whether a specific BSP element wishes to process its own body (inner BSP elements), or wish to just over the body. The BSP element can decide this during its DO_AT_BEGINNING( ) method. Therefore the BSP compiler generates the IF-statements to test for this condition.


The use of BSP elements is the second source of IF-statements on a BSP page.


Effectively the problem is that the complete layout of one BSP page is placed into one method in a class. And this method now has in its generated form an IF statement that is too large. Any solution must attempt to remove some code from the generated method (which means directly from the layout) and place it somewhere else. Below are three typical approaches that are recommended.
 

Solution Idea: Use Class Methods to Reduce Layout Code
 
 

Probably the simplest technique would be to strip code from the layout of the BSP page, and to place it into a method of class.


Very important: Long sequences of HTML on a page does not make much of a difference in the generated source of the method. The complete HTML sequence is replaced with out print_string statement to start at an offset and dump the string for a specific length.

For such work, one should look at large blocks of program code (<%...%> sequences), or for large blocks of print code (<%= .. %>). Each of these interrupts the HTML, which causes the print_string statement to be splitted into a large number of print_string statements. See the first code generation example above. Let us examine this example code again:

]]>
<!code>  <%@page language="abap"%>
<!code>  <%IF sy-uzeit < '1200'.%>
<!code>  Good morning <%= sy-uname%>,
<!code>  <%ELSE.%>
<!code>  Good afternoon <%= sy-uname%>,
<!code>  <%ENDIF.%>
<!code>  Welcome back to my great BSP application.
<!code>   


What we see is that the <%IF...%> sequence, and the <%=sy-uname%> causes many more print_string statements to be emitted, which contributes significantly to the generated code in the layout method.

One solution would be to place this critical code sequence in a separate method. Then the layout code would reduce to:

]]>
<!code>  <%@page language="abap"%>
<!code>  <%= cl_my_class=>greetings( name = sy-uname ) %>
<!code>  Welcome back to my great BSP application.
<!code>   


In the above coding, greeting is a method that has a returning parameter of type string. This string is then printed. The method itself would be:



<!code>  METHOD greetings.
<!code>    " name IMPORTING TYPE STRING
<!code>    " html RETURNING TYPE STRING
<!code>    IF sy-uzeit < '1200'.
<!code>      html = `Good morning`.
<!code>    ELSE.
<!code>      html = `Good afternoon`.
<!code>    ENDIF.
<!code>    CONCATENATE html ` ` name `,` INTO html.
<!code>  ENDMETHOD.
</pre>
 

Solution Idea: Use Smaller Views in MVC to Split Layout Code
 
 

The use of methods to contain rendering code works well with HTML code, but comes more complex once BSP extensions are used. Although it is possible process BSP elements inside methods, it is not easy to write and main. Recommended is to split large layouts into separate views. One typical example could be to place the content of each tab in a tabstrip onto a separate view.


Let us start with this example again, with the goal to place the
and its content onto a separate view:




<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb" prefix="htmlb"%>
<!code>  <htmlb:content design="design2003">
<!code>    <htmlb:page title = "Test Page">
<!code>      <htmlb:form>
<!code>        <htmlb:button text    = "myButton"
<!code>                      onClick = "HitMe!" />
<!code>      </htmlb:form>
<!code>    </htmlb:page>
<!code>  </htmlb:content>
<!code>   


As a first step we create a new view (called my_view.htm) with the displaced content (cut-and-paste from page to view):



<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb" prefix="htmlb"%>
<!code>      <htmlb:form>
<!code>        <htmlb:button text    = "myButton"
<!code>                      onClick = "HitMe!" />
<!code>      </htmlb:form>
<!code>   


On the page main page (or view!), place now a call to a controller that will embed the view. Keep in mind it is not possible to directly expand views in place. See also that we have now added the BSP extension "BSP".



<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb" prefix="htmlb"%>
<!code>  <%@extension name="bsp" prefix="bsp"%>
<!code>  <htmlb:content design="design2003">
<!code>    <htmlb:page title = "Test Page">
<!code>      <bsp:call url="my_controller.do" comp_id = "view">
<!code>        <bsp:parameter name="view" value="my_view.htm"/>
<!code>      </bsp:call>
<!code>    </htmlb:page>
<!code>  </htmlb:content>
<!code>   


As a last step, we create a new controller (my_controller.do). We add one attribute view TYPE string and implement the do_request method:



<!code>  METHOD do_request.
<!code>    DATA: page TYPE REF TO if_bsp_page.
<!code>    page = create_view( view_name = view ).
<!code>    call_view( page ).
<!code>  ENDMETHOD.
<!code>   


Passing parameters to different views can also easily be done via the controller.


Recommended Reading:


[MVC Documentation | http://help.sap.com/saphelp_nw04/helpdata/en/0f/ab3a3c9ca75402e10000000a114084/frameset.htm]

The BSP example application IT05 uses the above simple controller to embed any number of views.

 

Solution Idea: Use BSP Element Composition to Bundle BSP Elements
 
 
 
 

Solution Idea: Replace BSP Element Sequences with Light-Weight Sequences
 
 

Often BSP elements are just wrappers around simple HTML sequences. However, we have seen that such BSP elements can cause a large explosion in the generated code. The use of the BSP elements are practical, as the BSP compiler will help to valid that large sequences are correctly nested. However, in cases where the generated source code is to large, ir might help to replace a few of these sequences with native code.


Let us assume this typical form layout example:



<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb" prefix="htmlb"%>
<!code>  <htmlb:content design="design2003">
<!code>    <htmlb:page title = "Test Page">
<!code>      <htmlb:form>
<!code>        <htmlb:gridLayout rowSize="2" columnSize="2">
<!code>          <htmlb:gridLayoutCell rowIndex="1" columnIndex="1">
<!code>            <htmlb:label      text = "Name:"  for = "Name" />
<!code>          </htmlb:gridLayoutCell>
<!code>          <htmlb:gridLayoutCell rowIndex="1" columnIndex="2">
<!code>            <htmlb:inputField id   = "Name" />
<!code>          </htmlb:gridLayoutCell>
<!code>          <htmlb:gridLayoutCell rowIndex="2" columnIndex="1">
<!code>            <htmlb:label      text = "Email:"  for = "Email" />
<!code>          </htmlb:gridLayoutCell>
<!code>          <htmlb:gridLayoutCell rowIndex="2" columnIndex="2">
<!code>            <htmlb:inputField id   = "Email" />
<!code>          </htmlb:gridLayoutCell>
<!code>        </htmlb:gridLayout>
<!code>      </htmlb:form>
<!code>    </htmlb:page>
<!code>  </htmlb:content>
<!code>   
 

<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb" prefix="htmlb"%>
<!code>  <htmlb:content design="design2003">
<!code>    <htmlb:page title = "Test Page">
<!code>      <htmlb:form>
<!code>        
<!code>          
<!code>            <htmlb:label      text = "Name:"  for  = "Name" />
<!code>          
<!code>            <htmlb:inputField id   = "Name" />
<!code>          
<!code>            <htmlb:label      text = "Email:" for = "Email" />
<!code>          
<!code>            <htmlb:inputField id   = "Email" />
<!code>          </td></tr>
<!code>        </table>
<!code>      </htmlb:form>
<!code>    </htmlb:page>
<!code>  </htmlb:content>
<!code>   
 

<!code>  <%@page language="abap"%>
<!code>  <%@extension name="htmlb"  prefix="htmlb"%>
<!code>  <%@extension name="phtmlb" prefix="phtmlb"%>
<!code>  <htmlb:content design="design2003">
<!code>    <htmlb:page title = "Test Page">
<!code>      <htmlb:form>
<!code>        <phtmlb:matrix>
<!code>          <phtmlb:matrixCell row="1" col="1"/>
<!code>            <htmlb:label      text = "Name:"   for = "Name" />
<!code>          <phtmlb:matrixCell row="1" col="2"/>
<!code>            <htmlb:inputField id   = "Name" />
<!code>          <phtmlb:matrixCell row="2" col="1"/>
<!code>            <htmlb:label      text = "Email:"  for = "Email" />
<!code>          <phtmlb:matrixCell row="2" col="2"/>
<!code>            <htmlb:inputField id   = "Email" />
<!code>        </phtmlb:matrix>
<!code>      </htmlb:form>
<!code>    </htmlb:page>
<!code>  </htmlb:content>
</pre>

In the past few weeks in SDN, there have been a number of questions on handling HTMLB events. One of the biggest frustrations was with our

examples. This is a typical example:

<!code>  DATA: evt TYPE REF TO if_htmlb_data.

<!code>  evt = cl_htmlb_manager=>get_event_ex( request ).

<!code>  IF evt IS NOT INITIAL AND evt->event_name = 'button' AND evt->event_type = 'click'.

<!code>    ...

<!code>  ENDIF.

<!code>   

The first questions were always: Where can you obtain a list of events that are available? What are these magic strings? What happens if we write

them in upper/lower case?

The second problem was that of event dispatching. For the very old HTMLB library, we supported an interface if_htmlb_event that could be

implemented by an event handler class. However, this work was never done for the XHTMLB or PHTMLB libraries. Then a few days ago, we saw an example of

this technique being actively used with the newer libraries. So definitely it was time to clean this up.

Last, there was the question in the forum about a technique to build an OO framework for the event handling. If we have a button with onClick =

“save”, we would actually like to have the event dispatching done directly onto a method called “save”. After fiddling a little with the code, this

suddenly becomes relatively easy.

In the last two weeks, we have reworked the entire event handling code of the HTMLB family of libraries to address these three points. All of these

improvements will be shipped in the next service pack (620SP43 & 640SP05) that should be available in September 2004. This weblog will give an

overview of the three different techniques on how to handle HTMLB events. It is based on the new work we did. Some parts of this work in older service

packs; but especially for the event dispatching, the above service packs are required.

Test Program

is used to show the current index. Only a small part of the layout

code is shown here. The

is available.

<!code>  <xhtmlb:buttonGroup id = "btngrp" onClick = "TablePager">

<!code>    <xhtmlb:buttonGroupItem key      = "prev_page"

<!code>                            text     = "Previous Page"

<!code>                            design   = "PREVIOUS"

<!code>                            disabled = "<%=vIndex_prev_disabled%>" />

<!code>    <xhtmlb:buttonGroupItem key      = "next_page"

<!code>                            text     = "Next Page"

<!code>                            design   = "NEXT"

<!code>                            disabled = "<%=vIndex_next_disabled%>" />

<!code>  </xhtmlb:buttonGroup>

<!code>   

The output for the complete test program is shown below. In this display, the first visible row is currently one, and thus the button to navigate

to the previous page is disabled.

!https://weblogs.sdn.sap.com/weblogs/images/164/BP_HHE_001.GIF|height=0 width=545 height=270 |width=0 width=545 height=270 |src=https://weblogs.sdn.sap.com/weblogs/images/164/BP_HHE_001.GIF|border=0 width=545 height=270 !

Approach One: Manually Handling All HTMLB Events

The first approach to HTMLB event handling is to retrieve the event, and then simply investigate what type of event it is. This type of coding is

usually done in the OnInputProcessing handler of BSP pages.

The incoming event is retrieved with the call cl_htmlb_manager=>get_event_ex. In all cases, only this new method must be used. The older

get_event method is obsolete and not supported for the XHTMLB and PHTMLB libraries. Note that this method can be called more than once, and will

always return the same event.

The get_event_ex method will return an event object that implements at least the if_htmlb_data interface. This interface has a number of

interesting parameters that can be examined to see what type of event has been received. See also “

” for an in-depth discussion of the different event parameters.

<!code>  DATA: event TYPE REF TO if_htmlb_data.

<!code>  event = cl_htmlb_manager=>get_event_ex( request ).

<!code>   

<!code>  IF  event IS NOT INITIAL

<!code>  AND event->event_name = xhtmlb_events=>buttongroup

<!code>  AND event->event_type = xhtmlb_events=>buttongroup_click.

<!code>    CASE event->event_defined.

<!code>      WHEN 'prev_page'.

<!code>        vindex = vindex - vsize.

<!code>        IF vindex < 1.

<!code>          vindex = 1.

<!code>        ENDIF.

<!code>      WHEN 'next_page'.

<!code>        vindex = vindex + vsize.

<!code>        IF vindex >= LINES( sflight ).

<!code>          vindex = LINES( sflight ) - vsize + 1.

<!code>        ENDIF.

<!code>    ENDCASE.

<!code>  ENDIF.

<!code>   

This approach of event handling is very fast to program, especially on a BSP page. The disadvantage of this approach is that the code quickly

explodes once events for many controls must be handled.

Approach Two: Dispatching Events via IF_HTMLB_EVENTS Interface

The HTMLB rendering libraries also contain a technique to dispatch events to a handler class. For this, use the

cl_htmlb_manager=>dispatch_event_ex method. One of the parameters is a handler class that will accept the incoming event and process it. Typically,

this can be a separate developed class, a controller class, or even the application class.

In this example, we will use a handler class that has been developed separately. On the BSP page, in the OnInputProcessing method, the event

handling code now reduces to a few lines. All that is required is an instance of the handler class, and then the dispatcher is called.

<!code>  DATA: handler TYPE REF TO CL_SDN_HANDLING_HTMLB_EVENTS.

<!code>  CREATE OBJECT handler.

<!code>  cl_htmlb_manager=>dispatch_event_ex( request       = request

<!code>                                       page_context  = page_context

<!code>                                       event_handler = handler ).

<!code>   

The benefit of this approach is that the event handling code is placed into a separate class, where the full strength of the workbench can be used.

It reduces clutter in BSP pages. In addition, if new controls are added, no further plumbing code is required on the BSP page. All events will be

dispatched by this one call.

The important question is how will the handler class know what events are available, and what parameters each event handling method must have. For

this, an interface if_htmlb_events is defined. This interface contains all the possible events that can be fired by the HTMLB library. Each method

contains the correct parameters with which it will be called. Similar interfaces exist: if_xhtmlb_events and if_phtmlb_events for the other two

libraries.

!https://weblogs.sdn.sap.com/weblogs/images/164/BP_HHE_003.GIF|height=0 width=402 height=238 |width=0 width=402 height=238 |src=https://weblogs.sdn.sap.com/weblogs/images/164/BP_HHE_003.GIF|border=0 width=402 height=238 !</body>

Overview of Event Handling in HTMLB

HTML/HTTP does not support the concept of server events. At the lowest level, the only building block that is

available is forms in HTML, which can be submitted to a server. When a form is submitted, all input fields (which

includes hidden input fields) are transported to the server. Therefore, event handling in the browser reduces to

setting up specific predefined input fields (these are usually type=”hidden”) with values that reflect the event to

be sent to the server, and then submitting the form.

On the server, the event handling system will look at the incoming HTTP request. If it detects form fields with

well-known names (all HTMLB event input fields have a prefix ‘htmlbevt_’), it will signal an HTMLB event, and unpack

the relevant fields into an event object. Let’s look briefly at all the components that are involved in the event

handling system.

<!code>  Rendering Phase

<!code>    <htmlb:button id=”myBtn” onClick=”button_clicked”/>

<!code>    ... CL_HTMLB_BUTTON

<!code>        ... event = CL_HTMLB_MANAGER=>RENDER_EVENT_CALL(...).

<!code>        ... render onclick=’htmlbSubmitLib(...)’

<!code>   

<!code>  In Browser

<!code>    User clicks on button

<!code>    ... onclick is triggered, calls htmlbSubmitLib(...)

<!code>        ... Sets up a number of input fields with correct values

<!code>        ... calls form.submit();

<!code>   

<!code>  On Server

<!code>    event = CL_HTMLB_MANAGER=>GET_EVENT_EX( request )

<!code>        ... examines HTTP request for fields matching htmlbevt_*

<!code>        ... creates event object cl_htmlb_button, unpacks fields

<!code>   

During rendering, each control might require one or more events. This is usually done by wiring the HTML onclick

attribute with some JavaScript code that will handle the event. This specific, required JavaScript code is obtained

by a call to the method cl_htmlb_manager =>render_event_call. This method will return a sequence of JavaScript

code, which consists of one or more calls to the different JavaScript functions that are available for event

handling in the browser. The output of this method is internal only, already. In the past this output has been

improved a number of times. Do not try to concatenate this JavaScript output together directly, as this will cause

problems if the underlying event handling code is modified.

In the above examples, the JavaScript function htmlbSubmitLib is shown. However, the exact call that will be

generated depends on a number of factors, for example if a client side event is also involved, and whether the event

is listed in a predefined dictionary. Consider the output of the RENDER_EVENT_CALL method as a black box.

In the browser, once a control event is triggered, the JavaScript code in the onclick handler is executed. This

code calls the defined JavaScript code, which packs the event relevant information into hidden input fields, and

then submits the form.

At the server, the developer calls cl_htmlb_manager =>get_event_ex to see if any event was fired. (Note

get_event_ex is the newer replacement of get_event. The libraries XHTMLB and PHTMLB only work with the new

get_event_ex method. The HTMLB library has also been updated to work with this new method.) This method sees if any

form fields are in the incoming HTTP request that indicates an HTMLB event has been fired. If an event is detected,

the specific event handling class is found, instantiated and initialised with the event parameters. For all new

libraries, it’s expected that the rendering class is also the event handling class, and that this class implements

the additional interface if_htmlb_data.

When developing controls that use the HTMLB event system, there are only two relevant sections of code. The first

is the render_event_call for the JavaScript code that must be executed in the browser. The second is the

implementation of the if_htmlb_data interface, so that the HTMLB event system can initialize the event object with

relevant data.

RENDER_EVENT_CALL Method

Keep in mind that the actual generated JavaScript is placed

inside an HTML onclick sequence. In HTML, it is important to keep event bubbling in mind. One typical example is

when an anchor is used to render a control. If the onclick does not return false, the documentation  (http://help.sap.com/saphelp_nw04/helpdata/en/f8/7e1d3c55a0f503e10000000a114084/frameset.htm) for detailed instructions on writing BSP extensions and elements.  In the

workbench, create a new BSP element, and define a number of attributes. After the element is activated, all the

relevant classes are generated.

!https://weblogs.sdn.sap.com/weblogs/images/164/BID_UTHES_001.GIF|height=0 width=570 height=365 |width=0 width=570 height=365 |src=https://weblogs.sdn.sap.com/weblogs/images/164/BID_UTHES_001.GIF|border=0 width=570 height=365 !

The name attribute is used to specify the name of the JavaScript function. Alternatively, if the event_code

attribute is specified, the JavaScript code will be returned (placed into this string reference), and no JavaScript

function is rendered.

The attributes id, event_type, onClick (mapped onto server_event), event_defined and return_value are strings

that will be passed directly to the render_event_call. These strings must be statically known during the rendering,

and cannot be determined later in the browser. The attribute onClientClick (mapped onto client_event) can contain

any sequence of JavaScript code that will be executed in the browser when the event is triggered.

The attributes p1, p2, and p3 are used to specify the parameters of the rendered JavaScript function, and will

then be passed to the event handling code.

As this will be an “empty” control (meaning it can have no body text), only the do_at_beginning method is

implemented. The complete method is listed below, and then discussed.

<!code>  METHOD if_bsp_element~do_at_beginning.

<!code>   

<!code>    IF me->id IS INITIAL.

<!code>      me->id = 'id'.

<!code>    ENDIF.

<!code>   

<!code>    IF me->event_code IS NOT BOUND.

<!code>      IF me->name IS INITIAL

<!code>      OR me->name    CN '_abcdefghijklmnopqrstuvwxyzABCDEFGHIKLMNOPQRSTUVWXYZ0123456789'

<!code>      OR me->name(1) CA '0123456789'.

<!code>        me->raise_error( msg = 'Der Parameter "name" ist kein gültiger

JavaScript-Name.'(001) ).

<!code>      ENDIF.

<!code>    ENDIF.

<!code>   

<!code>    DATA: param_string TYPE string,

<!code>          param_count  TYPE I VALUE 0.

<!code>    IF me->p1 IS NOT INITIAL.

<!code>      CONCATENATE param_string `,` me->p1 INTO param_string.

<!code>      param_count = param_count + 1.

<!code>    ENDIF.

<!code>    IF me->p2 IS NOT INITIAL.

<!code>      CONCATENATE param_string `,` me->p2 INTO param_string.

<!code>      param_count = param_count + 1.

<!code>    ENDIF.

<!code>    IF me->p3 IS NOT INITIAL.

<!code>      CONCATENATE param_string `,` me->p3 INTO param_string.

<!code>      param_count = param_count + 1.

<!code>    ENDIF.

<!code>   

<!code>    DATA: event_string TYPE string.

<!code>    event_string = CL_HTMLB_MANAGER=>RENDER_EVENT_CALL(

<!code>                      BSP_ELEMENT          = me

<!code>                      EVENT_TYPE           = me->eventType

<!code>                      SERVER_EVENT         = me->onClick

<!code>                      CLIENT_EVENT         = me->onClientClick

<!code>                      CLIENT_EVENT_INLINED = 'X'

<!code>                      EVENT_DEFINED        = me->p0

<!code>                      PARAM_COUNT          = param_count

<!code>                      PARAM_STRING         = param_string

<!code>                      RETURN_VALUE         = js_return ).

<!code>   

<!code>    IF me->event_code IS BOUND.

<!code>      me->event_code->* = event_string.

<!code>    ELSE.

<!code>      IF STRLEN( param_string ) IS NOT INITIAL.

<!code>        SHIFT param_string LEFT.

<!code>      ENDIF.

<!code>      CONCATENATE ``

<!code>                    `function ` name `(` param_string `){ ` event_string `;}`

<!code>                  `</script>`

<!code>             INTO event_string.

<!code>      me->print_string( event_string ).

<!code>    ENDIF.

<!code>   

<!code>    rc = CO_ELEMENT_DONE.

<!code>   

<!code>  endmethod.

<!code>   

The first important step is to do a sanity check on the attribute name, as this string will be used to generate

the name of a JavaScript function. This is only done if the event_code is not returned directly.

In the next code block, the optional parameters p1, p2, and p3 are built into one string. This string will be

required twice. It is needed to build the interface for the JavaScript function and as a parameter list onto the

HTMLB event system call.

Thereafter, the render_event_call is used to retrieve the correct HTMLB event system JavaScript code. Note that

the returned code depends very much on the mix of input parameters. The actual string returned is handled as a black

box. No assumptions are made about the string or its format.

As the last step, either the event string is returned if so requested, or a JavaScript function is rendered that

can be used later to fire the event.

Handling Incoming Events

On incoming HTTP requests, you use the get_event_ex call to determine if an HTMLB event is available. The

cl_htmlb_manager will map the event onto the correct class, which is by default the same class that is also used for

rendering the control. It instantiates a new copy of this class, and then does a query for the if_htmlb_data

interface.

The method event_initialized will be called with all the standard attributes of an HTMLB event. In our case,

these values are restored onto the event attributes defined on if_htmlb_data. The last call will be to

event_set_parameters with all additional parameters that were available in the incoming HTTP request. These are also

restored into the class attributes.

<!code>  METHOD if_htmlb_data~event_initialize.

<!code>   

<!code>    me->if_htmlb_data~event_id          = p_event_id.

<!code>    me->if_htmlb_data~event_type        = p_event_type.

<!code>    me->if_htmlb_data~event_class       = p_event_class.

<!code>    me->if_htmlb_data~event_name        = p_event_name.

<!code>    me->if_htmlb_data~event_server_name = p_event_server_name.

<!code>    me->if_htmlb_data~event_defined     = p_event_defined.

<!code>   

<!code>    me->id            = me->if_htmlb_data~event_id.

<!code>    me->event_type    = me->if_htmlb_data~event_type.

<!code>    me->event_defined = me->if_htmlb_data~event_defined.

<!code>    me->onclick       = me->if_htmlb_data~event_server_name.

<!code>   

<!code>  ENDMETHOD.

<!code>   

<!code>  METHOD if_htmlb_data~event_set_parameters.

<!code>   

<!code>    me->p1 = p_param_1.

<!code>    me->p2 = p_param_2.

<!code>    me->p3 = p_param_3.

<!code>   

<!code>  ENDMETHOD.

<!code>   

With the handling of an incoming event, the integration into the HTMLB event handling system is complete, and the

control can be used.

Using the Event System

Using the HTMLB event system now reduces to using the control to generate a JavaScript function that can be

called from any other HTML or JavaScript code.

<!code>  <%@page language="abap"%>

<!code>  <%@extension name="bsp"   prefix="bsp"%>

<!code>  <%@extension name="htmlb" prefix="htmlb"%>

<!code>   

<!code>  <htmlb:content design="design2003">

<!code>  <htmlb:page>

<!code>   <htmlb:form>

<!code>   

<!code>   

<!code>     <bsp:htmlbEvent name="buttonPressed" p1="btnId"/>

<!code>  myButton1

<!code>   

<!code>     <% DATA: event_code TYPE STRING. %>

<!code>     <bsp:htmlbEvent event_defined="myBtn2" event_code="<%=event_code%>"/>

<!code>   

<!code> 

">myButton2]]>

<!code>   

<!code>   </htmlb:form>

<!code>  </htmlb:page>

<!code>  </htmlb:content>

<!code>   

Two different techniques can be used now. In the first, the name attribute is set, and one additional attribute

P1 is defined. This will result in a JavaScript function, “buttonPressed(btnId) {...}” been rendered out. In the

next step, raw HTML is used, where the onclick handler is now tied to the HTMLB event system. (The sequence

is used to generate a button in HTML that can be used to submit a form.)

The alternative technique is to set the event_code attribute. Because this attribute is flagged as a reference

attribute, the BSP compiler will pass a reference to the variable into the control. The control will write the

generated JavaScript code directly into this variable, so it can be used.

The (heavily edited) output seen in the browser is listed below.

<!code> 

<!code>   

<!code>   

<!code>   

<!code>      function buttonPressed(btnId)

<!code>      {

<!code>        htmlbSubmitLib('htmlb', this, 'bsp:htmlbEvent:EVENT:null', 'htmlb_form_1', 'id', 'buttonPressed',

1, btnId);

<!code>        return false;

<!code>      }

<!code>    </script>

<!code>   

<!code>  myButton1

<!code>   

<!code>  myButton2

<!code>   

<!code>  </form></body></html>

<!code>   

Incoming events are now handled exactly the same way as HTMLB events!

<!code>  DATA: event       TYPE REF TO if_htmlb_data,

<!code>        htmlb_event TYPE REF TO cl_bsp_htmlb_event.

<!code>  event = cl_htmlb_manager=>get_event_ex( request ).

<!code>  IF event IS NOT INITIAL AND event->event_class = 'CL_BSP_HTMLB_EVENT'.

<!code>    htmlb_event ?= event.

<!code>    ... use htmlb_event to see p1,p2,p3

<!code>  ENDIF.

<!code>   

The returned interface reference already has enough data to determine nearly all attributes of the event. Only if

the parameters p1, p2, or p3 are required, is the actual cast into the class required. The parameters are attributes

of the class, and not of the generic HTMLB event system.

If the event data is only displayed on the BSP example page, you have the following output:

!https://weblogs.sdn.sap.com/weblogs/images/164/BID_UTHES_002.GIF|height=0 width=283 height=220 |width=0 width=283 height=220 |src=https://weblogs.sdn.sap.com/weblogs/images/164/BID_UTHES_002.GIF|border=0 width=283 height=220 !</body>

In a previous BSP Programming: RSS = HttpClient + XML + XSLT, ten lines of code using the HttpClient was sufficient to fetch an RSS feed via HTTP. Once I mastered the art of programming the HttpClient, I had the idea to crawl through SDN weblogs to gather some statistics. As the information about all weblogs is not available via RSS feeds, I decided to fetch the HTML pages, and parse them.

Accessing SDN required an HTTPS connection, which has a few fine-points that must be kept in mind. This weblog shows the usual pitfalls into which I also landed, plus the workarounds. However, at one moment I was stuck, and decided to raise the stakes. Instead of using the HttpClient, why not use a real browser (HTML Viewer) integrated into the SAPGUI? This weblog will quickly touch the pitfalls when making HttpClient connections, and then look at an interesting alternative.

Making an HTTPS Connection

Making an HTTP connection with the HttpClient is very easy. Making an HTTPS connection is just as easy, if you first remember to import the certificate of the SSL partner! For SSL connections, the two partners exchange their certificates. The outgoing connection will only be established if the partner certificate can be verified against a copy stored in the database.

SAP ships very few certificate as standard. For other certificates required, you must get them from the partner directly. As an alternative, if the certificate is already available in the web browser, you can export it from there.

Once the certificate is available, it can be imported using transaction STRUST. See also the documentation on this topic. With this additional step done, HTTPS connections work exactly the same way as HTTP connections. There is only one small difference: traffic for HTTPS connections are not traced in ICM, due to security reasons (otherwise one could have used HTTP:).</p>

Handling Redirects

The HttpClient will automatically handle all rc=302 (Redirect) requests. However, there is one case where special handling is required. It is possible for the server to set a cookie during the redirect phase, and these additional cookies must be kept in mind when following the redirect. This is not currently being done by the HttpClient (although it’s now under consideration).

For an example, see this trace (strongly edited!):

<!code>  GET http://sdn.sap.com:80/ HTTP/1.1

<!code>  accept: /

<!code>  host: sdn.sap.com:80

<!code>               

<!code>  HTTP/1.1 302 Object moved

<!code>  Date: Wed, 23 Jun 2004 20:44:47 GMT

<!code>  Location: https://www.sdn.sap.com/

<!code>  Content-Type: text/html

<!code> 

Set-Cookie: ASPSESSIONIDCABABDDR=DJCHEEOALBBIBLPJKAOGDPBM; path=/

<!code>   

<!code>  GET / HTTP/1.1

<!code>  accept: /

<!code>  host: www.sdn.sap.com:443

<!code>   

What we see is that the first GET request is answered by the server with an rc=302 (Redirect) and a “Location” header is supplied. In addition, the server sets a cookie. However, in the default handling of the redirect, the next GET request (to the new location) does not contain the cookie.

Handling the redirects is very easy. Then the traffic reduces to the usual send-receive cycles, and cookies are handled correctly. A small change was made, to flag that redirects should not be followed, and the case of rc=302 was specifically handled in code.

<!code>  http_client->propertytype_redirect = http->co_disabled.

<!code>  ....

<!code>  http_client->receive( ).

<!code>  http_client->response->get_status( IMPORTING code = rc ).

<!code>   

<!code>  IF rc = 302.

<!code>    location = http_client->response->get_header_field( ‘Location’ ).

<!code>    me->GET( url = location ).

<!code>    RETURN.

<!code>  ENDIF.

<!code>   

Note that the GET() method is part of the crawler development, so as to give a higher level interface to the HttpClient. It just packages a number of HttpClient calls into one method.

Special Situation: Headers in HTTP Outgoing Requests

This section can best be described by starting with a small trace of the traffic:

<!code>  POST /SAPPortal/common/CreateNewCookie.asp HTTP/1.1

<!code>  cookie: ASPSESSIONIDCCRABTTC=DGHCFCHCLAKINDJFJAKAGJMO;

<!code>  content-type: application/x-www-form-urlencoded

<!code>  content-length: 46

<!code>  user-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)

<!code> 

host: www.sdn.sap.com

<!code>   

<!code>  portalUserName=user&portalPassword=password

<!code>   

<!code>   

<!code>  HTTP/1.1 400 Host Required In Request

<!code>  Date: Thu, 24 Jun 2004 18:04:50 GMT

<!code>  Content-Type: text/html; charset=iso-8859-1

<!code>  Content-Length: 447

<!code>   

<!code> 

<!code>   

<!code>   

Host Header Required

<!code>   

<!code>    Description: Your browser did not send a "Host" HTTP header field and

<!code>    therefore the virtual host being requested could not be determined. To

<!code>    access this web site correctly, you will need toupgrade to a browser that

<!code>    supports the HTTP "Host" header field.

<!code>   </BODY>

<!code>  </HTML>

<!code>   

For performance reasons, the ICM (in the kernel) translates all HTTP headers to lower case. The HTTP headers are also sent out in their lower case form, and not in the usual capitalized form. For the interested reader, I would refer to the HTTP/1.1 spec (RFC 2616 ): “Each header field consists of a name followed by a colon (":") and the field value. <b>Field names are case-insensitive</b>.”

The above message seems to be caused by a case-sensitive string processing sequence (where “Host” is not equal to “host”). Unfortunately, it is not clear who is returning the message. It could be any proxy, load balancer, dispatcher, server or even some custom written servlet along the way. This made it difficult to find (and negotiate) a workaround for this problem.

Using a Real Browser: The HTML Viewer

I was always comparing my requests with that of a real browser, so why not use a real browser? There is a complete browser integrated into SAPGUI (under Windows) that can be programmed with ABAP!

As the first step, I quickly read the online documentation , and looked at the test programs SAPHTML_DEMO1 and SAPHTML_EVENTS_DEMO. Then it was mostly cut-and-paste work to get the first version up and running.

Only small bits and pieces of the basic crawler will be discussed here. For a complete overview of the code, follow this link.

From the beginning, it was clear that any solution would have some JavaScript code in it (at least at the time of writing this text!). The problem is that the cl_gui_html_viewer does not expose its JavaScript APIs directly. These methods are flagged as protected. It was important to define a new class that inherits from the cl_gui_html_viewer class.

<!code>  CLASS cl_html_crawler DEFINITION

INHERITING FROM cl_gui_html_viewer

<!code>    PUBLIC SECTION.

<!code>    METHODS: load_url        IMPORTING uri TYPE string.

<!code>    METHODS: on_navigate_complete FOR EVENT navigate_complete OF cl_gui_html_viewer.

<!code>    METHODS: on_sapevent FOR EVENT sapevent OF cl_gui_html_viewer

<!code>                   IMPORTING action postdata.

<!code>    ....

<!code>  ENDCLASS.

<!code>   

The HTML viewer raises a number of events. The first interesting one is the on_navigation_complete event that is fired after the document has been loaded. In addition, it is possible to “talk” from the browser to the SAPGUI (effectively back to ABAP on the server) using special SAP events inside the browser. These will cause the on_sapevent method to be triggered.

Most of the glue code is copied from the demo programs and documentation and is not listed here. We will look at thee interesting aspects: the loading of a document in the HTML viewer, catching the event that the document has been loaded, and extracting the document content.

For the load_url function

For example:

<!code>  'GET__http://sdn.sap.com'

<!code>  'POST__https://www.sdn.sap.com/logon.asp__user=me__password=secret'

<!code>   

For the GET sequences, the code was very simple. The HTML viewer already contained a method show_url that effectively handled the GET completely. However, for a POST request, you require a complete HTML document with form that can be posted. For POST requests, the load_url method would build a complete HTML document, load it into the browser, from which it could be posted to the URL.

<!code>  METHOD load_url.

<!code>   

<!code>    DATA:  url      TYPE char255,

<!code>           method   TYPE string,

<!code>           ffs      TYPE string,

<!code>           ff       TYPE string,

<!code>           ff_name  TYPE string,

<!code>           ff_value TYPE string.

<!code>   

<!code>    SPLIT uri AT '__' INTO method url ffs.

<!code>   

<!code>    IF method = 'GET'.

<!code>      me->show_url( url = url ).

<!code>      RETURN.

<!code>    ENDIF.

<!code>   

<!code>    DATA: html TYPE TABLE OF char255,

<!code>          line TYPE char255.

<!code>    APPEND `` TO html.

<!code>    CONCATENATE '

<!code>    APPEND line TO html.

<!code>    WHILE ffs IS NOT INITIAL.

<!code>      SPLIT ffs AT '__' INTO ff ffs.

<!code>      SPLIT ff  AT '='  INTO ff_name ff_value.

<!code>      CONCATENATE '

<!code>      APPEND line TO html.

<!code>    ENDWHILE.

<!code>    APPEND '</form></body></html>' TO html.

<!code>   

<!code>    me->load_data( IMPORTING assigned_url = line CHANGING data_table = html ).

<!code>    me->show_url( url = line ).

<!code>   

<!code>  ENDMETHOD.

<!code>   

Given the example POST sequence above, the following HTML document is created:

<!code> 

<!code> 

<!code>   

<!code>   

<!code>  </form></body></html>

<!code>   

In this document, the onload is hooked, and once loaded, it will trigger a submit() call on the form. The form itself contains the action (target URL), as all the form fields are stored as hidden input fields in the form.

One line of source code is really worth highlighting:

<!code>    APPEND `` TO html.

<!code>   

ABAP is the only programming language that I know which supports two forms of quotes for creating strings. This allows us to write ABAP, HTML and JavaScript code in one line without any ‘escaping’ required.

Once the URL has been loaded, there is nothing to do but wait for the event signaling complete.

The problem is that there are API function to read the content from the browser. However, it is possible was to fire an event from the browser to the SAPGUI. Each SAP event is implemented as

<!code> 

<!code>   

<!code>  </form>

<!code>   

It is not possible to write this code directly into the loaded document, as that would destroy it. Therefore, I just used JavaScript to create the form dynamically (using createElement call), and to place the content into hidden fields. The content had to be split into a number of short strings, as the SAPGUI will place all input fields into a table of type CHAR255. So the JavaScript function just does the following steps: create new form, create a number of short input fields to hold segments of the content, and then hook this form into the document.

<!code>  METHOD on_navigate_complete.

<!code>   

<!code>    DATA: js   TYPE STANDARD TABLE OF CHAR255,

<!code>          line TYPE STRING..

<!code>   

<!code>    APPEND `function _Dump() {`                                          TO js.

<!code>    APPEND `  var _frm = document.createElement('form');`                TO js.

<!code>    APPEND `  frm.setAttribute( 'id',     'crawler' );`                TO js.

<!code>    APPEND `  frm.setAttribute( 'name',   'crawler' );`                TO js.

<!code>    APPEND `  _frm.setAttribute( 'method', 'POST' );`                    TO js.

<!code>    APPEND `  frm.setAttribute( 'action', 'SAPEVENT:SAVEDOCUMENT' );`  TO js.

<!code>    APPEND `  var _str = document.body.outerHTML;`                       TO js.

<!code>    APPEND `  var _idx = 0;`                                             TO js.

<!code>    APPEND `  while(_idx < _str.length) {`                               TO js.

<!code>    APPEND `  var _if = document.createElement('input');`                TO js.

<!code>    APPEND `  if.setAttribute( 'name',  'content' );`                  TO js.

<!code>    APPEND `  _if.setAttribute( 'type',  'hidden' );`                    TO js.

<!code>    APPEND `  _if.setAttribute( 'value',  str.substr(idx,200) );`      TO js.

<!code>    APPEND `  _frm.appendChild( _if );`                                  TO js.

<!code>    APPEND `  _idx+=200;`                                                TO js.

<!code>    APPEND `  }`                                                         TO js.

<!code>    APPEND `  document.body.appendChild(_frm);`                          TO js.

<!code>    APPEND `  document.all["_crawler"].submit();`                        TO js.

<!code>    APPEND `}`                                                           TO js.

<!code>    APPEND `window.setTimeout("_Dump();",5000);`                         TO js.

<!code>   

<!code>    me->set_script( script = js[] ).

<!code>    me->execute_script( ).

<!code>   

<!code>  ENDMETHOD.

<!code>   

Once the JavaScript function is injected into the browser, it is not immediately executed. From practical experience I saw that sometimes the browser was still busy loading images or executing JavaScript code. So a timer was set to execute the dump function only five seconds later. This also gave a few moments of time to see what was loaded, and verify that the crawler was still on the correct track.

Note that the functions to load the JavaScript code into the browser are protected, and can not be called when using an instance of the class cl_gui_html_viewer. This is the main reason for the inheritance approach, sothat we could actually get access to these two functions.

Five seconds later the form was submitted and the on_sapevent method is called. As input, a table is received that contains a number of rows, each of the sequence “_content=html”. All the lines are concatenated together again into one string. The final string is massaged to remove some of the HTML escaping that was done on the data, and the _content sequences.

<!code>  METHOD on_sapevent.

<!code>   

<!code>    DATA: content TYPE STRING,

<!code>          line    LIKE LINE OF postdata.

<!code>    LOOP AT postdata INTO line.

<!code>      CONCATENATE content line INTO content.

<!code>    ENDLOOP.

<!code>   

<!code>    REPLACE ALL   OCCURRENCES OF '%3D'        IN content WITH '='.

<!code>    REPLACE ALL   OCCURRENCES OF '%3F'        IN content WITH '?'.

<!code>    REPLACE ALL   OCCURRENCES OF '&_content=' IN content WITH ''.

<!code>    REPLACE FIRST OCCURRENCE  OF '_content='  IN content WITH ''.

<!code>   

<!code>    me->loaded_content( content ).

<!code>   

<!code>  ENDMETHOD.

The other supporting code is not shown here, as it is mostly plumbing. The complete code can be found here.</p>

Final Words

Using the browser inside the SAPGUI was actually a rather interesting challenge, and we learned a lot about how the browser integration was done, and the possibilities that this enabled. It was now possible to write a web crawler using a true browser with all of its features and idiosyncrasies. In the next Weblog this simple web crawler will be used to build a small SDN crawler, and then extract some statistics from the Weblogs.

In my previous weblogs in this series - BSP Performance: Measuring Roundtrip Latency and BSP Performance: Statistic Records for Server Latency - we looked at techniques for measuring the actual performance of a BSP application. The next logical question is: can we improve that performance? That question can be answered only if we know what the application in question is doing.

For this last piece of the puzzle, we will use runtime analyses to get a detailed picture of an application&#146;s runtime behaviour. (This weblog does not seek to analyze any specific application; rather, it explores the tools available and looks at expected output.)

Activating Runtime Analysis

The runtime analysis of a BSP application (of any HTTP request, actually) is activated in ICF (see transaction SICF). See also the online help.

After the runtime analyses have been enabled, execute a few test BSP pages.

Looking at the Data

The data collected by the runtime analyses can be seen using transaction SE30. Simply select &#147;Other File&#133;&#148; for the specific user, then double-click on any of the URLs listed.

The gory details are not interesting in the context of this weblog. Instead, let us look briefly at the type of information that can be learned from transaction SE30. The true value of this tool can only be learned from practical (home:) work. Also read the documentation.

The first interesting aspect is the Hit List. This shows all methods and functions called, and their gross and net times. To get an overview of which methods have the longest runtimes, sort on the net times. Also shown is the number of times each method is called. This often helps identify inefficiencies in the way that code is structured.

The Group Hit-List shows top entries grouped into different categories, including method calls, function calls, database access, and program load times.

Database access is usually the more expensive part of an application, and the Database Hit-List provides detailed information about databases that were referenced.

The Call Hierarchy shows the true calling sequence of the application. This can be a daunting display for complex applications, but helps to correlate the dynamic behavior of the application to the more static code one sees in the editor. This is especially interesting to understand paths taken through the code, and for the &#147;who called this method&#148; type of question.

It is always possible to use forward navigation (double-clicking) to get more detailed information about any specific sequence that is displayed. Use the blue &#147;Display Source Code&#148; button at any point to jump directly to the relevant source code.

Summary

This weblog has added the &#147;what is happening&#148; tool to the performance toolbox. This is the most interesting one, as it allows us to find hotspots in the code, and to zoom in the specific areas that can be improved.

Performance measurement (How fast does it go?) is the inverse of the question of scaling (How large the box to support N users per time unit?). Which highlights the central theme: measuring the performance of an application.

However, BSP pages are not applications, but merely the enabling technology to support the development of web applications. As such, each BSP application must be measured individually assess its performance characteristics. The first step is to test the overhead of the BSP runtime per se. Any application&#146;s performance will always include this basic overhead. This Weblog will concentrate on measuring the roundtrip latency for the BSP runtime. The roundtrip latency of an application can be measured similarly.

For measuring the BSP overhead, a small &#147;Hello World&#148; BSP page is created, and then fetched from the browser. This simple test measures the latency from browser to WebAS; authentication time; time to create the session; overhead to instantiate all the BSP runtime classes; and the call to the specific BSP page. This is the minimum overhead for each HTTP request onto a BSP page.

The tests are done for BSP pages of 1KB, 2KB and 4KB in size (up to 64KB is usual). These tests verify that there are no memory bandwidth problems within the system (memory copies, et. al.). It is expected that larger pages will take slightly longer, mostly due to larger transmission time on the physical network.

For a second series of tests, GIF images are fetched from the server. Access to the MIME repository is very expensive. However, the images are cached in the ICM cache, and the access time of subsequent does not relate to the first load time. Therefore, we always look only at the performance of the subsequent image loads.

Test Application

For the tests, a BSP application IT03 was used, which consists of a simple &#147;Hello World&#148; page, called &#147;text0kb.htm&#148;. This is the primary page for measuring the BSP runtime overhead. In addition, a number of &#147;textNkb.htm&#148; pages are available for testing. The IT03 application also includes a collection of different imageNkb.gif images.

Factor to Consider: URL Mangling

When doing performance measurements, the goal is to run the same test a number of times. However, most test programs are structured so that each test is run individually, the results recorded, and then the test is started anew. This dictates that specific once-off actions experienced only once by a user are measured continuously by the test.

URL mangling is one such an example. In BSP applications a redirect is done to embed specific information (language, client, etc.) on the first hit for the application. For example, when the user requests the URL &#147;/sap/bc/bsp/it03/page.htm&#148; from the server, a redirect is first done to the new URL &#147;/sap(bD1ZS0PRF1Rw==)/bc/bsp/it03/page.htm&#148;. For performance measurements, only use the mangled URLs, to exclude the once-off mangling from the results.

Factor to Consider: Authentication

A similar problem is that of authentication. When each test run is started, a GET request for an URL is done. However, the WebAS requires authentication from the user, and the HTTP request is returned (reason: authentication required). The browser then pops up a dialog requesting a username and password from the user. This data is encoded into the HTTP header (field &#147;Authorization&#148;), and the URL is requested again. On all subsequent requests to the server, the browser will automatically send the Authorization header with the request.

In this case, two roundtrips for each test are often measured. After the first request is rejected, the test programs will use the stored username and password to test again, and the double roundtrip time is measured for each test run. It is important to ensure that this authentication data is already part of the initial request, to measure the actual request time, and not the additional authentication roundtrip.

Factor to Consider: Program Load Times

At a technical level, a BSP page is a generated ABAP class, for which a load (compiled program) is stored in the database. When an URL is requested the first time, the required program is loaded from the database, and cached in the program buffer. On subsequent requests, the program can be used directly from the buffer.

This first time database load is very expensive, and influences averages times, especially on short runs. Therefore, in all tests, it is highly recommended to add a warm-up phase before the tests. This also ensures that all images required are placed in the ICM cache (which will be the expected case for normal usage).

Factor to Consider: Network Latency

In typical use of intranets, the latency over the network is so small (milliseconds), that it is not really noticed. However, when performance measurements are done in the same range, the network latency plays a large role.

In in-house tests, where multiple hops must be traversed from browser to server, this has added a noticeable additional latency of a few milliseconds. It is recommended that for tests where the absolute numbers are important, both client and server be placed on the same sub-network.

In addition, the packet size of the network plays a role. Ethernet packets can be a maximum size of 1500 bytes. Large data packets are split into smaller IP packets, then sent individually and reassembled on the receiving side by the TCP/IP stack &#150; so although the data is sent all at once, the data in not received in one packet. This has prompted some tools to give both the &#147;time to first byte&#148; (TTFB) and &#147;time to last byte&#148; (TTLB) measurement times. Thedifference between the two values reflects more on network latency than on server processing time.

When testing in WAN environment or via the Internet, the properties of TCP become noticeable. Especially for larger data volumes, the TCP window-size plays a large role. An initial window of data is transmitted, and then transmission is stopped until an acknowledgement is received. This causes the TTFB and TTLB to be disparate.

Factor to Consider: Median Time versus Average Times

Quite often over the duration of a test, a few measurement points are seen which are clearly too large and do not reflect the expected behavior. For example, assume a run of a 1,000 tests, where 999 of the tests completed in exactly 10ms. However, one URL hit took 3 seconds (3,000 ms). Then the average of the 1,000 runs will be 12.99ms, and not the expected 10ms. We see that a few out-risers are capable of affecting the average. For test tests, median times are used. For the above example, the median would have been the expected 10ms.

Tool: Microsoft Web Application Stress Tool

There are many programs available for stress testing web applications. In principle, any program can be used. For this Weblog, we will use the Web Application Stress Tool from Microsoft. It is quick to install, and contains all the necessary features required for simple stress testing.

To download the program, go to Microsoft&#146;s website, and search for &#147;Web Application Stress Tool&#148;. The interesting links currently are: Download the Web Application Stress Tool and Web Application Stress Tutorial.

Running a Test

With the stress tool, it is relatively easy to record a session. For the tests, the BSP test program IT03 is used. Pages and images of 1KB, 2KB, and 4KB are tested. As a first step, either record or manually create the script.

One of the first post-processing steps required is updating the server value. This field is not correctly set during script creation. Also all URLs that we are not interested in are deleted. Test URLs are kept in their mangled form.

Two additional post-processing steps are required. Double click on the first URL, check that the port is correct, and add the Authorization header. Apply this change to all URLs. Unfortunately, if the port is not correct, this must be manually updated for all URLs.

Finally, we configure a few set-up parameters for the test. In all cases, keep the stress level at &#147;1&#148; so that the true end-to-end latency is apparent without having test threads blocking one another. Configure a short warm-up time and the run time. As we have already configured the user information via the Authentication header, we now need to store any additional information.

Looking at the Data

Allow me a short story. Once, a long time ago, someone decided to compare their server platform against the BSP runtime. At the end of the tests, they wrote a small hero email: They could handle thousands of hits per second! The numbers were much better than those for BSP. What the colleagues forgot to check was whether they were also getting back 200 OK answers for each hit. What was actually measured was thousands of 401 Authentication hits! For a server, rejecting a request is much faster than completely processing it. (No, they never did email a retraction.)

Thus, the first check is to ensure that the test did actually run successfully. Look at the summary page, and ensure only OK return codes. The distribution of hits per page should be nearly the same.

Script Settings  Server:                       us4049.wdf.sap.corp  Number of threads:            1  Test length:                  00:03:00  Warmup:                       00:00:30     Result Codes  Code      Description                   Count      200       OK                            381           Page Summary  Page                                                  Hits  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/text1kb.htm     64  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/text2kb.htm     63  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/text4kb.htm     63  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/image1kb.htm    63  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/image2kb.htm    64  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/image4kb.htm    64   

The next step is to look at the results of each page test in detail.

Result Codes  URI:  GET /sap(bD1lbiZjPTAwMA==)/bc/bsp/sap/it03/text1KB.htm  Code      Description                   Count      200       OK                            64            Time to first byte (in milliseconds)  Average:                      295.17  Min:                          181.75  25th Percentile:              190.26  50th Percentile:              192.89  75th Percentile:              299.48  Max:                          3405.89     Time to last byte (in milliseconds)  Average:                      295.31  Min:                          181.86  25th Percentile:              190.37  50th Percentile:              193.01  75th Percentile:              299.60  Max:                          3406.01     Downloaded Content Length (in bytes)  Min:                          1026  25th Percentile:              1026  50th Percentile:              1026  75th Percentile:              1026  Max:                          1026   

Specifically check that the result codes are all 200 OK, and check that the downloaded content length is stable over all tests. These are the first indicators of problems. Specifically if the content length varies over one test run, it can indicate that the page ran into a problem, and is suddenly rendering different output.

Once we are sure that the test run was acceptable, we look at the &#147;Time to First Byte&#148; (TTFB) and/or &#147;Time to Last Byte&#148; (TTLB) values. We know from the WebAS architecture that only complete HTTP responses are transmitted. So once transmission starts, the ABAP part of the processing is complete, and the difference between TTFB and TTLB only reflects network transmission time. TTFB is the more interesting value when looking at server performance (in which case the test machine and server should be close to one another on the same network). TTLB is the value to determine the HTTP roundtrip latency.

The last interesting aspect is the large difference between the average and 50th percentile values. This is due to a few out-risers that can dramatically influence the average value (specifically over a short test time). Because of this, we usually use only the 50th percentile values. (To be quite truthful, the fact that the 50th and 75th percentiles are not close to one another would usually prompt me to repeat the tests over a very long time. But for the educational value of this Weblog, these numbers are sufficient.)

Looking At All Tests Together

Here is a summary of all the tests.

Probably the most important question is whether these numbers are good or bad. Looking at the absolute numbers, they are horrible in terms of what we have measured in optimal lab conditions. However, one must evaluate the numbers in context of the actual tests done. And here, the goal was to measure HTTP roundtrip latency. These tests were run from home, using a DSL connection onto the Internet, with (very expensive!) encrypted VPN tunneling. Furthermore, the tests were done against a development system, running a debug kernel (maybe 30 percent slower). So overall, we are happy with the numbers.

Interesting is that the values for TTFB and TTLB are very close for 1KB and 2KB sizes. However, for 4KB sizes we see a marked difference. This is attributed to the network and TCP properties as discussed before.

Looking at the difference between a1KB page versus an image, we see about 20ms. We know that after the warm-up phase, all images will be stored in the ICM cache, and willbe answered from there. Thus using the TTFB for the image as a rough indicator of the network latency (ICM processing at kernel level is definitely below 1ms!), we get an estimated 20ms processing time for the 1KB page. This is inline of our expectations for a debug kernel. (Estimates confirmed with measurements in the office: 50th percentile was 23ms.)

Summary

This Weblog has not attempted to put absolute perfect numbers on the table. Rather, a more realistic usage scenario was chosen. The first and most important goal was to show that it is relatively easy to make HTTP roundtrip latency measurements. Furthermore, the obtained numbers were slightly interpreted, just to give a feeling for how these numbers should be &#147;read.&#148;

Importance is that these are base line numbers that show the complete end-to-end latency for the specific scenario. This includes the actual transmission time, plus the processing time of the BSP runtime. Using a very small page, we can show that the BSP runtime itself is quite fast up to the point where the application coding is started. As a next step, one must measure a complete application. From there, it can be determined what part of the total time is attributed to the BSP runtime.

Be careful when interpreting single numbers to get a sizing number. It is important that a realistic scenario through the application be used, and that the results of the different URLs are used to calculate the true overhead of the server. Also consider the expected latency a user will experience in a similar network environment and the effects of MIME objects.