juergen.gatter

    6 Posts

    Last BPath lesson was presented some time ago. The performance investigations took slightly more time then expected, mainly due to other work. But let us concentrate on Bpath and the current lesson.

    BPath. Lesson 05. Sub, String & Get-function(s) plus global variables brought us a view over quite a lot of functions. This time we have only limited content in respect to new functionality but a bigger chapter in respect to performance.

    Unstructured Returns & Sorting 

    To allow easy access to the data we introduced the possibility to work on unstructured return types, such as integer values and strings. The rule is simple, if the declared return type is not structured; every write access to it is treated as writing to the value itself (irrespective of the target name used). Read access is not supported (use global vars instead).

    The following example returns the index of the first passenger beginning with "S".

     

    ~INT2/SearchResFlightRel/FlightBookRel[@PASSNAME~="S*"]{!RETURN=INDEX()}
    Code Example 30, ~INT2 or ~STRING, unstructured return data

     

    Another useful feature is the possibility to sort the result table according to your needs. The feature includes the sorting of sub query result tables. Sorting is treated as feature of the structure definition. This means, in case you want to sort the result the structure has to be stated explicitly. Dynamic structure definitions as ~* are allowed. The sorting field names are added encapsulated by [ ... ], the sort-order ('A' or 'D') might be added as option triggered with a colon (default is ascending order). 'T' option can be used to sort data as text. The star operator is used to indicate the current order, this means, if a star is part of the sort order definition, stable sorting is done.

     

    ~*CRMS_BOL_UIF_TRAVEL_BOOK_ATT[!FORCURKEY,!FORCURAM:AT,*]/SearchResFlightRel{!DS:=SUB(~*[!CONNID:D]/FlightConnectionRel/*$)}/FlightBookRel/*$
    Code Example 31, ~*[!FIELD:AT,*], sorting

     

    The main sorting is done on the foreign currency (ascending, since sort order is not specified) and then the amount (in foreign currency), ascending as text. Sorting is done stable.

    The inner sort order is just for display purposes - the sub query returns anyway only one entry in the table (as every flight has only one connection).

     

    The sorting is done at the very end of the evaluation. The sorting used for filtering, assignments, etc remains as defined by the underlying relations.

    Direct Indexing

    Up to version 2.1 (a BPath version not released to customers) any filter condition which returned a numeric value (e.g. n= 2 or n=-1) was implicitly evaluated with a check against the current index (means [ i =index()]). This is theoretically fine, but results in a very poor performance in case the numerical value does not depend on the attribute values.


    With the released version any filter returning a numeric value is directly indexed, in case it fulfills certain conditions. Directly indexed means that only the requested entity is evaluated, all other datasets are ignored and not further processed.

    The conditions are:

    • no access to any attribute in the filter
    • no subquery in the filter
    • no assignment block between relation and filter

    The result of any query should not change, but the performance should be significantly better in case direct indexing can be used within large collections.

    Please note that a negative indexing refers to entities counted from the end of the list, -1 is the last object in the collection. The syntax does not allow to use negative literals directly, [-1] has to be specified as [0-1].

     

    A rather crude example:

     

    ~STRING/SearchResFlightRel{!!Dummy=0-1}/FlightBookRel[!!Dummy]/@PASSNAME$
    Code Example 32, [0-1], direct indexing

     

    Performance Evaluations

    Unfortunately it is not easy to discuss performance seriously, because of the following reasons:

     

    • Absolute performance is highly machine dependent.
    • The BPath execution is dependent on the performance of the used relations, which can vary a lot.
    • Caching effects have to be encountered.
    • Even if the performance of a BPath statement execution might be pretty fast in terms of "doing all things it does pretty fast", but it may trigger actions which can be avoided if a different way would be chosen.
    • It is difficult to judge what "an optimal" performance would be, to which process we compare to. Practically all code examples can be further optimized, especially when you do special coding ignoring standard layers.

    The following simple evaluations were done in one of our systems, the goal of the investigations was to have an idea which performance you might expect and which guidelines may be considered. The machine where I did the tests was running in a solaris zone (virtual machine) with 2.6 GHz and 10Gb of RAM. It is a non-Unicode system.

     

    See below the table of results. The first column contains the BPath query, which was always executed against a collection of 10 "flights". The "result" column contains the number of datasets returned, whereas the "touched" column also contains all entities which were touched during execution. The "Frst" column holds a checkbox whether we measure the complete execution ("X") or whether we measure without the startup time (means we reuse the parser). Only the first example is measured with startup performance, but it is anyway only about 3 ms.

    The next column holds the needed time for the execution in microseconds, followed by the time required by the parser plus the stack.

    The second last column contains the percentage of time elapsed in BPath compared to the time elapsed in the lower layers (GenIL, application specific coding as relation loading) whereas the last column simply holds time in BPath divided by number of touched entities. In other words the time required for BPath to evaluate an object lying on the execution path.

     

    image

    Performance Learnings:

    1. Start up performance is about 3ms (can be avoided with parser reuse except for the first call).
    2. An easy BPath statement needs about 300 micro seconds per object it accesses during the execution.
    3. Complexity costs. In case the BPath statement is formulated with more complexity, e.g. with the usage of subs and functions, execution time rises, e.g. to 2500 micro seconds with the last example.
    4. Typical usages which evaluate at least one real relation and assemble data into a result table and work on uncached data need about 20-40% of the complete time within BPath itself. At least within the implementation of our Flights.
    5. Direct indexing can pay off as with the eighth example which needs substantially less time in BPath itself.
    6. Scaling should be fine, there seem to be no effects ruining the performance with big collections.
    7. Stack processing costs a significant proportion of time, there might be room for improvement (on the cost of maintainability unfortunately).

     

    Well, all these evaluations are done against "Flights", which are kind of artificial demo object structure. It is difficult to judge, how it looks when we work on real life objects.

    So I did some more tests in another system, which contains better data. System characteristica are comparable.

    My demo program reads a list of contacts with the BuilContactAdvancedSearch query where with telephone covers '0*' and country is 'DE' to avoid working on irregular data. Then my BPath query simply tries to collect all addresses with the following query:

      ./BuilContactPersonAddressRel/*$

    The query returns 33 entries, exactly one address per contact.

    An execution of the BPath requires 683 ms against uncached data with 25.5 ms in our BPath code itself. This makes it to 0.38 ms per touched entity (we have 2*33  = 66 entities) within our BPath classes.

    On cached data the whole execution time shrinks down to 32 ms, still with 25.5 ms in our BPath code. This means in this case Bpath directly causes for about 80% of the required time.

     

    So, in the end this example fits to the results from the table above and I hope you now have an idea about the performance you can expect from a BPath query.

     

    That's it for today. Lesson 7 will cover the model check mode and parser reuse plus some small language topics as attributed parent relations and attribute longtext.

    Hello everybody,

     

    After the excursion into various topics last time (BPath. Lesson 04. Enhanceable structures, aggregations, derefencation and transposition. And object retrieval.) we are looking forward to presenting some functions. Or at least kind of, since subs are not filed under functions from a syntactical point of view:

    Sub functions 

    sub-functions are triggering a new BPath statement starting on the actual entity:

     

    ~*/SearchResFlightRel{!DIST=SUB(FlightConnectionRel/@DISTANCE);!VALUE=0}$/FlightBookRel{!VALUE=!VALUE+1}/*
    Code Example 23, SUB(.../@FIELD), sub functions returning attributes

     

    In the early days a new parser was instantiated for any sub query. This was changed and integrated into the syntax as some future features were too difficult with several installations of BPath.

    Note that on this way only one field can be returned with every sub function.

     

    Well, but there are other ways:

     

    ~*/SearchResFlightRel{!DS=SUB(FlightConnectionRel/*);!VALUE=0}$/FlightBookRel{!VALUE=!VALUE+1}/*
    Code Example 24, SUB (.../*), sub functions returning structures or tables

     

    Looks quite similar to the code above, but instead of attribute we sub-request a structure. This structure is consequently included into the target structure as reference. You may do the same with subs returning tables, which are also included as references.

     

    Goal of this feature is the possibility to assemble hierarchical structured return data of course.

     

    Some further remarks:

     

    You may nest sub-functions even further and run a sub-query within a sub query.

    One may transfer data from the embedding query to the sub query using the global variable table (see below). But it is not possible to access the data of the embedding structure from within the sub query. 

     

    String functions 

    To ease working with strings, some string functions were introduced as the following:

     

    SearchResFlightRel/FlightBookRel[strcomp(@PASSNAME,"CA","äöüÄÖÜ")]/*$
    Code Example 25, STRCOMP(), string comparisons

     

    The STRCOMP function offers the string comparison functionalities as known in ABAP. These are: "CA", "CS", "CP", "CO", "CN", "NA", "NS" and "NP". The parameters are all of type string in the same order as in ABAP; this means that the comparator has to be specified (as string) as second parameter.

    The function returns a Boolean Value.

     

    For the often used CP (covers pattern) comparison a different syntax is also available: 

     

    SearchResFlightRel/FlightBookRel[@PASSNAME~="*sch*"]/*$
    Code Example 26, ~=, covers pattern

     

    The basic string functions LEN( string ), LEFT( string, len ) , RIGHT ( string, len ) and MID( string, offset, len ) were also included. In contrast to ABAP they are not dumping if len is exceeding the string length (then the complete string is returned) or if the offset exceeds the length (then an empty string is returned).

     

    ~LISTZEILE/SearchResFlightRel/FlightBookRel[LEN(@PASSNAME)>0][LEFT(@PASSNAME,1)=UPPER(RIGHT(@PASSNAME,1))]{!ZEILE=MID(@PASSNAME,1,LEN(@PASSNAME)-2)}$
    Code Example 27, LEFT, RIGHT and CENTER ... oops sorry MID. And LEN.

     

    Code example takes all non-empty Passenger Names where the first and the (uppercased) last character are equal and returns the content in between.

     

    Get(), Local and global assignments and the global variable table

    Well, we talked quite a lot on assignments, somehow navigating around the question whether "=" or ":=" should be used. Maybe it is time to squeeze some words on this topic between all the examples:

     

    The original version followed a simple approach when it came to assignments. The return content was simply never cleared and overwritten by the next row (at least for the standard case). This is fine in most of the cases but when we used dereferenced targets for transposing data it turned out that the data was never cleared and hence incorrect. Unfortunately there is also no way to clear the data manually.

    Hence we switched the concept completely and introduced the concept of locality and globality. Since the targets are not declared, the difference has to be expressed with the assignment itself.

    A local target will be cleared after a row is written to the return table. The content of a global target will 'survive' the writing to the return table. Locality is expressed with the = operator, for a global assignment the := operator has to be used.

    Technically the return structure will be cleared completely after adding a row to the return table. The content of the global targets is then filled again as done with the last assignment which is stored in the global variable table (see next chapter).

     

    Note the usage as follows:

     

    1. For normal fields written by each and every row local assignments with = should be used.
    2. (De-referenced) Entries only touched by selected rows and valid only for these rows local assignments must be used.
    3. Summation (concatenation...) fields must use global assignments.

     

    In case you do not like to use mixed types of assignments you can avoid the := operator by simply using global variables in all cases were the globality is needed.

     

    Oem, ok, but what are global variables?

     

    Ah, here they come:

     

    ~*/SearchResFlightRel{!1=0;!!SUM=0}$/FlightBookRel{!1=!1+1;!!SUM=!!SUM+(1000*@LUGGWEIGHT);!AVG=!!SUM%!1}
    Code Example 28, !1= or !!VAR=, aggregations with global variables

      

    This is basically the same example as number 21, solved with global variables, so the intermediate variables do not appear in the return structure.

     

    As learnt before, you may assign data to targets which are part of the returned data. Sometimes the user might want to calculate and store data which is not used immediately within the target structure. To do so the Global Variable Table was introduced which stores all variables. Technically it's a hash table with the variable name as key.

    In general the following rules apply:

     

    • All identifiers beginning with a number are treated as variable, e.g. !1 refers to a variable and also !0Var. !Var refers to a target (means an attribute in the returned structure). Anyway attribute names beginning with numbers are not allowed in data structures.
    • In case you want to use speaking names without numbers the following syntax may be used !!Variable. The second ! indicates that it refers to a variable. 
    • A variable is always global, hence it lives "forever". At least till the complete statement is executed.
    • All sub-queries share the same variable table.
    • A variable is initialized with the first usage on the left side of an assignment.

     

    In some cases the initialization of aggregation data is very hard or even impossible. In case a query starts from a collection the syntax does not allow any assignment on the highest level (which is the collection). This means, if the collection contains 3 items, any assignment is executed at least 3 times.
    For dereferenced summations it might be also impossible to know whether the target is available or must be instantiated. So we introduced a GET(x,y) method, which has the following semantics:

    If x can be evaluated x is returned. If the evaluation of x leads to the access of a not initialized variable / target or attribute (last one makes hardly sense, since it is known at design time) y is used to initialize the return.

    So a typical usage is !s=GET(!s,0)+1. Please note that the failed access is propagated, this means a statement like !s=GET(!s+1,1) would lead to the same result as the statement before. GET methods can be nested also.

    The following example illustrates a non-trivial usage of the GET function, in a scenario where it can not be avoided since the field is added dynamically.

     

    ~*/SearchResFlightRel$/FlightBookRel{!1=LEFT(@PASSNAME,1);!1=IFF(!1="","EMPTY",!1);!1^!:=GET(!1^!,0)+1}
    Code Example 29, GET, dynamic counters

     

    The code extracts the first character of the passenger name. The iff statement is required to avoid creating fields with empty names (leading to a short dump). Then we try to access the column with the corresponding name ( e.g. column A ). If it is available we increase its value by one, otherwise we create it newly.

     

    Before closing this lesson I have to make a small confession. Above example and also example 22 is only working after applying note 1586957 which solves several smaller problems I identified over the past days. The note also introduces the possibility to 'keep the global variable table alive'. This means you might use information stored by one BPath query at the execution of another BPath query.

    I will add a small chapter on the available notes in one of the coming lessons, currently 4 notes are recommended.

     

    That's it for today. Lesson 6 will bring sorting and unstructured returns. And some performance investigations in case I manage to do so.

    Hello,

     

    we are already at the fourth lesson of our course on BPath. With the third lesson (BPath. Lesson 03: Functions and Assignments) we entered the world of assignments, which we further discover with this contribution.

     

    Our Example Data Model

     

    Before we kneel ourselves even more into BPath examples a small picture showing the data structures we run our example BPath statements against. The examples are basically using the colorized objects and relations:

     

     

    image

     

     

    Enhanceable Structures, Aggregations, Dereferencation

     

    Within the last lessons you learned how to declare the return type of your BPath query and you also learned how to assign data to target fields. Now you might ask yourself what happens, when you use a target field which is actually not existing. Well, good question, easy answer: It is added. But only if you use enhanceable structures:

     

    ~*CRMS_BOL_UIF_TRAVEL_BOOK_KEY/SearchResFlightRel{!Newfield=Index()}/FlightBookRel/*$
    Code Example 18,~*STRUC, enhanceable structure

     

    Mind the star! If you place a star between tilde and structure name you are able to add artificial fields to the structure. The new field will be added as soon as it is used as assignment target somewhere in the BPATH. The type of the new field follows the type of the right side of the assignment. If the new field is used a second time the field will correspond to the already added field.

    In case you do the same coding against a non-enhanceable structure (by omitting the star), you will receive an error message.


    A special type of enhanceable structure is the empty strcuture:

     

    ~*/SearchResFlightRel{!Newfield=Index()}/FlightBookRel/$
    Code Example 19, ~*/, Empty Struc

     

    As long as there is no attribute added, the structure will exist only virtually. If there are attributes added, the structure is created from scratch.
    Be carefully with the * / move-corresponding operation in this respect, since a move-corresponding to an empty structure makes no sense.

     

    Coming to a different topic, accumulations and aggregations. You might argue that we have already built an aggregation within the last lesson. That's right, but there is one more point to capture:

     

    ~*/SearchResFlightRel{!VALUE=0}$/FlightBookRel{!VALUE:=!VALUE+1}/*
    Code Example 20, Rel$/SecondRel, Accumulations

     

    Did you notice the difference?

    Now the dollar appears no more at the end, but after the first relation. Why that? Well, the dollar indicates when an entry has to be written to our return table. And now we do not want to write an entry for every booking we find, just for every flight.

    Note, that all assignments are done on the way downward. This means the left assignment is done first and then the corresponding increment operations. But what, if we want to use the data coming from the subtree for calculations higher (e.g. calculating the average). Well this can not be done this way (actually I am still searching for a feasible way to make it possible and readable), but a solution is to do the calculation on the lower level and each calculation overwritten the former one, so the last one wins:

     

    ~*/SearchResFlightRel{!VALUE=0;!WEIGHT=0}$/FlightBookRel{!VALUE:=!VALUE+1;!WEIGHT:=!WEIGHT+(1000*@LUGGWEIGHT);!AVG=!WEIGHT%!VALUE}/*
    Code Example 21, aggregations

     

    In this example the test against division by zero can be omitted, since the calculation is only done, if value is at least 1, but in general this problem has to be kept in mind.

    Again note, that for all aggregation assignments := has to be used.

     

    Let us make BPath a bit more complicated and introduce dereferencation. This feature is mainly usable for transpositions, means exchanging columns and rows (storing row data in columns). So the feature might be pretty helpful if you want to export data Excel using a dynamic second dimension.

     

    ~*/SearchResFlightRel{!TEST="HALLO"}$/FlightBookRel{!CTX="PS"+INDEX();!CTX^!=@PASSNAME}/*
    Code Example 22,* !x^!=, dereferencation

     

    Here again the dollar moves to the front, to the SearchResFlightRel relation. Since otherwise the return structure is empty at this point, a test field is introduced. The second assignment block first fills the CTX field with a newly constructed field name as PS1, PS2, ... The second assignment takes the content of CTX dereferences it as target and fills it with the Passname of the current attribute structure. In our example one of the flights has about 450 bookings (probably a 747), so the fields PS1, PS2, ... PS450 would be added and filled to the return structure.

    It is also possible to use dereferenation within a expression (with the read part), and it is also possible to dereference attribute fields (!TEST^@).

    Performance has to be kept in mind here. First of all all new fields are added one by one. Secondly, as in our example where we return a table, the structure for row x (which has say 100 fields added) is not necessarily the same as for row x+1 (with 102 fields). Hence they can not be added to the same table. In fact a new table with the structure for row x+1 has to be constructed and all old entries have to be converted to this new structure. This might cause a performance problem in case the number of fields is constantly rising together with the row number.

    Note that dereferencation typically uses assignments with "=", otherwise the content might be not cleared for the next row.

    Object Retrieval

    In case you have taken a look at the BOL interface definitions you might have noticed that GET_PROPERTIES_BY_BPATH is not the only method which is using BPath. There is a method called GET_RELATED_ENTITIES_BY_BPATH which is obviously also using BPath. Signature is pretty much similar, but instead a data reference it returns a reference to an IF_BOL_ENTITY_COL entity. In other words it returns a Business Collection (actually it returns a CL_CRM_BOL_ENTITY_COL). The sense should be straight forward; instead of returning the actual data, the source objects are collected.

     

    Even though this functionality was there from the beginning, it was never enhanced as it counterpart GET_PROPERTIES_BY_BPATH. The basic functionality as filtering should work, some of the further enhancements as assignments have no expression on collections (as we can not dynamically construct objects). Some of the coming functionalities are also of no use when returning Business Collections.

     

    Note, that from a syntax perspective there is no difference between both calls. Functionality related to data (as assignments) would be executed also with the GET_PROPERTIES call. This means the data as such is assembled internally, but afterwards never used.

     

    For a demonstration of this functionality we use example 12 ( SearchResFlightRel/FlightBookRel[INDEX()<=(SIZE()%2)]/*$ ):

    DATA lv_bpath_result_col type  ref to IF_BOL_ENTITY_COL.
    lv_sourcecode = 
     'SearchResFlightRel/FlightBookRel[INDEX()<=(SIZE()%2)]/*$'.
    lv_bpath_result_col =  lv_result2->GET_RELATED_ENTITIES_BY_BPATH
    ( IV_BPATH_STATEMENT = lv_sourcecode ).

     

    The next lesson covers a lot of functions including string functions and it introduces the possibility to start sub-queries. We also introduce GET() which allows another interesting usage of dereferencing.

    Hello everybody,

     

    I hope everybody had a fabolous Easter time and we can concentrate with fresh energy on our path to BPath wisdom.

     

     

    Within the second lesson we introduced filters and structures (  ()), now we put the spotlight to two different topics:

     

     

    BPath also includes the concept of functions. Functions are provided by the system (they are hardcoded and it's not possible to dynamically add functions). A function can have 0 to n parameters. The concept supports optional parameters also. Some functions (e.g. concatenate which will be explained later) can have an arbritrary number of parameters. The types of the parameters are checked during execution.

     

    Let us begin with some function examples:

     

    The parameterless function Index() returns the index of the most inner object in the surrounding collection. Size() returns the size of the surrounding collection.

     

     

    SearchResFlightRel/FlightBookRel[INDEX()<=(SIZE()%2)]/$<br />+Code Example 12, INDEX() & SIZE()</p><p> +</p><p>Apparently the code examples returns the first half (rounded up) of the bookings of every flight. Note that % is used for division, as I did not want to overload the / symbol.</p><p> </p><p>Today() is also a function without parameters, returning the actual date. The first function with parameters is NOT(x). The usage with a boolean parameter should be straight forward. NOT is also defined for parameters of different type, the function returns true, exactly if the parameter is the NULL value of the respective type ( false, "", 0, #19000101# ). Note that date literals are encapsulated by #-symbols.</p><p> </p><p>SearchResFlightRel/FlightBookRel[Not(@ORDER_DATE>(Today()-400))]/$

    Code Example 13, Today() & Not()

     

     

    +* </p><p>The example returns all Bookings which are not done less then 400 days ago.</p><p>You like the questionmark operator within C+ ? Well, I do :-). So the next function to be discussed is iff(bool, var1, var2). var1 and var2 can be of any type, they even do not have to have the same type.</p><p> </p><p>SearchResFlightRel/FlightBookRel[@LUGGWEIGHT<IFF(UPPER(@INVOICE)="X",22,20)]/$

    Code Example 14, IFF(), & UPPER()

     

     

    +* +</p><p>The filtering is done against the weight of the luggage which has to be below 20 (kgs, in case nobody changed the base) for non-invoicers, and 22 for flyers marked as "x" for INVOICE. To be on the save side, a upper case transformation is done on the invoice field.</p><p> </p><p> </p><p>So, now we enter the world of assignments. Assignments are maybe the most important enhancement to the XPath-like access as they allow iterative thinking in an otherwise recursive environment. And they are the base of many of the features explained in the next lessons.</p><p>After any relation a "filter block" may appear, that was part of lesson 2. Now we introduce an assignment block which may appear at the same place, before or after a filter block. An assignment block is encapsulated with {} and it contains (an) assignment(s). </p><p><br />But maybe we begin with an example:</p><p> </p><p>~CRMT_NAME_VALUE_PAIR/SearchResFlightRel{!NAME=@CITYFROM}/FlightBookRel{!VALUE=@PASSNAME}/$

    Code Example 15,{!F=@V}, basic assignments

     

     

    Well, the concept of assignments allows to calculate the content of fields on every level. The target has to be prefixed with a "!" and corresponds to a field of the target structure (which has to be explicitely defined).

    Note that the move-corresponding initiated by the * at the end of the source code is in fact useless, since no fields are transported.

    The right side of the assignment works with the same evaluation evaluation engine as used with filter, so it is possible to state more complex assignments as the following:

     

     

    CRMT_NAME_VALUE_PAIR/SearchResFlightRel{!NAME=(@CITYFROM&"/")&@CITYTO;! VALUE=TODAY()-@FLDATE)<br />Code Example 16,{!F1=(@C&"/")&@D; !F2=@E}, complex assignments separated by semicolon</p><p> </p><p>As seen, two (or more) assginments are separatable using a semi colon. It is also possible to refer to the target itself in the assignment:</p><p> </p><p>CRMT_NAME_VALUE_PAIR/SearchResFlightRel{!NAME="NO Boardings"; !VALUE:=""&((0!VALUE)1))

    Code Example 17,{!F:=!F+1}, expressions on targets; automatic conversions

     

     

    The second assignment shows how an assignment can refer to its own target (or other fields of the target structure). How this can be used intelligently we show later.

    You might have noticed that this example uses a ':=' instead of '=' as in the examples before. Unfortunately there are two types of assignments in BPath. Within one of the next lessons I will explain the rationale behind together with some more possibilies. As for now just use ':=' (global assignment), in case you want to refer to the result later. And '=' otherwise (local assignment).

     

    Here it just does some kind of counting, !VALUE is increased by 1 with every entry. The statement here is a bit more complex then required since !VALUE is declared as string, where an operation as 1 obviously makes no sense. Typically all operations contain some automatic conversions, where possible. So if you add a string to a number "(@N@S)", the result will be a number where the string is interpreted as number. If you do it the other way round "(@S@N)", the number is converted to a string and concatenated to the string. This means that "0@S" and ""@N are nothing else but simple conversions. Hmmm, well. I have to admit ((0!VALUE)1) can be shortened to "(1!VALUE)".

     

    This example is a good entry point to the operation table, which was promised in our last lesson. Here it comes:

     

    !https://weblogs.sdn.sap.com/weblogs/images/49754/BPath_Operation_table._2.jpg|height=320|alt=image|width=527|src=https://weblogs.sdn.sap.com/weblogs/images/49754/BPath_Operation_table._2.jpg|border=0!</body>

    Welcome back to our course on BPath. As you may remember from the first lesson (BPath. Lesson 01: What is BPath?) we introduced BPath and we discovered the first easy BPath statements.

    Now we face the question, how our BPath statements are included into our ABAP environment. The following code example coding should illustrate the ways to access BOL with BPath.

    At first we need a Business Collection where we start from:

    DATA lv_compset type CRMT_GENIL_APPL.
    DATA lv_bol_core TYPE REF TO CL_CRM_BOL_CORE.
    DATA lv_dyn_query TYPE REF TO CL_CRM_BOL_DQUERY_SERVICE.
    DATA lv_result type ref to IF_BOL_ENTITY_COL.



    lv_compset = 'UIF'.
    lv_bol_core = cl_crm_bol_core=>get_instance( ).
    lv_bol_core->start_up( lv_compset ).
    lv_dyn_query = cl_crm_bol_dquery_service=>get_instance(
     'UIFAdvSearchFlight' ).
    lv_dyn_query->set_property(
    iv_attr_name = 'MAX_HITS' iv_value = '5' ).
    lv_result = lv_dyn_query->get_query_result( ).

    At the end of this code fragment, we have a Business Collection which should contain 5 query result objects. Unfortunately BPath does not work on Business Collection directly, so we have to convert it to a Business Collection of uniform type first: 

    DATA lv_result2 type ref to CL_CRM_BOL_ENTITY_COL_UT. lv_result2 ?= CL_CRM_BOL_ENTITY_COL_UT=>Convert( lv_result ).

    Above statement constructs the required Business Collection with uniform type. To fetch a data result with a BPATH query from it, the following syntax has to be used:

    Data lv_bpath_result type ref to data.
    Data lv_sourcecode type string.
    lv_sourcecode = 'SearchResFlightRel/FlightBookRel/*$'.
    lv_bpath_result = lv_result2->GET_PROPERTIES_BY_BPATH(
      IV_BPATH_STATEMENT = lv_sourcecode ).

    If you want to execute the BPath query on a single object, the syntax is similar:

    DATA: lv_element type ref to CL_CRM_BOL_ENTITY.
    lv_element = lv_result->GET_LAST( ).
    lv_bpath_result = lv_element->GET_PROPERTIES_BY_BPATH(
      IV_BPATH_STATEMENT = lv_sourcecode  ).

     

    Before we get back to some more examples, some words on the Interpreter. BPath statements are directly translated into action (so it's no compiled language). At the very beginning, it was implemented as top-down parser using recursive descent.  But after the first enhancements, it turned out that this approach is hardly maintainable and not extendable. So the parser was migrated to a stack machine.

    The parser is implemented in the CL_WCF_BPATH_PARSER class. At first the scanner (method SCAN_NEXT) comes into picture which scans the input token by token. The scanner works context free, in means the result only depends on the currently processed characters and nothing else.

    The parser itself works in two passes. The first pass parses only the syntax without knowing anything about the semantics (it even does not know whether a specified relation or attribute name is valid), the second pass does the action. The parser uses two control tables which are built up at the beginning. The first holds all identifiers, identifiers are a level above tokens, not completely matching. The second table holds the possible status transistions controlling the parsing. In theory all parsing is done table controlled and only the action is done "coded". But I guess in practise things are not that clean, but I haven't checked now.

    The parsing includes a lot of going forward and backward as for each entity the corresponding BPath fragment has to be evaluated again. To avoid too much operations for scanning, the first pass stores the intermediate results in a table also. There are mechanisms to avoid the assembly of the internal tables in case the data is already available.

    As the recursive information is no more hold implecitely within the calling hierarchy, it has to be held explicitely. To do so a new stack class called CL_WCF_BPATH_STACK is used, implementing the stack with some pretty uncommon inventions. BPath actually uses quite a lot of stacks during processing.

    The language itself does not differentiate between upper and lower case except where the underlying layer requires it (e.g. BOL is using case sensitive relation names). All elements must succeed directly after each other, it is not allowed to use blanks or carriage returns for separation.

     

     

    ~CRMS_BOL_UIF_TRAVEL_BOOK_KEY/SearchResFlightRel/FlightBookRel/*$
    Code Example 7, ~STRUC, normal structures

    In the examples of the last lesson the structure of the returned entity is set implicitly. It refers simply to the attribute structure of the corresponding object. Sometimes this is not required or wanted. So it is possible to set the structure explicitly as above.


    The example uses the key structure which consists of 4 fields. The data is always filled with a move-corresponding statement (also when structures match). It is responsibility of the caller that the structures match, i.e. that the move-corresponding does not try to copy data to fields with the same name but of different type.
    It is recommended to place the structure definition directly at the beginning, even though the syntax allows otherwise. With the first access to the structure, it has to be defined of course.

     

    SearchResFlightRel/FlightBookRel[2]/*$
    Code Example 8, [2] (basic filtering)

    The [ ] clasps the filtering. The further evaluation is only done if the filtering can be done successfully. In our example we exclude all entries which are not the second entry within the FlightBookRel collection. 
    The filtering always relates to the most current relation. If we use the above source code on a collection the statement would return the second booking for every flight of the collection.
    You may filter on every relation you specify. Syntactically it is correct to put several filters on one relation, eventough the result will be the same as if you use a corresponding conditional filter.

     

    SearchResFlightRel/FlightBookRel[@CLASS="C"]/*$
    Code Example 9, [@CLASS="C"] basic conditional filtering, string literals

     

    There are two ways to filter, the first is operation on the index as in the previous chapter, the second is using boolean operations. If the statement is evaluated to true we proceed. Otherwise we check the next entry.
    Please note, that string laterals are built using double quotes. It is possible to mask a double quote by stating two double quotes after each other.

    Of course, the filtering is not restricted to simple comparisons with the equal operator. Let us check the next example:

     

    SearchResFlightRel/FlightBookRel[(@CLASS<>"C")&(@FORCURKEY=@LOCCURKEY)]/*$
    Code Example 10, [(@CLASS<>"C") &(@ FORCURKEY=@LOCCURKEY)], Comparisons and Bool-OPs

    As you see it is also possible to use other comparison operators ( more exactly <, >, <=, >=, <>, = ). And it is possible to use the boolean functions and (with the & symbol) and or (using | ).

     

    SearchResFlightRel/FlightBookRel[(@SMOKER+@INVOICE)="X"]/*$
    Code Example 11, [(@SMOKER+@INVOICE)="X"], basic arithmetic operations

    This example uses arithmetic operators, in this case as string concatenation operator. Both fields used are either "X" or empty so the expression basically evaluates to true if exactly one of the flags is set.

    Please note that all operations are only working with 2 operands. An 'or' with three operands has to be expressed using appropriate brackets.

    Beside above noted boolean and comparison operators the standard operators + (addition), - (substraction), * (multiplication) and % (division) are supported. The full operation table will be provided in the next lesson.

    In general BPath operates with 4 data types: Strings, Dates, Numerics and Booleans. Within an intermediate value of an expression any of the above types may be used, but on the highest level of a filter it has to be either a boolean or a numeric value. For a numeric value an implicit check against the index is done. Note that negative value may be used also, as -1 represents the last entity of the collection (-2 consequently the second last, and so on).

     

    With the next lesson we do something completely different. Assignments. 

     

    To answer the question in the header at the very beginning, here comes a quote, raised some time ago after the first part of of development was done:

     

    BPath is a new way to access BOL objects, which allows data / object access controlled by a dynamic, XPATH-like character string.

    The initial request is very simple. Dynamic access to SAPs BOL/Genil Layer. Something that evaluates requests such as
    "../AddressRelation[@MAIN="X"]/EmailAddressRelation[@BUSINESS="X"]/@EMAILADR".
    Something like XPath for XML data. Some of the concepts were taken over, some didn't fit, and somehow the topic got its own life.

     

    Well, these statements are still valid, with two remarks: The "own life" expressed in the statement got us already pretty far away from the origins. And with the latest enhancements it is possible to attach BPath to other frameworks also, not only to BOL. But since we should begin at the beginning and anyway only one other "driver" (which allows access to internal tables) is developed as of now, above "definition" remains to be fine for the moment.

     

    My name is Jürgen Gatter, at the time of development I was part of the CRM UI Framework team (which turned into a TIP Core UI framwork team nowadays). With this blog here I want to showcase the feasibilities of an interpreted computer language developed over the past years within our team, which is called BPath and is available with WEBCUIF 7.0 EhP1. More exactly it is the starting point to a blog series covering the features of BPath, as for today we only discuss the fundamentals and ideas behind.

    The beginning of the development was nearly as simple as expressed in the quote above. BOL should be enriched with a feature allowing to access the data in an XPath-like manner returning either a Business Collection or the data itself. So let us first take a look on the question to which extend we can compare XML data and Business objects:

    • XML data is always organized as a hierarchical, finite tree. This is not the case for Business Data.
      Even though business data may appear to be hierarchical organized (on storing a Business Object one typically saves an hierarchically organized sub-tree), in general it is not. Relations can occur from any node to any other node and infinite pathes can be found without problem. So we have to conclude that Business data is neither hierarchical nor finite. Consequently it is not possible to apply some of the features of XPATH (as the searches on all nodes of the subtree).
    • The start object for the Bpath has to be provided always (equally the XML document has to be provided for XPath). Since we always operate from the starting object the differentiation between relative and absolute pathes makes hardly sense.
    • Since hierarchy and finiteness are not guaranteed even for a (business) root object, there is no way to "scan" the complete tree. As a consequence the "//" operation as known in XPath (I cite from w3schools: "Selects nodes in the document from the current node that match the selection no matter where they are") has no counterpart in Bpath.
    • The data returned by the XPath is also an XML entity and can be structured very inhomogeneous. The return of BPATH is either a flat collection or an data structure. In both cases there are limitations in respect to the inhomogeneity we can handle. For example it is practically not possible to deal with requests as //book | //cd
    • Some concepts as "Parent" and the prefixing of attributes with @ and the notation in respect to relations are quite similar in both concepts. Note that in XML the names refer to the objects, whereas the names used in BPATH refer to the relations between the objects.

    Even though the comparison with XPath is very good to illustrate the startup, we will leave these grounds already in lesson 2 or 3. So I will no more point to similarities and differences with XPath from here.

     

    Enuff theory, let us try out some BPath statements, even though you still do not know how to apply them - that's one of the topics of the next lesson. Just some remarks in advance to sketch the environment:

    • BPath is available with WEBCUIF 7.0 EhP1, shipped with CRM 7.0 EhP1 (and to my knowledge with the SAP Business Suite 7 Innovation 2010)
    • Base of a query is either a Business Object or a "Business Collection with Uniform Entity Type" (CL_CRM_BOL_ENTITY_COL_UT)
    • for all examples we use the standard "FLIGHT" model, more exactly we use an UIFSearchResFlight query result object or collection as base.
    • If a query execution runs into a problem, a shortdump will be raised. This might be either the dump from the underlying layers or a CX_WCF_BPATH_PARSING_ERROR.

    So, here comes the examples:

     

    *
    Code Example 1: *, fetching the attribute structure 

    The "shortest BPath statement which does something". Well ok, it is not the shortest BPath statement of all, since the emtpy statement is also a valid statement. But - surprise - the empty statement does nothing.

    The '*' is a bit overloaded in BPath, it is used as multiply operator, and as we have it here, as "get it all"-operator. And 'all' means the complete attribute structure. If your statement was fired from a Business Object, the query returns the attribute structure from the respective object. If it was executed from a Business Collection, it returns the attribute structure of the first found object in the collection.

     

    @CITYTO
    Code Example 2: @attr, fetching an attribute 

    CITYTO is one of the attributes in the attribute structure, and exactly this attribute is returned (correctly typed) from the first object. As you might notice, the usage of the @-symbol is similar to XPath.

     

    ./@CARRID
    Code Example 3, ./ , relation to myself

    We are nearing ourselves to a different topic, the usage of pathes. The first is the path to the object itself, doing not much, but making the code slightly more readable.

     

    SearchResFlightRel/FlightBookRel/*
    Code Example 4, Relations

    This examples uses two relations to reach the object we finally retrieve all attributes from. The first relation brings us from the Query Result Object to the root object, whereas the second relation brings us to the UIFBook object dealing with the bookings. Note that FlightBookRel is a 1:n relation, so even if we are starting from a business object, the entity which we are working on is now a collection. But still, only the first entity is returned.

    As in XPath the slash is used as separator. Unfortunately this turned out to be a problem in our case, since the slash might be used within relation or attribute names also. This was detected quite late (otherwise I would have chosen the backslash instead). In case your names contain slashes, these slashes have to be masked with a prceeding backslash and note 1535509 has to be applied. See for more information there.

     

    ../*
    Code Example 5, .. (Parent relation)

    Just for completeness reasons I mention here the parent relation which is defined for every access/dependent object (root objects don't have parents) and is represented by two dots following standard patterns. Technically there is another small difference to other relations since the parent relation returns the parent object, whereas normal relations return a collection. As for BPATH users this difference is hidden.

    As for query result objects the parent is not defined also, above query would result in a parser error when executed on a UIFSearchResFlight object complaining that the target of the parent relation is either not existing or not ambigous. The second case will be treated later.

    With the first examples we always returned a single element. But what if you wan to assemble the complete information? This is shown in the next example:

     

    SearchResFlightRel/FlightBookRel/*$
    Code Example 6, $ (add to table)

    The dollar sign makes the difference. Up to now, the evaluation was stopped after the first entry found and the corresponding attribute or structure was returned. Now a table (of structures or of the attribute) is returned. This is also the case if only one entry is found.
    The return structure is built up during procession, this can not be done if no data is found. In general applies to all BPath queries: If no data is found, NULL is returned.

    The dollar may appear after any relation, but as it is used here we suggest to add it at the end. The differences between the approaches will be explained later.

     

     

    That's it for today.

    With the next lesson we cover the proper integration (including ABAP code) and we do further BPath examples highlighting structures and filtering.

    Filter Blog

    By date: