This web log continues the ideas developed in the web log Performance Problems Caused by Nonlinear Coding. There, it was stated that a good program should scale linearly, which means that the processing of an order, for example, with 500 positions should maximally need 10 times longer as a similar order with 50 positions. The web log explained that, in practice, linearity is not automatically given, as small programming bugs are easily overlooked in tests with 1 or 5 positions. It was outlined how a practical test to identify nonlinear coding could work. Such a test is based on two detailed runtime measurements with different amounts of data which would uncover the hidden nonlinearity. The total amounts of example data can be rather small.
This web log turns theory into practice. A report provides the comparison of two measurements:
First, decide what you want to trace. Your test case must be bug free, all the customizing settings should be chosen the way a customer would use the program. The database tables must contain all the necessary data. The test case should reflect a typical customer use case.
For the nonlinearity check, the test case must be executed with two different amounts of data, i.e. items, positions or lines. It is necessary that there is a substantial increase from N1 to N2, such that x = N2/N1 should be 5 or, even better, 10. Also, the smaller measurement should not be N1 = 1, because this is very special. N1 should be at least 5 but larger is better.
Details on how to prepare the trace files can be found in A Tool to Compare Runtime Measurements: Z_SE30_COMPARE. Try combinations (N1, N2) between (5, 50) and (20,100) first and use larger combinations later, when you are sure that there are no very serious nonlinearities or after these have already been fixed.
For each line in trace 1 the comparison report tries to find a line with identical keys in trace 2. The key fields are call, program and line. It is important to split the call into two parts:, in type of call and name of call. For example, ‘Call Method XYZ’ is split into ‘Call Method’ and ‘XYZ’, and ‘Read table IT_123’ into ‘Read Table’ and ‘IT_123’ etc. This is important because call type and call name play quite different roles in the comparison task. The call type is also classified into call classes, which results in another but dependent key field. So the five fields call class, call type, call name, calling program and calling position, are the key fields to identify one line uniquely.
The comparison might appear to be quite simple. However, the following problems make the task more complicated:
Fortunately, there is some redundancy in the five key fields, so that two compare modes are possible which work with different subsets of the five key fields:
Before the compare is executed, the program takes care that the key fields are unique. If the calling position is used, then internal table calls are aggregated, because internal tables locally defined in a subroutine get new names in each execution of the subroutine and are not aggregated by the ABAP Runtime Analysis (SE30) in older releases. This aggregation is actually an advantage of the compare program, and therefore this feature is also implemented in newer releases of the SE30. The compare using the call name aggregates all different calling positions, which leads to the same result as the full aggregation of the SE30.
Once the key fields are identified, the values of the net time, executions and gross time of the two measurements can be compared.
The results are displayed in an ALV table:
Now, how does this information help to identify nonlinear coding? As explained in Performance Problems Caused by Nonlinear Coding, we can expand the total runtime as a sum of different scaling behaviors
T(N) = A * 1 + B * logN + C * N + D * N * logN + E * N * N + ...
But we can assume that the individual calls measured by the ABAP trace are so fine granular that they show no combinations of scalings but only one pure scaling! These can be constant, linear, quadratic or even more.
Constant : Net(N) = a => Net2/1 = a/a = 1
Linear : Net(N) = c*N => Net2/1 = c*N2/(c*N1) = x
Quadratic : Net(N) = e*N*N => Net2/1 = e*N2*N2/(e*N1*N1) = x*x
where x is the ratio between the two amounts of data x = N2/N1.
The logarithms should not be handled as separate scalings but together with the other next lower natural scaling. The logarithm and the variability of real-life measurements propose the following classification intervals for the measured values of Net2/1:
Reduced : Scale = R => Net2/1<0.5
Constant : Scale = C Net(N) = a => Net2/1 = 0.5 ... 2
Linear : Scale = L Net(N) = c*N => Net2/1 = 2 ... 2x
Quadratic : Scale = N Net(N) = e*N*N => Net2/1 = 2x ... 2x*x
Cubic : Scale = N Net(N) = g*N*N*N => Net2/1 = 2x*x ... 2x*x*x
Order 4 : Scale = N Net(N) = i*N*N*N*N => Net2/1 = 2x*x*x ...
The factor 2 comes from log(100) / log(10) =2 and ‘Scale’ is the column in the result screen.
We should mention that strictly speaking, functions, forms and methods are still combinations of different scalings, especially if they contain many operations on internal tables. But the method works quite well, since even a very complicated function will contain only 10 or 20 factors while a whole program contains several thousand factors. A nonlinear factor will, therefore, dominate the scaling of the function very easily. However, best results are achieved if internal table operations are also traced.
As the net time in the SE30 hit list is the total time for all executions, it is the product of the numbers of executions and the time for one execution. This also means that the scaling of the net time depends on the scaling of the executions and the scaling of the time per execution. For example, a net time with a quadratic scaling can be caused by a quadratic increase in the time per execution or by a quadratic increase of the execution numbers.
Both possibilities appear in practice and require a different analysis of the cause for the nonlinearity:
The report result identifies the scaling of the net time but also the scaling of the executions. Therefore, the ratios of executions are classified in the same way as the net time. Then we can write the scaling exponents x, y and z of net time, executions and time per execution as
Net x = Exec y + Time/Exec z with x, y, z = 0,1,2 3, or 4
The time per execution is not an explicitly measured value, it is only calculated. It is not displayed by the report. As it supports the line of argumentation and I have sometimes added in the text in brackets.
Displayed is the combined field
Net x - Exec y with x, y = 0,1,2,3, or 4.
Only combinations (x, y), where x is not smaller than y, make sense, because the scaling of the net time can not be smaller than the scaling of the executions. The scaling of ‘Time per execution’ is given if you take the hyphen as a Minus-sign, i.e. Net x Exec y = Time/Exec z.
The interesting combinations are the following ones:
The green combinations are unproblematic.
The quadratic contribution ‘Net 2’ can have three causes:
Cubic and order 4 scalings can create even more combinations. Then, the analysis is done in the same way. You must try to reduce the scaling order step-by-step either by changing the executions or the cost per execution.
First you should decide whether to include internal tables in your trace. It is especially useful if your program contains large functions with numerous operations on internal tables, because then it can be cumbersome to figure out which operation causes the problem. If the program code has a very fine modularization, where each method contains only one or two internal tables, then it is not necessary to trace internal tables.
Then you should check whether there are quadratic executions in the trace. It is not necessary to start with the cubic contributions as they are often a side-product of quadratic executions. If there are quadratic executions then sort the compare result by the gross time of the larger trace, i.e. by gross2. Scan the list from the top for the first appearance of a quadratic execution either by checking the column ‘Nonlinear’ or the column ‘Exec 2/1’. For this line, read the source code. Double-click on the line, if the calling position was downloaded, otherwise search the line in the original trace and view the source code there. Try to figure out where the quadratic executions come from and how they can be avoided.
From the source code you can also determine the call stack from top to bottom. Just go into the source and drill down to the calls inside. All lower levels will have also quadratic increases of executions or even more. Go back into the result list and scan for the next quadratic executions; check whether there are calls which are not in the already known call stack.
Also, check the major quadratic times per execution. To do this, sort the list by net2 and scan for the largest quadratic net time contributions, which are not caused by quadratic executions. When you find a line, then review the source code.
Nonlinear coding is very often connected with suboptimal nested loops on internal tables. Often programmers are not aware that the following operations on internal tables scale linearly with the size of the internal table and are therefore not suited for use inside a loop on another internal table:
Read also the web log Runtimes of Reads and Loops on Internal Tables.
When all identified nonlinearities are corrected, then I would recommend you to run the test case again with a larger object size (N1, N2) between (50, 500) and (100, 1000). You can only be confident, that the program scales linearly even to very large amounts of data, if no nonlinearities appear in such a large test case.
More information on performance topics can be found in my new textbook on performance (published Nov 2009). However please note, that it is right now only available in German.
Chapter Overview:
In the book you will find detailed descriptions of all relevant performance tools. An introduction to database processing, indexes, optimizers etc. is also given. Many database statements are discussed and different alternatives are compared. The resulting recommendations are supported by ABAP test programs which you can download from the publishers webpage (see below). The importance of the buffers in the SAP system are discussed in chaptr five. Of all ABAP statements mainly the usage of internal tables is important for a good performance. With all the presented knowledge you will able to analyse your programs and optimize them. The performance implications of further topics, such as modularisation, workprocesses, remote function calls (RFCs), locks & enqueues, update tasks and prallelization are explained in the eight chapter.
Even more information - including the test programs - can be found on the webpage of the publisher.
I would recommend you especially the examples for the different database statements. The file with the test program (K4a) and necessary overview with the input numbers (K4b) can even be used, if you do not speak German!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
4 | |
3 | |
3 | |
2 | |
2 | |
2 | |
1 | |
1 | |
1 | |
1 |