March 9, 2012
The activation of SAP in-memory optimized Data Store objects (DSOs) may be controlled by an administrator to limit overall memory consumption and/or manage CPU resources to optimize system performance. SAP has provided access to system parameters collectively called Memory Consumption Reduced (MCR) that allows the fine tuning of the system during DSO activation.
By design, all DSOs are subject to the MCR’s initial parameters settings (“_DEFAULT”) and as a result there is no need for any modifications of these parameters unless standard DSO activations lead to errors due to lack of memory resources and/or exceed time estimates due to too few parallel processing threads be created to process the load. Fortunately, standard SAP functionality provides access to define specific system settings to achieve the desired performance.
Specific MCR settings can be made for a specific DSO object via the transaction code SM30 and the subsequent modification of table view V_RSODSOIMOSET. Separate table entries may be created with unique system parameter settings that will be invoked for a specified DSO. In the table below, the system delivered default “_DEFAULT” is listed along with specific entries for DSOs HSD_O01B, HSD_O01C, HSD_O01D, and HSD_O01E:
Additional entries may be added by entering the table maintenance mode and selecting the “New Entries” option:
When creating a new entry, the user will specify the technical name of the target DSO, the package size to be processed, and the number of packages to be processed in parallel. A check box is provided to instruct the system not to apply any MCR processing to the specified DSO.
Example of a completed parameter entry for DSO HSD_O01E specifying 200,000 records per package and 6 concurrent execution threads:
Test Sequence:
A series of tests were conducted to illustrate some typical results for various MCR settings. These tests were conducted on a BW 7.3 SP07 system connected to SAP HANA appliance 1.0 with approximately 30 million records being processed. :
BW memory usage was determine by monitoring overall database memory consumption and time using the transaction code STAD. Filter on the program "RSODSACT1" and (optionally) the user id executing the data activation:
Test Procedure:
Table V_RSODSOIMOSET was modified for each test, changing the package size and/or number of package parameters.
The specified DSO was loaded with approximately 30 million records then the data was activated. Database memory consumption and time was monitored using the transaction code STAD. Filter on the program "RSODSACT1" and the user id executing the data activation.
The number of activation processing streams is directly related to the "Number of Packages" parameter when using MCR. To view the generated multiple process streams enter the HANA system and select the top system identification node (the node in the Navigator system hierarchy directly above your CONTENT and CATALOG folders) by double click it. This will open the system overview information screens. Click on the Performance tab and sort on the column "Application User". The id of the user executing the activation process can now be monitored to view the generated multiple process threads. Note: if you set the "Number of Packages" parameter to 10, you can expect to see 10+1 process threads in the Performance display. The extra thread allows for a communication link back to BW. As each process thread completes the data waiting for activation is eventually consumed. As a result the number of threads will tend to decrease as the process is executed.
Example Data:
Test Number: DEFAULT
MCR Id: _DEFAULT
Package size: 100,000 (system default)
Number of Packages: 8 (system default)
DSO loaded and waiting for data activation:
HANA Administration view prior to starting activation:
HANA Administration view after starting activation (note the 8 new ActivationQueuePackage threads generated:
Transaction code STAD in BW system to monitor memory consumption:
BW log for the DSO activation, used to determine overall activation time:
Test “E” No MCR:
A trial run was executed where MCR was turned off for the specified DSO:
An out of memory condition on the BW (ABAP) side was created. The error appeared in the DSO activation log :
Error log text:
The STAD trace shows only the initialization of the activation process:
TEST F:
This test tried to push the limits of processing capability of the HANA Db by forcing very large package sizes to be processed. The net result was many additional HANA processing threads being created (which appear to be for master data lookups) that eventually overloaded the system (I counted over 1100 process threads being created that eventually overwhelmed the system and created an out of memory condition on the HANA side):
Over 1100 processing threads were created….resulting in an out of memory condition on the HANA side:
Test Summary/Conclusions:
Conclusions:
1. Test E (No MCR) illustrates the need for implementing MCR with large data to prevent BW overload
2. Comparing Tests DEFAULT and A: decreasing package size increases activation time
3. Comparing Tests DEFAULT and B: assigning fewer processors will increase activation time.
4. Comparing Tests DEFAULT and 😧 increasing package size reduces activation time but may increase memory consumption.
5. Test F (very large package size) shows there are limits to how much data the HANA system can process without exceeding memory constraints.
Additional material on this topic:
Note 1646723: “BW on SAP HANA DB: SAP HANA-opt. DSO Activation Parameters”
All readers are encouraged to ask questions or express their own views on this topic.
- SAP Managed Tags:
1 Comment
