Step-by-Step Guide in Processing High-Volume Messages Using PI 7.1's Message Packaging

[Former Member]

Introduction:

SAP NetWeaver Process Integration is continuously improving its capability to handle high-volume message processing. Message packaging is one of the new features to accomplish this. When handling high-volume asynchronous messages, message packaging can improve performance by combining multiple messages as one package.

Processing multiple messages as one package optimizes overall performance in PI by reducing load times, database processing and context changes between the different pipeline steps. All of these are done without effecting the design, development or monitoring of the messages.

Note: The information provided here is based PI 7.1 SPS4.

Prerequisites:

PI 7.0 SPS13 or PI 7.1.

To enable packaging in your PI systems, the patch level must be at least PI 7.0 SP12. Otherwise the systems can only process single messages. Non-central Adapter Engines, in particular, must be at this level. If not, you must disable sending of packages to these Adapter Engines.

Runtime Processing and Feature Overview:

As asynchronous messages arrive from a sender, they are stored in a queue. In the past, the qRFC scheduler took a single message from the queue and processed it. The qRFC scheduler is now able to process a set of messages (a package) in one LUW. Depending on the number and size of the messages in the queue, this procedure improves performance considerably.

The performance gain are from:

• Several Messages being processed in one service call.
• Pipeline program context is loaded once and several messages are processed within that context.
• Enable mass operations on database, where reads, writes and commits are done on blocks of messages instead of on an individual messages.

In addition, there is no impact on the monitoring and applications level. Messages are still atomic, just the processing is improved. There is no impact on database, error handling, application or even qRFC queues. Therefore, monitoring is also not impacted.

Example Scenario:

We have a continuous flow of messages sent to the Integration Server. Every second a message will arrive. To optimize resources, instead of processing each message individually, we want to package them up into 30 messages per package. In order to do so, we will wait up to 60 seconds for the messages to build up in the queue.

Note: This is just an example, not a recommendation on how a packaging should be configured for such a scenario.

Configuration Steps:

The basic steps are as follow: (I will go into each with more details.)

1. Enable packaging for the integration server.
2. Create "Package Configuration Type".
3. Create "Sender/Receiver Definition".
4. Create "Specific Package Configuration", using the informaton from the above 2 steps.

1. Enable packaging for the integration server:

Use tx: SXMB_ADM -> Change Configuration (F9), create a new entry:

Category: RUNTIME
Parameters: PACKAGING
Current Value: 1 (for active)

As you can see, the PI 7.1 system I am using, the default setting is "active" or "1". For PI 7.0, the default is "inactive" or "0".

Packaging is turned on by default on all PI 7.1 systems. There is no reason not to use it, since it does not effect the processing or monitoring of messages. It can only improve performance.

2. Create "Package Configuration Type:

Use tx: SXMS_BCM -> New Entries (F5), create a new entry:

Configuration Type: any name (in this example, the name is PACKDEMO)
Description: any text to describe the configuration type
Wait Time: the time in seconds to wait if the queue does not contain the number of messages to be packaged as configured in "Number" (in this example, the wait time is 60 seconds)
Number: the number of messages per package (in this example, we want each package to include a maximum of 30 messages)
Maximum Package Size: the maximum size of the package (in this example, we do not want the package size to exceed 2MB)

With the above configuration the packaging process is as follow:

1. The package will retrieve messages from the queue, up to 30 messages. But, if there are less than 30 messages, then a timeout of 60 seconds will occur. If there are 30 or more messages, then the package containing 30 messages will be created and processed. Another package will be create to contain the remainder for the messages, at 30 messages per package, until the queue containing less than 30 has been packaged, at which point, it will go into timeout.
2. If a timeout occurred and expired, the package containing however number of messages will be processed. Another package will be created from the messages in the queue, based on the rule in the previous step.
3. During the creation of the package, if the max package size is reach, the package will be released and processed, even if the number of messages/package has not been reached.

If you notice the DEFAULT configuration type in the above screeshot, the "Wait Time" is set to "0" or none. This means that messages will be collected into a package from the queue, up to 100 messages, and will be released immediately with no timeout, even if the 100 messages/package has not been reached.

3. Create "Sender/Receiver Definition:

We will now select the messages for a specific interface which we want to package. In this example we want to package messages based on sender information.

Use tx: SXMSIF -> New Entries (F5), create a new entry:

Sender/Receiver ID: any name (in this example, the name is PACKAGE_DEMO)
Description: any text
Service: Business Service used in the Integration Directory's Receiver Determination (in this example, the business service is BLService)
Interface Name: Interface Name used in Receiver Determination (in this example, the interface name is TestValidSource_Async_OUT)
Interface Namespace: Interface Namespace used in the Receiver Determination (in this example, the interface namespace is http://test.com)

4. Create "Specific Package Configuration"

Now, we will put the information from the above 2 configurations into a packaging scenario: which sender and sender interface will be effected; and, how will the messages be packaged and processed.

Use tx: SXMS_BCONF

Then,

1. Highlight the row "Inbound Processing from ..."
2. Double-click on "Specific Package Configuration"

In the subsequent screen, click to "Change" mode:

Next, click on "New Entries" or F5:

Using the the drop-down selection, populate the values for "Possible Application", "Sender/Receiver ID", and "Configuration Type":

Finally, save the configuration. You will be requested for a transport change list.

Testing:

A file sender adapter is used for sending the messages. In the communication channel, the polling period is set to 1 second.

When the file sender communication channel is activated, we will see the following in tx: SXI_MONITOR:

As you can see, for the 1st minute, all the messages are waiting to be processed. After 60 seconds, the packages will be created and processed. There is no change to the monitoring of each of the individual messages in SXI_MONITOR.

Package Monitoring

To monitor message packaging, we must turn it on first, using tx: XMSPKSTATCNF

1. Click on "Switch On".
2. Click on the green check button to enter it.

Then, bring up the monitor by using tx: XMSPKSTATMON

When executed, we get the following screen:

Double-click on the row for more detailed view:

Also, click on "Display Events (F6)" for more packaging information:

Packaging Info:

In Summary, using Message Packaging:

• Increases throughput for asynchronous messages on the Integration Server.
• Packaging is switched on globally, but can also be configured based on specific scenario.
• Packaging monitors are available.
• No changes in monitoring, error handling, and development.