Modes of Data Processing in DTP

[Former Member]

In my last blog-post I covered Package Size & Impacts of Package Size in DTP. Continuing the same trend of DTP, I am about to write on a topic discussed on SCN forum on & off. I intent to write this blog-post because I myself struggled with knowing the exact way of how the processing of data with allocated processors happens inside a DTP & which is the best available mode. After deep understanding, I thought of writing a blog-post on this very interesting topic. Let's start.

To start with, there are 3 modes of data processing available in DTP which are listed as below:-

1. Parallel Extraction & Processing

2. Serial Extraction & Immediate Parallel Processing

3. Serial Extraction & Serial Processing

Let us get into details one-by-one.

1. Parallel Extraction & Processing :

As the mode-name suggests, it actually extracts the data & process the data parallel i.e. simultaneously. In this mode, all available processes extract the data parallel & as soon as extraction is completed the data processing starts. Here data processing is updating the data to the target. In this mode, there is a possibility that one process might be updating the data packet to the target, while others might still be extracting. Hence, the work is done parallel.

SAP also refers this processing mode as P1.

Example: If we assign 6 parallel processes then each process extracts one data packet at a time & updates the same data packet to the target. Hence, in this case 6 data packets will be processed simultaneously & updated to target accordingly. This process continues till the end of all packets available.

2. Serial Extraction & Immediate Parallel Processing :

When we select this mode for data processing, the process extracts the data from source & after complete extraction, the data packet is processed by another process. In the meantime the 1st process again starts extracting another package. And the cycle continues.

Now, here if the 1st processor has not completed extracting the 1st package yet then a new process will start extracting depending upon the number of parallel processes assigned/allocated. Hence, as the name suggests it serially extracts the data from source & as soon as extraction is finished the packet is processed & updated to target.

SAP also refers this processing mode as P2.

Example: If we have 6 processes assigned for this mode then one process will do the extraction of packets one by one & other processes will do the processing & updating to target parallel. As explained above if 1st process is still extracting then 2nd process will jump in for extraction of next packet. This procedure takes place serial extraction & is further processed parallel.

3. Serial Extraction & Serial Processing :

In this mode only one processor firstly extracts the data packet from source & then the same processor will update the data packet to the target without involvement of any other processor. Once the data packet extraction from source & update to target is finished, the processor moves to next data packet.

This processing mode is rarely used because of performance optimization issues & is not recommended generally.

SAP also refers this processing mode as P3.

Example: As we cannot allocate or define maximum number of background processes for this mode it is obvious that one processor will extract & update while other will do other jobs. 

Summary & Performance Comparison:

Although the selection of Processing Mode in DTP depends upon the combination of Data Source, Transformations/ Mapping & Data Target the selection of processing mode should be smart enough to optimize the performance of DTP. Based upon above information, I can state that P1 has better performance over P2 & P3 whereas P3 has the lowest performance over P1 & P2.

Hope this blog helps you in understanding how the processing of data from extracting data from source to processing it to updating it to data target is done in DTP by the  processing types.

Author: Shubham Vyas

Company: Mindtree Ltd. (Bangalore/India)

Created on: 27 June 2015