Which Parallel processing is faster?

[Former Member]

There are various ways in which parallel processing can be implemented e.g.

1. Call function STARTING NEW TASK which uses dialog work process.

2. Using background RFC (trfc) .. call function IN BACKGROUND TASK AS SEPARATE UNIT

3. Using submit via jobs

I want to know which technique is fastest and why?

[yuri_ziryukin]

Depends on your task.

If you need to do large number of calls with small packages and relatively quick processing, then 1 or 2.

1 if you need the results of each processing unit and want to implement resource controlling, 2 - if you are not interested in results.

3 is useful if each unit requires longer processing and you would like to implement some logging (use the spool or job log functionality).

[Former Member]

Yuri, Thanks for your response. I understand your point very well and i have the know-how of implementing these but what i am looking for is there a architectural difference in these different work processes which might make a different in runtime leaving apart over head of tracking and monitoring for different approaches.

[yuri_ziryukin]

Yuri, Thanks for your response. I understand your point very well and i have the know-how of implementing these but what i am looking for is there a architectural difference in these different work processes which might make a different in runtime leaving apart over head of tracking and monitoring for different approaches.

Sure there is an architectural differences.

3. Batch jobs.

Here batch jobs are occupying batch work processes. I'm not quite familiar with the resource controlling of batch jobs. May it cause the problem that some important system jobs will not run because your jobs occupied all available batch processes?.. Or you will have to implement resource controlling yourself keeping an eye on the number of running parallel jobs of your own type.

1. Asynchronous execution will start immediately in another dialog work process.

2. Transactional RFC starts at the moment of COMMIT WORK statement in your calling program. This method is normally used to ensure transactional consistency of the data (all or nothing principle). E.g. all changes of the tRFC calls will be written only if the calling program did COMMIT WORK and no ROLLBACK. In the case 1. the changes of parallel processes will be persisted to the database.

Case 1. is normally used if you need to organize parallel processing of logically independent objects.

[Former Member]

Yuri, Thanks for the response again and my apology if i did not clearly write that i am looking at architectural difference between the work processes used in these techniques which can make a difference. like starting new task uses dialog work process, jobs uses background work process and not very sure about background RFC as is does not show in SM50 but in SM58. So my question "Is there any architectural difference in Dialog work processes, background work process and tRFC due to which one could run faster than other." I think i should have had "Which work process is faster" as suject.

[ThomasZloch]

I dare say that the throughput of work processes on the same box is independent of the type "background" or "dialog" or whatever. There is many other factors than can influence the overall performance, too many to list here.

You could make yourself very valuable by creating three test scenarios based on the different techniques, process the same amount of data with each and publish the results here for further discussion.

What do you think?

Thomas

P.S. the boxes I have seen so far usually have more dialog processes than anything else (at least during daytime), so my bet is on parallel RFC using dialog processes

[former_member192616]

Hi,

regarding performance there is no difference, no matter where your ABAP code is running... may it be

DIA, BTC, UPD, UP2 .... .

There are some difference like timeout (DIA can timeout, BTC not) or memory allocation sequence. (depending

on the plattform and configuration BTC may allocate memory differentyl compared to DIA or BTC may be allowed

to allocatee mmore memory... ).

Besides that it is ABAP code (byte code) executed by the ABAP VM within an operting system process (disp+work)

where the process type does not matter... .

Bottomline:

The workprocess type does not influence your performance.

Yuri mentioned the relevant topics that drive your choice.

One more point:

Additionally sytem administration / operation should be involved. sometimes different operation modes are used for

day and night processing and there are less dia processes available at night... . That has to be considered if you go

for asynchronous RFC and you need to discuss this with your basis administration as well.

Kind regards,

Hermann

[yuri_ziryukin]

Yuri, Thanks for the response again and my apology if i did not clearly write that i am looking at architectural difference between the work processes used in these techniques which can make a difference. like starting new task uses dialog work process, jobs uses background work process and not very sure about background RFC as is does not show in SM50 but in SM58. So my question "Is there any architectural difference in Dialog work processes, background work process and tRFC due to which one could run faster than other."  I think i should have had "Which work process is faster" as suject.

See the answer of Hermann above. He summarized it in a best way.

[Sandra_Rossi]

Hi,

\[...\] or memory allocation sequence. (depending on the plattform and configuration BTC may allocate memory differentyl compared to DIA or BTC may be allowed to allocatee mmore memory... ).\[...\] 

I thought that because of the memory type allocation sequence (for Unix-like systems), it made dialog workprocesses slower than the background WP, especially when big processings are done: it happened to me several times that a dialog program was running for more than 20 minutes, and when ran in background it was done in 10 minutes. Is it really only an issue with SQL buffers not loaded yet, I doubt... Any idea?

About the initial question, I agree with Hermann, there's no obvious answer, but will depend on situations. One, for instance: if there is a very big job to run at a given moment (impossible to run small units on a wide time f'rame), it is obvious to me that we should not use RFC, but use parallelized batch jobs. Really?

Sandra

[former_member192616]

Hi Sandra,

differences in response time or batch should not be influenced by allocation seqence. A detailed root cause analysis needs to be done if there is a reproducible example.

Kind regards,

Hermann

[volker_borowski2]

Hm,

wasn't BTC equipped with a lower OS priority?

So if the CPUs are humming busily, the BTC get's less cycles compared to DIA?

(Not very sure about this, could be valid for version 3 or 4, but outdated now.)

No difference as long, as there are idle cycles available.

Volker

[former_member192616]

Hi Volker,

i have never heard of different priorites between DIA and BTC

(only from higher priorites for ENQ in benchmarks...)

Kind regards,

Hermann

[Sandra_Rossi]

Hi Hermann. Thanks. It was probably a too much quick interpretation... I will be more careful next time it happens, to find out what is the real cause.

Sandra

[phil_soady]

The throughput of the various workprocess is NOT identical.

I was a little surprised to discover this. Is seems the DISP+WORK behaves differently under different workprocess types.

Clearly a background process does need/ doesnt support dialog processing and the CLASSIC PBO/PAI process.

All I can say, is TEST it on your kernel.

We have done testing on this since every millisecond is import to us.

Dialog processes are sometimes twice as slow as baclground tasks depending on what is happening.

Im talking abap the ABAP execution time here.Not DB, RFC, external call , PURE ABAP execution times.

DIALOG was simply slower than Background processes.

TRY it: Build a report. SUBMIT report in background.

Run the same abap in dialog.

Include GET Runtime statements to measure the execution time microseconds.

fill it with PERFORM X using Y , CALL FUNCTION B or CALL CLASS->Method_M

set the clock resolution high, and measure the total execution time.

When running HUGE interfaces, processing 10 of millions of steps every day, this is of genuine consideration.

ALSO NOTE:

The cost of open JOB submit via job close should also be measured.

If your packets are too small, then background speed is lost to the overhead of submission.

The new Background RFC functions should also be tested with the SAME CODE.

happy testing

Phil

[yuri_ziryukin]

The throughput of the various workprocess is NOT identical.
> I was a little surprised to discover this.  Is seems the DISP+WORK behaves differently under different workprocess types.
> Clearly a background process does need/ doesnt support dialog processing and the CLASSIC PBO/PAI process.
> 
> All I can say, is TEST it on your kernel.
> 
> We have done testing on this since every millisecond is import to us.
> 
> Dialog processes are sometimes twice as slow as baclground tasks depending on what is happening.
> Im talking abap the ABAP execution time here.Not DB, RFC, external call , PURE ABAP execution times.
> 
> DIALOG was simply slower than Background processes.
> 
> TRY it:  Build a report. SUBMIT report in background.
> Run the same abap in dialog.
> Include GET Runtime statements to measure the execution time microseconds.
> fill it with  PERFORM X using Y , CALL FUNCTION B or CALL CLASS->Method_M
> set the clock resolution high, and measure the total execution time.
> 
> When running HUGE interfaces, processing 10 of millions of steps every day, this is of genuine consideration.
> 
> ALSO NOTE: 
> The cost of open JOB   submit via job  close should also be measured.
> If your packets are too small, then background speed is lost to the overhead of submission.
> 
> The new Background RFC functions should also be tested with the SAME CODE.
> 
> happy testing
> Phil

Dialog might be slower only due to the GUI communication or difference in ABAP source code. In some standard SAP applications there is a different processing depending on SY-BATCH flag.

Technically, as it was already mentioned several time above, both work processes (dialog and batch) are pretty identical (with slight differences in memory allocation).

So please don't confuse the community.

[andrs_sarcevic]

There is a nice framework provided by SAP to implement Parallel Processing with aRFC, on Package SPTA,

Check testing program SPTA_PARA_TEST_1. I've implemented this method with very good results, and you don't need to bother thinking on the synchronization, or distribution across multiple application servers, as all of that is already done by FM SPTA_PARA_PROCESS_START_2.

I normally used bgRFC when I don't care about the result, as someone already mentioned it. There is a very good article on this subject in SDN titled "Increase the efficiency of your RFC communications with bgRFC"... I couldn't find the link, sorry.

Regards,

Andres.