How to handle the large volumes of data retreived from BSEG

[Former Member]

Hello,

I need retrieve 1 period of FI document item from BSEG table, it's around 1 million of records. I really concern about the performance. there are 2 options,

1st: retrieve all data at a time, store it in internal table. I am afraid it will out of memory.

2nd: retrieve the data by package n lines in select statement, this will also increase the complexity of logic, since I need find some clearing document.

both these 2 options I think not good, is there better option to improve the performance. thanks.

[Former Member]

Hi Damon,

See if you can make use of primary or secondary index.

and specify as many clauses in Where clause to narrow the search.

Regards...

Sultana

[former_member214709]

Dear Damon,

Make sure that you use primary key for comparing/selecting data from BSEG table as it is a Cluster Table.

Use BUKRS,BELNR,GJAHR in your select statement as these are the primary keys. Hence these can make your database access efficient.

Regards

Dinesh

[Venkat_Sesha]

Do not use direct select statement to retrieve the data into Internal tables. As you said you will get some performance issues. sometimes Memory issue with Dumps.

So use the Select with OPEN CURSOR and then FETCH CURSOR. This is the best way to fetch millions and millions of data. Example is shown below in the below mentioned program

RFKKOP05FRM

Logic will be like this

OPEN CURSOR.

SELECT STATEMENT

FETCH CURSOR in Do-EndDo. Based on your logic..

CLOSE CURSOR.

Hope this helps., thanks

[former_member193464]

hi

Use Fetch cursor with package size...
what do you need to do after you fetch records in the internal table ? If you need to do some processing than usin parallel processing technique can be helpful.. if it meets your requirement.

[former_member214709]

yes open cursor with package size

[yuri_ziryukin]

Hello Damon,

whether you use packaged data transfer or not does not influence the performance very much. For the performance of the selection itself the most important is to provide the beginning of the primary key which is MANDT (automatically provided by the system), BUKRS (company code, normally not selective at all), BELNR (document number, I guess you don't have it in your select) and GJAHR (fiscal year). Missing document number will result in a quite bad performance.

Not sure if you know, but there are some index tables like BSID, BSIK, BSIS, etc. plus similar for cleared items: BSAD, BSAK, BSAS.

Please examine these tables and check if you can better use one/two of them as it will result in a much better performance.

The package size will only help you to reduce the amount of consumed memory. And for this one (surprisingly not mentioned by anyone else) please make sure that you are selecting ONLY NECESSARY fields from BSEG, as the table contains 305 fields and I am almost 100% sure you don't need all of them.

These were my 5 cents.

Regards,

  Yuri

[Former Member]

Hi Yuri,

I need retrieve 1 month FI document item from bseg table, it will be around millions of records. so package size is necessay for the performance.

And thanks for your suggestion, I will get document header first from BKPF, then retrieve items from BSEG table, so primary key is already known.

[yuri_ziryukin]

Again, Damon, package size is more to control your memory consumption, not for the performance.

And! If you first select from BKPF and knowing the document number go to BSEG, then you will definitely do it with "FOR ALL ENTRIES" option, right? Then the package size is not even going down to database (at least ABAP help tells that). The restriction is applied in the database interface.

It is good to use it (to make sure your program does not dump due to memory shortage), but has less to do with performance itself.