STRING_SIZE_TOO_LARGE dump in SE38 Program

[Former Member]

Hello,

I am trying to load a large 20 GB flat file in my Z application table using open dataset command. Program runs fine till 3.4 million records and fails with dump STRING_SIZE_TOO_LARGE.

I can see around 3.4 million number of entries in Z table but it gives dump STRING_SIZE_TOO_LARGE at line 

READ DATASET p_file INTO lw_tab.               

lw_tab is of string data type, I even tried to change it to C with length 260000, but receiving same error,

The file has around 85 columns and probably 15 million rows. Is there any way to load file without splitting it..!

Harshawarhan

[former_member201275]

A 20gb file is never going to load into abap memory!

[Former Member]

The question has to be, why the hell are you trying to load a 20Gb file straight into an ABAP program?!

[Former Member]

Actually, we need to compare the reporting performance efficiency of BWonHANA system with BW on MSSQL

We want to show case to client.. how much faster they data can load into BWonHANA system and how much efficient is the reporting on BWonHANA systems.
Client has provided us the file of 20GB to be loaded into BWonHANA system and they have loaded the file in BW data source from infopackages in BW on MSSQL system.

As we are facing some Idoc issues while loading this file through Infopackage in BWonHANA system, we were thinking of the option to load the file into BW PSA  through ABAP Program.

We have already asked client to split the files but just wanted to check if we have any other way to handle this predicament.

[Former Member]

Hi,

I've got to be honest, it doesn't bode well for your customer if you can't even get the data loaded correctly...

I don't know much about loading data into BW or HANA as it's not my bag, but I'd suggest you research the standard methods for achieving this and work through them.  If they don't work, review OSS and/or fix your logic.  Ultimately, you won't be able to load such an amount of data via a simple ABAP program in one go.

Cheers,

G.

[Former Member]

Hi,

Why not break the File into small pieces and run the program N number of times.

[thanga_prakash]

Hello Harsha,

From the dump you can clearly see that, because of the String length, error occurred.

SAP can handle string up to length of 2145386496, might be in one of the segment/cell in the file which you are trying to upload has string length more than the above length, so it gave you this error.

Refer to the second point in the below link, it is controlled by a profile parameter.

https://help.sap.com/abapdocu_70/en/ABENMEMORY_CONSUMPTION_2.htm

Regards,

Thanga

[former_member686620]

Hello Harshawardhan,

Have you found any solution to your problem? I'm facing a similar one and Basis team have modified ztta/max_memreq_MB parameter up to 6141 lenght but it didn't work. Do you have any ideas?

Thanks.

[Former Member]

How do you open the dataset?  If you open for input in BINARY mode then you need to control the amount of data per single READ in the READ DATASET  MAXIMUM LENGTH setting.  But sounds more like this is a case of individual records with line-ends in the file, in which case it should be opened for input in TEXT mode.  If so you should only load a single record of 185 columns into your lw_tab variable at a time and not be creating any long strings.

Jim