As per the business requirement we have a file interface, which has to process huge file in XI containing half a million records. We decided to split the file at every 5000th record and process the file.
I used the option “Record sets per Message” for splitting the message and did the necessary configurations in the ID. Once the configurations are complete when I tested the file I observed that splitting happens very fast at 5000th records and Idocs are posted into SAP IS-U but surprisingly after 2.25 lakh record the splitting stops and the rest of the entries in the files does not get processed. I don’t see any errors in sxmb_moni or RWB or Visual admin logs but the file vanishes after posting 2.25 lakh records.
We thought there are some explicit settings for reading huge files, which has to be done in J2EE stack. After wasting lot of time in exploring that we thought we will split the huge file at every 2.25 lakh records manually and process it .In that process the source file is split into 3 files and then tried to upload the 3 files separately. I have noticed strange phenomena that only 2 idocs are posted from the second file and file again gets vanished and no error logs are seen even this time anywhere. As a part of understanding the weird phenomena I removed the top 10,000 records and put the file again. And my exercise worked!!! It gave a CC error in the RWB that it is not able to convert the 1500 record due to the error in data.
The outcome is: 2.265-lakh record of the file is the error record, which contains ‘,’ inside the content of the field that has caused this problem.
"ROW",76479006,"U",20031125,"T","FIELD","","T",88000,"F88","Y","FIELD",1985,",5890061","Y","Y"
",5890061" is the error field as the field separator ‘,’ is coming as the content .I removed ‘,’ manually and uploaded the file 2 successfully and it posted all the records this time.
This is the same reason due to which the adapter is processing exactly 2.25 lakh records initially and doesn’t give any error logs. So we need to skip the ‘,’ if it comes in the content using file CC parameters in the ID.
Have a look at sample configuration to skip the field separator if it comes in the content of the file. I edited the actual configuration due to the confidentiality purposes.
Good news is that it just took 15 minutes for processing 65 MB file and splitting the file at 5000th record after doing the file CC configurations as shown above but the simple problem of ',' in the content has taken lot of time in trouble shooting due to the lack of proper logs and integrated tools for debugging adapter level failures .I assume that it will be atleast provided in the upcoming versions of SAP XI.
11 Comments
