SAP ECC Integration with Informatica Power Exchange - BCI Datasources

[Former Member]

Hi Experts,

I am working with Informatica and SAP ECC. We are extracting data in Informatica from SAP ECC via. Business Content Datasources.

We are using more than 30 datasources including Master (Attr, text) and Lo cockpit data extractors.

It turned out that during Initialization or delta loads of data we cannot trigger the datasources in parallel i.e say 3-4 datasources in parallel as in Informatica during staging of data all the parallel datasources data will come at once and then process workflow in Informatica will not know to which exact data target  data should go.

There is a particular mechanism in Informatica of data load:

1 Listener -> 1 Staging Area -> many processworkflow (which direct data to datatargets)

But the problem is if there is only 1 staging area then we cannot go for parallel data loads from SAP ECC.

I am still not sure about 1 staging area but my Informatica expert told me this. I have worked in SAP BW and there we have a lot of flexibility.

I think it is not possible that extraction from ECC to Informatica must be sequential and no parallel as in future we will be having 100's of datasources and loading everything in sequence will take days to complete only one load.

Is there any way to load data parallelly in Informatica from SAP ECC?

Any suggestion please!

Regards

Amit

[Former Member · Accepted solution]

Hi Amit,

Informatica supports extracting from multiple Business Content Datasources in a parallel mode. I have heard about several customers who are using several hundred data sources in parallel mode.

The forum to post these questions would be on Informatica forums

https://mysupport.informatica.com/community/my-support

You could also reach out to Informatica support to get your issue resolved.

Regards,

Thomas

[Former Member]

This message was moderated.

[Former Member]

Hi Amit,

Thanks a lot for you reply.

Please find below the comments for your queries -

• Can you check if the INFA system is able to connect to SAP?

           No, we are not able to connect to SAP through INFA, this is the point where we are stuck.

           It would be great if you can please tell us how we can test this connection similar to how we                    test for Oracle/Teradata.

• RFC destination from SAP able to connect to INFA in SM59?

          Yes, we are able to connect.

• Also you need to maintain correct entries in saprfc.ini file

          In the RFC file, we are providing the same destination for TYPE R and TYPE A connections - Can           you please share us the sample enteries and what should be the destination for TYPE R and                 TYPE A?

• Can you let me know if you have used the SAP delivered program /INFATRAN/ZINFABCI to generate the connections?

          Yes, we have used the same.

Kind Regards,

Naveen

[Former Member]

Hi Naveen,

Please check saprfc.ini for RFC connections on your local machine from where you are running the Designer.

It looks you have incorrect or missing RFC connection entries on local from where you are running the client.

See if it helps.

Thanks,

Om

Message was edited by: Omprakash Gurubaxani

[Former Member]

Amit and Guru,

Thanks for your reply. We are now able to connect to SAP ECC and extract data on full load. But, we are not able to do delta extraction on BSEG table in SAP ECC even after delta initialization in Informatica, basically the changes are not getting triggered. We have not defined any timestamp parameters in Informatica process mappings but we have followed PWX document for delta configuration. Can someone please tell us what configuration we need to do in both Informatica and SAP ECC for a successfull delta extraction (Delta Initialization, delta updates and delta repeat).

Thanks,

Naveen

[Former Member]

Hi Naveen,

                   We are also trying to integrate SAP ECC with Informatica using BCI method. we don't know how and where to start this configuration. would you please share us the detail steps required to configure the connectivity between SAP and Informatica. Also it would be great if you can post one sample RFC how an saprfc.ini file should be in particular to this BCI method.

Appreciate your help!!!!

Thanks,

Narasimharao

[Former Member]

Hi Narasimharao,

Sorry for the late reply. Have you made any progress with the configuration? Please let me know if you have already started and facing any issues.

Kind Regards,Naveen

[Former Member]

Hello Thomas and Kiran,

Thank you for your reply.

We have already reached to informatica support but they denied that we cannot do parallel loading in informatica - Personally I do not believe that.

As informed by our informatica team we can have only one listener per one logical system. Now flow is like below:

1 Logical System -> 1 Listener -> 1 Staging Area -> Workflow for respective datasource.

if we load data of many datasources in 1 staging area at the same time there will be no indication of which data belong to which datasource and workflow. So to separate data of different datasources we can load data of only one datasource at one time and not in parallel.

Even I believe that there must be a way to put them in parallel and load data for many datasources at one time.

But may be  I am missing some point here and not able to suggest on parallel run due to orchestration developed in Informatica.

It would be great if you can give me some ideas how can I acheive this -

May be multiple logical system, listeners, staging areas etc.

Please suggest.

Thanks

Amit

[Former Member]

Hi Amit,

We need your help for one of the issue we are facing while connecting to SAP ECC through BCI in Informatica, the logical system in SAP is already defined and we have the listner mapping also running, but we are getting the below error while generating the BCI mapping. It would be really great if you can please help us in this regard,

Kind Regards,

Naveen

[Former Member]

Hi Naveen,

Can you check if the INFA system is able to connect to SAP? And the RFC destination from SAP able to connect to INFA in SM59?

Also you need to maintain correct entries in saprfc.ini file.

Can you let me know if you have used the SAP delivered program /INFATRAN/ZINFABCI to generate the connections?

Thanks

Amit

[Former Member]

Hi Amit,

Did you get any solution of the problem you reported? Do we need any configuration other than which mentioned in the 'PWX_951HF4_SAPNetWeaverUserGuideForPowerCenter_en'?

We want to achieve below functionality:

• Loading data from multiple SAP BW Extractors/Data Source from one SAP system in parallel
• Loading data from SAP BW Extractors/Data Source from multiple SAP Systems in parallel

Thanks,

Malhar

[Former Member]

Hi Amit,

You can try:

Multiple Logical Systems -> Multiple Listeners -> Multiple Staging Areas -> Workflow for respective datasource/staging

This will enable parallel load.

Thanks,

Om

Message was edited by: Omprakash Gurubaxani

[Former Member]

Hello Malhar,

Sorry for late reply.

For your first case of multiple BW extractors from one SAP system in parallel, I do not think Powercenter allows to do so yet. You will have to go in sequential extraction. We have contacted Informatica Support for this query and we did not got a positive reply for parallel extraction instead we were told that do seqeuntial extraction. We have done many trials and tests but not able to run the extractors in parallel. Main problem here is that for one logical system there will be only one listener running in INFA which virtually restricts us to run anything in parallel.

If we try to run two extractors in parallel then at staging data will be mix of two datasources and following workflow will not be able to differentiate the data and will curropt the data (ideally data loading workflow will go haywire).

To overcome this we have minimized the data loading time as much as we can - by restricting the data load via selections and dynamic loading for text datasources based on attribute datasources. This applies to even if the datasources are FULL loads. Our list of 35 datasources it takes less than an hour even in sequential load.

For the other functionality of data from multiple SAP system I think it would be easier as you will have separate logical system and listeners running for both SAP systems.

However complexity will arise when data will be recieved in Informatica as you will have to clean the data from two systems and parse it into single dimensions - for ex. customer, vendor data etc.

If you are able to crank the parallel loading of multiple extractors I would be really interested to implement it in my project so please let me know

Please let me know if you have any questions on modeling side of Informatica.

(I have added skill of modeling in Informatica along with SAP BW now )

Regards

Amit

[Former Member]

Hi Om,

Yes this is what we are doing now going forward with other modules like in FI, SD, Inventory etc.

So we plan to implement one logical system per module which will spare us loading 100s of datasources in seqeuntial mode.

Thank you for your advice.

Regards

Amit

Message was edited by: Amit Dahiya

[Former Member]

Hi Amit,

Thanks a lot for the reply.

We also contacted Informatica Support about this issue and we got below reply from them.

Please note that you can connect to multiple SAP system using BCIintegration and run sessions to extracts from multiple datasource simultaneously.Also we dont have any documents other than SAP Netweaver User guide for BCIintegration details.

You can also refer to the below links as well:-

https://www.youtube.com/watch?v=fU92g-vDJXc

https://www.youtube.com/watch?v=sqDZ6CN2RvU

But, I think because of single staging target (Source_For_BCI) we can not execute multiple send request workflows to read data from multiple data sources in parallel.

Regards,

Malhar

[Former Member]

Hi Om, Hi Amit,

As per the PWX_951HF4_SAPNetWeaverUserGuideForPowerCenter user guide, /INFATRAN/ZINFABCI ABAP program creates the logical system with default name as INFACONTNT. We can not specify different name while creating logical system in one SAP System. It means we can not create more than one logical system in one SAP system with different names.

Thanks,

Malhar

[Former Member]

Hi Amit, Hi Om,

In processing mapping, I found below SQL override in Source qualifier DSQ_For_Mapping_For_0ARTICLE_TEXT. (Sample mapping provided by Informatica in BCI_Mappings.xml)

SELECT Source_For_BCI.IDocRecord, Source_For_BCI.DocumentNumber FROM Source_For_BCI WHERE Source_For_BCI.DocumentNumber in ( select Source_For_BCI.DocumentNumber  FROM Source_For_BCI where Source_For_BCI.DocumentNumber > '$$LASTDOCUMENTNUMBER' AND LTRIM(RTRIM(SUBSTR(Source_For_BCI.IDocRecord, 137,30))) = '0ARTICLE_TEXT' )

So I think we can load data sources/IDocs parallel in Source_For_BCI table and then while reading data from this table in Processing mapping we can specify above filter to read expected IDoc. After reading all IDocs from Source_For_BCI table, we can then run cleaning mapping.

One more thing, if we can load only one IDoc at a time in Source_For_BCI table, then why Informatica specified above filter in processing mapping.

Thanks,

Malhar

[Former Member]

Amit and Guru,

Thanks for your reply. We are now able to connect to SAP ECC and extract data on full load. But, we are not able to do delta extraction on BSEG table in SAP ECC even after delta initialization in Informatica, basically the changes are not getting triggered. We have not defined any timestamp parameters in Informatica process mappings but we have followed PWX document for delta configuration. Can someone please tell us what configuration we need to do in both Informatica and SAP ECC for a successfull delta extraction (Delta Initialization, delta updates and delta repeat).

Thanks,

Naveen

[Former Member]

Hi Naveen,

It entirely depends on what data extract you are using.

Are you extracting direcly from BSEG? I am assuming you are using 0FI_GL_4 for the extraction as it uses data from BSEG.

There are few points you need to consider for delta extraction from FI datasources:

1. There is a time safety limit of 2 hours (set in table BWOM_SETTINGS table). Which means if you make changes to any data record then it will only appear 2 hours of change.

2. Table TPS31 requires following entries to enable delta change capture for extraction:

Process	Lnd	Applic.	Function module
00005011			BWFIP_WRITE_AEDAT_POINTER

3. Also check the entries of table BWOM2_TIMEST for the extractor. Please check if extractor is initialized.

4. If you are looking at delta queue in RSA7 after making changes - it will not be visible there unless there is a successful delta extraction from Informatica.

Please let me know iif you need more clarification.

Thanks

Amit

[Former Member]

Hi Malhar,

Yes I agree that Informatica ABAP program automatically creates the logical system by its own.

But My plan is to do the configuration manually for another logical system and treat it as second logical system. However, I have never created two logical systems pointing to same destination - not sure if that possible or not but I will let you know the results and if success then will expain the steps for second logical system. Unfortunately my basis team is too conservative on giving authorization for this configuration to me (they want to do it by themselves )

Thanks

Amit

[Former Member]

Hi Amit,

Thanks a lot for your help, the configuration that you listed are done but still no success on the delta extraction Here are couple of questions :

1. Whether we can do delta update for 0FI_GL_4 datasource once per day?

For Example, Today we Initialised the delta in sap ecc and extracted full data to target through informatica. After that we changing few data and inserting few data in datasource. Whether we should be able to do delta update today itself or we need to wait till tomorrow.

2. In BCI MAPPING wizard STEP 3, what steps we need to do for delta initialization and delta update? If possible tel it clearly in detail.

It would be really great if you can please send your contact number and available time to navbms.eee@gmail.com. We will have some small discussion about our queries. Sorry I could not send you a direct message as we are not connected.

Kind Regards,

Naveen

[Former Member]

Dear Amit,

I am not aware that whether parallel processing of data from standard extractors is possible or not .

But Aware of a scenario where we are pulling the data from ECC system to Teradata using informatica,

and using parallel processing from Source, as Thomas suggested you can check with Informatica support as well on this issue,a small change in System settings may solve your problem.

Regards,

Kiran KL

[Former Member]

Hi Experts,

Any suggestions please.

Regards

Amit

[colm_boyle]

Hi Amit,

this issue is posted in the wrong area. However, you can take a look the following on your system and see if they are possible, I copied this information from customer messages submitted to SAP so hopefully it helps...

"if you look in txn /SAPAPO/SDP_EXTR for datasource 9A**** you see

that parallel processing profile Z_PARAL009 is assgined. As per

definition of this parallel processing profile 8 parallel processes will

be used."

please check to see if you have a similar parallel profile in your source. Also, in each case I checked it seems to be dialog processes being used, not sure why.

Hope this helps, you might have more luck with the 'dataservices' discussions,

Colm

p.s. generally you can maintain control parameters between a SAP source system and BW in the following location...

-> source system

  -> TA SBIW (Data Transfer to the SAP Business Information Warehouse)

     -> General Settings

        -> Maintain Control Parameters for Data Transfer

But I don't think this is relevant to your issue.

[Former Member]

Hello Colm,

Thank you for your suggestion.

I checked in my system and unfortunately could not find anything feasible.

Me and my team as now designing the ETL informatica but have to go in sequential data load only as Informatica do not provide parallelism yet.

I actually asked the question in this forum because my source system is ECC and we are extracting data in informatica using the BCI in ecc. And eventually I will be working with similar strategy in SAP BW too. Luckily we have a lot more flexibility there

Thanks Again.

Thanks

Amit