cancel
Showing results for 
Search instead for 
Did you mean: 

BI7 : Change full load to delta load - things to do?

Former Member
0 Kudos

http://wiki.sdn.sap.com/wiki/display/profile/2007/04/30/Differencebetweenfull,intialanddeltaupdatemodes

Hi.

Since am not familiar with BW request clarity as to how to achieve the following :

We are on BI 7.

On our BW server, One of the Existing process chain loads data as full load daily Till date.

This schedule runs for about 4 to 6 hours during the nights. And the data seems to be growing.

And doing full load, now seems, not that meaningful for data analysis.

So we now feel we need not do a full load every day. But do only the incremental load.

How to make this effective in our existing process chains.

I went thro the 2 links above, but a few things am not clear.

What exactly needs to be done to change a full load into a delta load.

Also do not wish to have any data loss happening.

We run the schedules on a particular process chain for BW around 1 am.

Say from tomorrow I want to have the data loading as delta.

What exactly are the steps to be followed.

Precautions to be taken

So that we have the correct data load and no data is lost - while doing this.

Things to do am not clear :

do initialization with data transfer / or without data tranfer ?

The make it delta?

So that from tomorrow all data loading becomes delta ?

Or

since in our scenario,

daily basis it is full load.

Just make it delta load,

so that whatever changes will run in the next days schedule.

What all things are to be done stepwise.

if anyone could advise.

That would be very helpful.

Also any documentation which explains all these if someone could share the link please ?

Many thanks

indu

Edited by: Indumathy Narayanan on Aug 26, 2011 5:50 PM

Accepted Solutions (1)

Accepted Solutions (1)

Former Member
0 Kudos

Hi,

As suggested earlier check whether your DS supports delta or not, after that you can delete the yesterdays request and put the current request as delta init with data transfer, the very first delta without any selection will behave as full load only. After that once the data load is completed you can change the data load type to delta and next day onward it will simply extract delta data.

Regards,

Durgesh.

Answers (6)

Answers (6)

Former Member
0 Kudos

Follow up request to clarify :

Hi All.

Could anyone clarify this :

Our PC extracts data from R/3.

While making this change of init.

Is there any downtime required for R/3 transactions - if any in particular.

What if init is running, and system R/3 is alive. Will it not happen that some user interaction happens / and related data will be lost - while extracting data ?

Will running init - lead to data loss ? in any scenario? If yes, things to do?

Can any one advise.

Thanks for the help.

Regards

indu

Former Member
0 Kudos

Hi

Our PC extracts data from R/3.

While making this change of init.

Is there any downtime required for R/3 transactions - if any in particular.

till now you are running full load daily, i think users will not work while your loads are going on.

If you any users will work when your loads are going on in BI, then you need down time.

What if init is running, and system R/3 is alive. Will it not happen that some user interaction happens / and related data will be lost - while extracting data ?

If Init is running and some user enters data in R3, you will loss that data.

That is the reason in most of the cases.

we will do init without data transfer first.Then if we want any historical data load it as full repair.

Will running init - lead to data loss ? in any scenario? If yes, things to do?

Yes. you will loss data which loaded between this time.

Regards,

Venkatesh

Former Member
0 Kudos

Thank you Venkatesh.

You made my day.

Regards

indu

Former Member
0 Kudos

Thanks.. shall check the PC and the details advised.

former_member188080
Active Contributor
0 Kudos

Hi,

Kindly look at following links as well

http://help.sap.com/saphelp_nw04s/helpdata/en/44/9821620553053de10000000a1553f6/content.htm

Generic Delta

If a field exists in the extract structure of a DataSource that contains values which increase monotonously over time, you can define delta capability for this DataSource. If such a delta-relevant field exists in the extract structure, such as a timestamp, the system determines the data volume transferred in the delta method by comparing the maximum value transferred with the last load with the amount of data that has since entered the system. Only the data that has newly arrived is transferred.

To get the delta, generic delta management translates the update mode into a selection criterion. The selection of the request is enhanced with an interval for the delta-relevant field. The lower limit of the interval is known from the previous extraction. The upper limit is taken from the current value, such as the timestamp or the time of extraction. You can use security intervals to ensure that all data is taken into consideration in the extractions. After the data request was transferred to the extractor, and the data was extracted, the extractor then informs generic delta management that the pointer can be set to the upper limit of the previously returned interval.

The delta for generic DataSources cannot be used with a BW system release prior to 3.0. In older SAP BW releases, the system does not replicate DataSources for master data and texts that were made delta-enabled using the delta for generic DataSources.

Determining the Generic Delta for a DataSource

...

1. Choose Generic Delta.

2. In the subsequent dialog box, specify the delta-determining field and the type for this field.

3. Maintain the settings for the generic delta:

a. Specify a security interval.

The purpose of a security interval is to make the system take into consideration records that appear during the extraction process but which remain unextracted (since they have yet to be saved) during the next extraction.

You have the option of adding a security interval to the upper limit/lower limit of the interval.

A security interval should only be specified for the lower limit when the delta method results in a new status for changed records, in other words, when the status is overwritten in BW. In this case, duplicate data records that could arise in such a safety interval have no affect on BW.

b. Choose the delta type for the data to be extracted.

The delta type is used to determine how extracted data is interpreted in BW and which data targets in which it can be posted.

With the delta type additive delta, the record to be loaded for summarizable key figures only returns the change to the key figure. The extracted data is added in BW. DataSources with this delta type can supply both ODS objects and InfoCubes with data.

With the delta type New Status for Changed Records, every record to be loaded returns the new status for all key figures and characteristics. The values are overwritten in BW. DataSources with this delta type can write the data into ODS objects and master data tables.

4. Save your entries.

Delta transfer is now possible for this DataSource.

After generating the DataSource, you can see this from the marking for the field Delta Update on the DataSource: Customer Version screen.

In systems from release 4.0B, you can display the current value of the delta-relevant field in the delta queue.

Thanks and regards

Former Member
0 Kudos

Hi all.

Thanks for the replies and advise.

I shall check these on a dev environment before we could touch anything on the production.

Yes. We are pulling data from SAP R/3 -for the BW process chain.

And We are on BI 7.

If so, then, if downtime is required for doing this change, then we will have to understand again, why this downtime is required for R/3 or for BW. or both ? And why so. And then plan accordingly. Because both are production environments.

And too many things - i wish to understand before we do it.

The alternate suggestion given was : to create everything from the scratch on the dev, as in production.

so that no objects are re-used in the new process chain. Then do the delta change whatever required after understanding what needs to be done. And then run it parallelly as a separate job as delta. Along with the full load and check for atleast a month - before we could say yes, and move it to production. So that we absolutely know, that this delta load is working fine and data is not lost anywhere. This could take time for us. When everything is unfamiliar.

Otherwise, I dont see any ways of comparing data validation between a full load and a delta load.

Although we could slightly push this data of delta load to a staging table in BODS and check it.

And that sounds a bit risky . Though one could say, we could always bring it up from back up if things do not work.

Thanks again to all.

Regards

Indu

Edited by: Indumathy Narayanan on Aug 29, 2011 12:50 PM

Former Member
0 Kudos

Hi Indu,

I suppose you are loading your data from R/3 to BW. First check whether your datasource is delta enabled or not. If not please make it delta enabled.

Then create two IP under your datasource - one with delta init and other with normal delta.

For first time, run delta init. It will act as a full load and load your data till PSA.You can manually run this IP and load data(not via process chain). Then you add the IP with normal delta in your process chain instead of the previous IP with full load. So from next time you just trigger the process chain and data will arrive in your BW server.

As you are using BI 7.0 you need a transformation and DTP to load data from PSA to target. I think you do not need to make DTP as delta load. Use DTP as full load because in PSA only delta records will come.

Do one more thing. Just add a step in your process chain to delete previous PSA requests. Then when you trigger the PC, it will first delete the old PSA requests and then load data to PSA. So your PSA will contain only the new request. Then a DTP with full load will transfer all records to target. It will save your time and improve performance.

Thanks,

Indrashis

former_member209728
Active Participant
0 Kudos

Hi Indumathy Narayanan,

The first and foremost thing you will have to check is that is your datasource delta capable. This is because if you are using some genric datasource there might be some possibility that your data source may not support delta loads.

Now in the worst case scenario if your data source is not delta capable, then please check the selections for which the full load is done especially that do you require all the data? e.g you might be loading many months of data but the postings might have been locked for those periods.So if the loading is not happening for those perios/months/year then there is no point of loading that data daily. In this way you can implement what is called in the language of BW as Pseudo Delta. Implementing this may reduce your data volume.

Check out the below links for Pseudo Delta

http://wiki.sdn.sap.com/wiki/display/BI/PerformanceTuningofLoadswheredeltaisnotpossible

Regards,

Kush Kashyap

Former Member
0 Kudos

Hi Indu

As you told your PC will run at 1am and it is full load.I think you don't have any users who will enter data in R3 system at night time.

follow the below steps.

as you are loading daily full, i hope every day you are deleting yesterday request.

once your full load IP runs today.

1)create one IP for delta delta intialization

2)under update tab, select "Initialize delta process" and select "initialize with out data transfer"

3)this enables delta queue for your data source in RSA7(you can check this)

5)create one more IP for delta and in update tab select "Delta update" option

4)now remove your full load IP and delete yesterday request steps from your PC and replace with delta IP.

If you are working on BI 7.0, no need to create one more DTP for your delta loads

1)change the extraction mode from full load to delta load in DTP.

2)run this DTP with option "set the status for delta with out data extraction"

3)remove your delete yesterday request step from PC

can you provide your data flow in PC

Regards,

Venkatesh

Edited by: Venky1903 on Aug 26, 2011 6:47 PM

Former Member
0 Kudos

Hi Venkatesh/ Kush Kashyap / Durgesh.

Good morning and thank you very much for the suggestions.

But one thing which is bothering me is - do we need to have a downtime on the BW server -

For making this change from full load to delta load ?

I dont think so. At the most, if the data is not reflecting today, it would reflect tomorrow.

Is it not so ?

This change we are intending to make only in one of the jobs.

And when i checked the data source, it has grown up a bit large too, since we started, over the years, and hence the need felt for a delta loading... rather than full load.

Thanks, will come back to you .. with more understanding of the process chain flow,

so that am more clear of what things need to be done to make this effective in the coming weeks.

Thanks again.

Kind regards

Indu

Edited by: Indumathy Narayanan on Aug 28, 2011 10:59 AM

Edited by: Indumathy Narayanan on Aug 28, 2011 10:59 AM

Former Member
0 Kudos

Hi Indu,

If you are moving data between data targets in BI system, no need to go with downtime.

Down time comes in picture if you are loading data from R3 to BI system.

If you can provide process chain flow then we can analyze in a better way.

Regards,

Venkatesh