cancel
Showing results for 
Search instead for 
Did you mean: 

d filter on selected fields during Extraction in Data Service

Former Member
0 Kudos

Hi

We would like to add filter on the data source to extract only some specific range of values. For a huge dataset (around 5M), we only need probably 200 records. Instead of extract the whole 5M records, we need to find a way to apply a filter during or before the extraction. ( The 'where' condition in Query is only do the selection after extraction, which is not what we want)  Pl give me some help on that! Thanks!

CC

Accepted Solutions (0)

Answers (5)

Answers (5)

mageshwaran_subramanian
Active Contributor
0 Kudos

Use JOIN : Load the key values in another table and join with your source table.

Example : You have a table with 5m address records but you are interested in countries US ,AU and SG. Place these country codes in a separate table and join with the main to table to filter out the records.

Former Member
0 Kudos

Hi Magesh,

Thanks for your reply. My resource is ODP object from ERP, instead of tables. 'Load the key values in another table' , is it going happen in Data Service? Our case is, in ERP, RSA6, I checked a few columns as 'selection'. And then , we would like somewhere in data service I could set specific values for those columns and then just read data set based on those values. Should I use global variable?

Former Member
0 Kudos

Source is the ODP object import from ERP via RFC

mageshwaran_subramanian
Active Contributor
0 Kudos

Why do you not want to use WHERE?.Explain.

Use JOIN : Load the key values in another table and join with your source table.

Example : You have a table with 5m address records but you are interested in countries US ,AU and SG. Place these country codes in a separate table and join with the main to table to filter out the records.

Former Member
0 Kudos

Using 'Where', the extractor still read the whole data set. and then filter out the specific records based on where condition.

former_member198401
Active Contributor
0 Kudos

Whats your Source Database

Regards

Arun Sasi

Former Member
0 Kudos

We use RFC to connect with ERP system to get data.

Former Member
0 Kudos

Hi,

You can use SQL transform to extract data.Select TOP 200 * from source table.