All,
I have a pretty much straight Dataflow, wherein i am just trying to do a group by on a source table and load it into the Target.
The Source tables has around 200 Million records in total.
When 1 run this job, i see the following in the trace log. But, its not fetching any records. Job is getting successfully completed with 0 rows.
" Cache statistics determined that data flow <DF_XXXXXXX> uses <0> caches with a total size of <0> bytes. This is less than(or equal to) the virtual memory <1609564160> bytes available for caches. Statistics is switching the cache type to IN MEMORY. "
When i run the generated sql from the Data Flow on the Database directly i am able to see the data.
Did any one come across this before ?
Any thoughts / comments would be helpful.
Thank you.
Hello
Try running the job with SQL Loader tracing on - and then execute that SQL directly against the DB.
The monitor log shows the rows counts through each transform, are they all 0?
Do you have multiple configurations? If so, are you using the right one?! ;-)
Michael
Hi Michael,
Thank you for your quick response.
Tried the SQL loader tracing option on...Still same issue.
Yes, all the row counts through each transform is 0.
No, i don't have multiple configurations.
All,
Tried using the SQL transform and it works.
But, when used the source table and does the mappings..it failed ..still no rows..tried with bulk load option as well, doesn't work.
Any thoughts ?
Thank you.
Hi Raghavender,
Just a few questions:
1) Do you see the magnifying-glass on the source and can you see the records when you click on it?
2)You said easy workflow...there could be a push-down.
- What happens when you try with a target on the data-store of the source (same database), does that work?
- In both cases, when you click on the DataFlow and validate->optimized SQL, what does it look like?
- When you execute that SQL directly on the database, do you get the rows you are looking for?(at least the update one..)
Regards
Norbert
Hi Norbert,
To give you more clarity on the issue -
1. Yes, i can see the records on the source table by clicking on the magnifying glass.
2. Yes, there is a push down with Insert/Append.
- I did try with a target on the same data-store of the same db. It didnt work. still same error..
- In both the cases, the target is the same database just different schema's - It looks like Insert/Append.
- Yes, i see data directly on the database.
Do let me know if you have any more questions.
Thank you for your time.
Hi Raghavender,
You mentioned two different datastores and a push down. Can you execute the SQL with both schemas?
When you write click on the Job there should be a check on something mentioning use implicit data base links (or similar). Does the result of the SQL change when you uncheck that ?
If I remember correctly a pushdown on different datastores results in one SQL query that has be executed by one of both schemas. That user will execute the SQL. What happens if the other user executes it on your system? (Or did you try the SQL with both?)
Regards
Norbert
Hi,
a couple of thoughts/questions, what is the source DB? have you tried capturing the SQL sent by DS in the DB itself to ensure it is sending what it claims?
regards,
Adrian
Hi Adrian,
Source and Target DB is the same Oracle DB.
When i check the optimized SQL its just doing an Insert/Append.
Hi,
can you check the SQL that is actually sent to the DB, use a DB tool to catch this and check if it is the same as DS claims to be sending.
How long does the query take to run on the DB - could it be it is timing out?
Hi Raghvender,
Aren't you getting an error or warning ? Have you tried running job in debug mode ?
And also check you are not picking '0' length column from source.