The job process could not communicate with the data flow process.

[Former Member]

Hi All,

I am performing address/firm match on some 8.4 million records. The job runs and processes around 5.71 million records and then throws a strange error.

The error says "

(12.2) 10-17-12 09:12:40 (E) (3145762:0001) RUN-050409: |Session Job_STS_Test

  The job process could not communicate with the data flow <DF_STS_Firm_Address_Match> process. For details, see previously

  logged error <170105>."

That's all it says and it doesn't have any previous logged error or anything. Does any one has any idea what this error is and how it can be rectified?

Thanks and Regards,

Madhur

[Former Member · Accepted solution]

Hi,

it is quite likely to be running out of memory - can you check the memory usage whilst it is runnnig?

regards,

Adrian

[Former Member · Accepted solution]

Yes, I checking the same. Do you think it would be the server memory issue or Data Base? I am assuming server, but it normally gives "Low virtual memory" warning if the server runs out of memory and strangely it doesn't say it this time around.

[Former Member · Accepted solution]

Hi,

you do not always get the low virtual memory warning - I would use the Windows Task Manager to watch the job run and see how much memory the various al_engine processes are using.

Also, is this DS3 or DS4?

regards,

Adrian

[Former Member · Accepted solution]

Hi Adrian,

The server is on Linux. I did see when my job ran, the space used shoots up. I processed 5 million records instead of 8 millions and it could run fine.

Can you tell me if there is a way or any setting which I can change so that my job doesn't use that much of the space while running? The space used by the job gets freed up automatically after the job finishes, but the Address/Firm Match and other Match transforms (I am assuming because it performs "order by" and other queries on the data) take a lot of space while executing the job. For example, it was taking up 20%(of 15 GB) while executing 4.6 million records. Is there any settings in the transform or somewhere which I can do to avoid it?

Thanks and Regards,

Madhur

[Former Member · Accepted solution]

It's DS3

[Former Member]

My job is writing to the disk on job server, but I am not able to figure out where it's writing, which file it's writing and how I can stop it from writing that file. The error says

“Data flow DF_STS_Firm_Address_Match - Cannot write onto Disk (files). OS Error: <No space left on device>

Session Job_STS_Test - The job process could not communicate with the data flow <DF_STS_Firm_Address_Match> process. For details, see previously logged error <170105>.”