Issue with file events

[former_member1244799]

Hi All,

We are always getting the below message with the file events. .

'File event EVENT_ED1 encountered errors and will not be detected again until 2015/01/18 23:20:05,000 or when the process server is restarted'

I have to restart the process server every time I face this issue. When I restart it, I get the below message.

'File events may have been missed for *.txt in D:\INTRPTS\Interface\ED1'

Please advise.

Regards,

Ramana

[nanda_kumar21 · Accepted solution]

There is some issue with the platform agent detecting the file event.

What operator messages do you have related to this? check the operator messaged that were raised around the same time.

Check the file permissions for the user under which the platform agent is running on the directory and files inside them.

Make sure you checked the "Check if file in use" option for the file event.

thanks

Nanda

[former_member1244799 · Accepted solution]

Hi Nanda,

Could you please advise on the below scenarios.

SCENARIO 1 with overwrite flag set to NO :

Directory: D:\INTRPTS\Interface\ED1

Pattern: TEST*.txt

MOve Directory: D:\INTRPTS\Interface\ED1\CLNT120\testrun.txt

Poll interval: 15

overwrite file: NO

Check if file is in use: enabled

a) I placed the file of type 'TEST*.txt' in the source directory.

b) The event got raised, renamed the file to 'testrun.txt' and moved the file to CLNT120 folder.

c) I have not cleared the event and placed one more file of type TEST*.txt.

Below are the operator messages at this point

"File event EVENT_ED1 not raised as the target move file D:\INTRPTS\Interface\ED1\CLNT120\testrun.txt still exists, and the overwrite flag is not set."

"File event EVENT_ED1 encountered errors and will not be detected again until 2015/01/27 16:43:59,000 or when the process server is restarted"

d) Now I deleted the file 'testrun.txt' manually from CLNT120 folder.

e) Then I cleared the event manually.

Now the event is not raised for the second file.

f) I have to restart the process server at this point.

g) When I restarted the process server, the event gets raised, for the second file, renamed and moved it.

Below are the operator messages at this point.

"File events may have been missed for TEST*.txt in D:\INTRPTS\Interface\ED1"

1. Why the event is not raised when I deleted the file manually from the target directory?

2. How can I resolve this issue without restarting the process server every time?

SCENARIO 2 with overwrite flag set to YES:

Directory: D:\INTRPTS\Interface\ED1

Pattern: TEST*.txt

MOve Directory: D:\INTRPTS\Interface\ED1\CLNT120\testrun.txt

Poll interval: 15

overwrite file: YES

Check if file is in use: enabled

a) The event got raised, renamed the file of type TEST*.txt to 'testrun.txt' and moved the file to CLNT120 folder.

b) I have not cleared the event and placed two more files of type TEST*.txt in the source directory.

c) The file in the CLNT120 folder got overwritten and I see two more pending events apart from the event which is raised.

1. How come the file in the CLNT120 folder got over written twice eventhough I have not cleared the first event.I assumed that the file in the destination directory gets only overwritten when I clear the first event. Please advise.

Regards,

Ramana

[former_member1244799 · Accepted solution]

Also Nanda to let you know the platform agent is running under local system account.

Regards,

Ramana

[nanda_kumar21 · Accepted solution]

Scenario 1:

CPS is not allowed to overwrite, hence it registers that as an error, because now it cannot move the file to the location as requested. CPS self defense kicks in, stops monitoring for any more events, because, this may cause potential problems, if there are jobs looking for new file in the target directory.

for the first error, it will set a large time value until which the file event will not monitor the folder which is an hour, i guess. After that it will scan again and if the error still persists, it will double the time value until which it will not monitor the file.

This is expected behavior.

Scenario 2:

The sequence how the file event works is.

1. Detect the file
2. Move the file
3. Raise the event

This essential, because a file may be created anytime in the source directory and the event should be ready to detect it, hence the move happens first and doesn't care if you clear it or not.

If you don't clear it, you will see events in Pending status.

thanks

Nanda

[former_member1244799 · Accepted solution]

Thank you so much Nanda for making things so clear.

Could you please advise on the below scenario also.

Job should run whenever there is a file in the directory.

1) I have 3 files of type 'Test_*' existing in a directory at the same time. I need to rename each file to 'Test_1' and process one file at a time.

As I don't want to overwrite the file when the file is being processed. If I do this using the scenario 1, the event raises again for the second file after 1 hour and process it. For the third file one more hour it takes to process. Could you please advise how can we do this in a better way.

2) If I have 3 files of type 'Test_*' in a directory. I need to process the oldest file first. Can we do this using the events.

3) If the file event won't raise due to some issue, is there any way a person can be notified.

Please advise.

Thank you in advance.

Regards,

Ramana

[nanda_kumar21 · Accepted solution]

1. Create a job definition of a type(*.bat, *.sh) that is supported by native OS in which the platform agent is installed.
   • add a logic to move the file from one directory to another.
   • Rename the file.
2. In the file event, do not select the option to move, leave it empty.
3. Create a job chain
   • Step 1 - above job definition
   • step 2 - your actual job that processes the file.
4. Add the file event to the above job chain as wait event and set the automatic submit as Always.
5. Create a lock, add that lock to the job chain created above.

When the first file arrives, event will raise, job chain will start.

Step 1 - move the file and rename it

Step 2 - run the job and process the file

When the second event arrives, the job chain cannot start because the first instance is still using the lock. Once the first instance completes successfully, then second instance will start, moves the file, renames the file and second step kicks in.

thanks

Nanda

[former_member1244799 · Accepted solution]

Thank you so so much Nanda for answering my questions so patiently and making them so clear.

[former_member1244799 · Accepted solution]

One final question Nanda,

If I have a file event which is not raised when a file arrives due to some issue, is there any way we can know that the event is not raised.

Regards,

Ramana

[nanda_kumar21 · Accepted solution]

There is no default way to check that.

You can always play around with custom job definitions(Redwood script or native OS script) to setup extra checks.

For example, monitoring the operator messages for error related to File Event, but i think monitoring all Operator messages to find the one for File Event will be expensive on system resources.

thanks

Nanda

[former_member1244799 · Accepted solution]

Thank you so much Nanda for giving me good insights on this file events.

Regards,

Ramana

[gmblom · Accepted solution]

Hello,

The approach I like to take here is by not using the Autosubmit on file events, but instead use Never and plan the task without a submit frame (maybe a time window if you need it) and set the Restart Behavior on the Job Definition to restart on Completed (and on other statuses as well if required, I would recommend to restart with a delay on Error though )

Now, what happens is that the job will be scheduled and will go in EventWait until the file arrives, it processes the file, restarts and goes EventWait again until the next file arrives (or event gets raised).

The interesting about this is that you can setup a filter of EventWait jobs that are in EventWait > 1 day for instance. This makes it easy to measure how long an event was not raised.

Regards Gerben

[former_member1244799 · Accepted solution]

Thank you very much Gerben.  Nice setup.

Regards,

Ramana