File/Hadoop CSV input : Discovery of file fails

[satyadeep_dey]

Hello ,

I'm trying to make the project as described in -

"Hands-on Tutorial: Building a Simple SAP Sybase ESP Project" for SAP Sybase ESP version 5.1 SP04.

I create a project where I add an input adapter of type - File/Hadoop CSV input.

Then in edit properties I enter the directory and the file name . In advanced tab I also set Has Header = TRUE.

Then  I click on Schema Discovery and this raises error - "Failed to parse discovery file : parse of file failed ".

I've imported the sample "towermon "project from SCN and this shows the same error !

I've done this before on SP02 and it used to work . Of course the File/Hadoop CSV input adapter did not exist in that release.

Can't quiet figure out what I'm doing wrong . Is this a bug in the software ?Any ideas how to fix this ?

Best regards,

Saty

[Former Member]

Hi,

I have the same problems as you had. I tried the solutions that are posted here, but no improvements. I will post it with screens

Firstly, I created File_Hadoop_CSV_Inptut1 as in tutorial:

Then I clicked on the Schema Discovery button

And chose this server (I don't have any other option)

After that I'v got error:

or this one:

Could you please help me?

Thank you!

Jan Bednar

[Former Member]

Hi Jan,

What version of ESP Studio are you using?

Thank you.
Beverly Duquette

[Former Member]

Hi Beverly,

I'm using version 5.1

Thank you!

Jan Bednar

[Former Member]

In ESP Studio, Help/About will show the full esp version.   Can you paste the entire version?

Thanks,
Beverly

[Former Member]

Hi,

Version: 5.1.110.00/20151206.1/SP11/winnt/x86_64/64-bit/OPT/Sun, Dec 06, 2015  5:41:30 PM

Thank you!

Jan Bednar

[Former Member]

Hi Jan,

I'm using the same ESP version and schema discovery worked ok for the File Hadoop CSV adapter.
Here are a few suggestions:

- When you start ESP Studio, right mouse click and select 'Run As Administrator'.
- Make sure you have full permissions to the folder and file.

Discovery failure can be caused by insufficient privileges.

If you still see the error, can you post the error messages from the frameworkadapter.log file located at %ESP_HOME%\studio\logs.

Thank you.
Beverly Duquette

[Former Member]

Thank you for your advices. I found exception in frameworkadapter.log that file parametersdefine.xsd threw exception, because I tried to create MQTT adapter, but there was mistake in this file.

Thank you for your help!

Jan Bednar

[former_member202429]

Hello,

I too got the same error. It worked for me, by placing the files in a folder with no space and no capital letters. e.g.: "c:\xyz\data\"

Thanks,

Ameya

[Former Member]

Hello,

I had the same problem you have mentioned here.

If you have any space in your directory path for example, '/ESP Hands-on Tutorial_Sample Code/towermon3/data', it won't work and will give you error "Failed to parse discovery file: parse of file failed..."

You have to give some path like '/ESP_Hands-on_Tutorial_Sample_Code/towermon3/data'

(basically you need to handle space characters in your directory path)

Thanks,

Bhumi Patel.

[JWootton]

I checked with the development team, and confirmed that the file toolkit adapter can support spaces in the path and/or filename.  The user can simply specify the names with spaces, but remember NOT to surround with quotes.  That's the trick

[Former Member]

Hello,

Two things to check:

1)  I don't know which file you are using from the "towermon" project.  One of the CSV files (input-streams.csv) prepends the stream name and opcode:

isSensorStream,i,2011-06-30T17:00:00.000,32.835,-114.188,Tower,Yuma T70,1,104,0

The other CSV files do not.  So you will have to toggle the "csvExpectStreamNameOpcode" parameter to true/false (the default is true).

2) There is a bug in the adapter discovery mechanism where it will return the error Failed to parse discovery file : parse of file failed ":

   749503 - Schema discovery does not work for toolkit adapters when unnecessary properties are listed on the adapter.

So for example, in text authoring mode, if you have this:

ATTACH INPUT ADAPTER File_Hadoop_CSV_Input1 TYPE toolkit_file_csv_input PROPERTIES

csvExpectStreamNameOpcode = TRUE ,

dir = 'c:/temp' ,

file = 'input-streams.csv' ,

dynamicMode = '' ,

removeAfterProcess = FALSE ,

csvDelimiter = '' ,

csvDateFormat = '' ,

csvTimestampFormat = '' ,

csvHasHeader = FALSE ,

pollingPeriod = 0 ,

scanDepth = 0 ;

You have to delete the unnecessary properties:

ATTACH INPUT ADAPTER File_Hadoop_CSV_Input1 TYPE toolkit_file_csv_input PROPERTIES

csvExpectStreamNameOpcode = TRUE ,

dir = 'c:/temp' ,

file = 'input-streams.csv' ;

Then switch back to visual authoring mode and try discovery again.

Hope this helps.

Thanks,

  Neal

[satyadeep_dey]

Hello Neal,

Tried this . Doesn't work .

Regards,

Satya

[Former Member]

Hello Satya,

Perhaps you could give us more information so we could help you?  Like maybe tell us which file you were trying to discover from the "towermon" example?  Or provide us with sample data?

If you would like to share you screen with me, I can also see what is going on that way.

Thanks,

  Neal

[satyadeep_dey]

Hello Neal ,

Just downloaded the towermon example again from SCN . I was getting this problem with Adapter1 . This works now with the latest files (weather-sensor-inventory.csv).

Thanks,

Satya