Data Services 4.2 HDFS(HDP 2.4)

[Former Member]

Trying read data from HDFS and load it to HANA. Job is simple without any transformation but it's not working and gives following error

8631 2822256384 RUN-050011 7/16/2014 9:25:10 AM |Data flow New_DataFlow3|Reader Query__AL_ReadFileMT_Read

8631 2822256384 RUN-050011 7/16/2014 9:25:10 AM Error: <HDFS Failed Connect to INBLRLLSSC114.apj.global.corp.sap in reader>.

8618 1423800064 RUN-050011 7/16/2014 9:25:27 AM |Data flow New_DataFlow3|Reader Query__AL_ReadFileMT_Read

8618 1423800064 RUN-050011 7/16/2014 9:25:27 AM Error: <HDFS Failed Connect to INBLRLLSSC114.apj.global.corp.sap in reader>.

Complete log is attached.

Job to read and load from HDFS( HDFS file format) to HDFS works but HDFS to HANA or HDFS to local file doesn't work.

It seems environment setup file provided with Data Services ( hadoop_env.sh) is for older version of hadoop.

Data Services is installed on Linux and Linux host is not part of Hadoop cluster. Installed all required hadoop component (hive,pig) on Data services host. Connectivity from Data services host to Hadoop cluster is working fine. Infact there is no issue with data loading from hive to Hana.

Any help/clue is appreciated to resolve this issue.

job_log.txt.zip

[Former Member · Accepted solution]

This problem is solved by setting HADOOP_HOME and setting libhdfs.so

[former_member324081 · Accepted solution]

Dear Sanjay,

  we have been encountered the same error that you have solved.

  I have followed your solution, but without results. 

  Our environment is Hadoop 2.2.0 and data services with SAP Data Services (Version 14.2.4.668) installed in the same server.

  In your experience,do you have any other suggestions that can be helpful to us ?

Thank you for your time

best regards

[Former Member · Accepted solution]

Dear Sanjay,

I am getting the same error you got. Unfortunately I don't know what do you mean with:

"This problem is solved by setting HADOOP_HOME and setting libhdfs.so"

I think in our environment the HADOOP_HOME is set properly and the directory containing the libhdfs.so is included in LD_LIBRARY_PATH.

Could you please share your knowledge.

Thank you,

Pablo