Upgrade error - ERROR => DlLoadLib()==DLENOACCESS - dlopen

[former_member427720]

Hello,

during our upgrade from 4.6C to ECC 6.0 EHP5 we have encountered in phase CHECKS with following error msg:

1.

  SAPup broke during phase
NTACT_CHK in module PREP_GENCHECKS / General checks

Error Message: No lines found in logfile matching 'PN......\.SBX'

2.

EXECUTING /usr/sap/scratch2/SBX/UPG/abap/exenew/tp (/usr/sap/scratch2/SBX/UPG/abap/exenew/tp) ntaction ntop=chkant SBX pf=/usr/sap/scratch2/SBX/UPG/abap/bin/
DEFAULT.TPP srctt=DDNTT srctf=DDNTF dsttt=DDNTT dsttf=DDNTF intdel=YES -Dntact_timecmp=no protyear=40
This is /usr/sap/scratch2/SBX/UPG/abap/exenew/tp version 380.04.07 (release 720)
ERROR: No connect due to DbSl Load Lib Failure
ERROR: The values set for DIR_LIBRARY ('/usr/sap/scratch2/SBX/UPG/abap/exenew') or dbms_type ('ORA') are invalid.
ERROR: Connect to SBX failed (20130731105030).
Warning: Parameter INTERRUPT is no longer used.
Warning: Parameter DAYLIGHT_SHUTDOWN is no longer used.
Warning: Parameter WITH_TACOB is no longer used.
Warning: Parameter IMPDP_BY_EVENT is no longer used.
Warning: Parameter INTERRUPT is no longer used.
Warning: Parameter DAYLIGHT_SHUTDOWN is no longer used.
Warning: Parameter WITH_TACOB is no longer used.
Warning: Parameter IMPDP_BY_EVENT is no longer used.
Warning: Parameter DBCONFPATH is no longer used.
TRACE-INFO: 1:  [     dev trc,00000]  Wed Jul 31 10:50:29 2013                                                       168661  0.168661
TRACE-INFO: 2:  [    dlux.c  ,00000]  *** ERROR => DlLoadLib()==DLENOACCESS - dlopen("/usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o") FAILED
TRACE-INFO: 3:                          "Symbol resolution failed for /usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o because:
TRACE-INFO: 4:                          Symbol strcat (number 150) is not exported from dependent                          7  0.168668
TRACE-INFO: 5:                            module /oracle/client/10x_64/instantclient/libclnt"  (errno=8,Exec format error)

tp returncode summary:

TOOLS: Highest return code of single steps was: 0
ERRORS: Highest tp internal error was: 0249
tp finished with return code: 249
meaning:
  connect failed due to DbSL load lib failure
Process with ID 25690362 terminated with status 249

3. current env

TERM=xterm

AUTHSTATE=files

SHELL=/usr/bin/tcsh

HOME=/home/sbxadm

USER=sbxadm

PATH=/oracle/SBX/102_64/bin:.:/home/sbxadm:/usr/sap/SBX/SYS/exe/run:/usr/sap/toolbox/bin:/usr/sap/toolbox/lib/perl5/site_perl:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java5/jre/bin:/usr/java5/bin

TZ=MEZ-1MSZ-2,M3.5.0/02:00:00,M10.5.0/03:00:00

LANG=C

LOCPATH=/usr/lib/nls/loc

LC__FASTMSG=true

ODMDIR=/etc/objrepos

DEFAULT_BROWSER=/usr/bin/mozilla

LOGNAME=sbxadm

LOGIN=sbxadm

HOSTTYPE=rs6000

VENDOR=ibm

OSTYPE=aix

MACHTYPE=rs6000

SHLVL=1

PWD=/usr/sap/scratch2/SBX/UPG/abap

GROUP=sapsys

HOST=af6hg036

REMOTEHOST=160.118.200.22

SAPSYSTEMNAME=SBX

DIR_LIBRARY=/usr/sap/SBX/SYS/exe/run

THREAD=NOPS

dbms_type=ORA

dbs_ora_tnsname=SBX

dbs_ora_schema=SAPR3

ORACLE_PSRV=SBX

ORACLE_SID=SBX

ORACLE_HOME=/oracle/SBX/102_64

ORACLE_BASE=/oracle

NLS_LANG=AMERICAN_AMERICA.US7ASCII

SAPDATA_HOME=/oracle/SBX

NLSPATH=/usr/lib/nls/msg/%L/%N:/usr/lib/nls/msg/%L/%N.cat

LIBPATH=/oracle/client/10x_64/instantclient_10202:/oracle/SBX/102_64/lib

OS is AIX.

Can you please help us?

Thank you!

Rgs,

Ivan

[former_member427720 · Accepted solution]

Hello all,

thank you for your help. Problem is solved.

Problem was in variable in environment.

OLD:  LIBPATH=/usr/lib:/lib:/usr/sap/SBX/SYS/exe/run:
/oracle/client/10x_64/instantclient_10204/:/oracle/SBX/102_64/lib

NEW: LIBPATH=/usr/lib:/lib:/usr/sap/SBX/SYS/exe/run:/oracle/client/10x_64/instantclient_10204

I think the problem was /oracle/SBX/102_64/lib, cause in one note 819829, is written following:

The library search path ('LD_LIBRARY_PATH', 'SHLIB_PATH',
'LIBPATH') MUST refer to only one instant client directory. It MUST NOT contain
more than one instant client directory or any Oracle Home directory
('$ORACLE_HOME/lib'), regardless of the order.

Rgs,

Ivan

[former_member251231]

Have you update "lib_dbsl" library? I dont remmember  but I think is "dboraslib.so"

[former_member427720]

Hi,

yes, this was done.

Rgs,

Ivan

[Former Member]

Hi,

Please check permissions of file /usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o

Regards,

Sohrab Kapoor

[Former Member]

Hi,

LIBPATH=/oracle/client/10x_64/instantclient_10202:/oracle/SBX/102_64/lib

Did you adjust this environment variable after Oracle client update ??

Regards, KK

[former_member427720]

Hi,

yes, you are right. We have tried to set this enviroment in many ways due to SAP notes:

Note 602843 - Environment settings for R/3/Oracle on UNIX
LIBPATH=/usr/sap/<sid>/SYS/exe/run

Note 830578 - Environment variable on Unix for ora<sid> and <sid>adm
LIBPATH=/usr/sap/C11/SYS/exe/run:/oracle/client/10x_64/instantclient

Currently the LIBPATH is set:

LIBPATH=/usr/sap/SBX/SYS/exe/run:/oracle/client/10x_64/instantclient_10204

Thank you!

Rgs,

Ivan

[former_member206552]

Hi Ivan

your current path is this

Currently the LIBPATH is set:

LIBPATH=/usr/sap/SBX/SYS/exe/run:/oracle/client/10x_64/instantclient_10204

and your instant client points to 10202 remove this one

rm -r /oracle/client/10x_64/instantclient

rwxrwx    1 orasbx   dba              41 Feb 29 2008  /oracle/client/10x_64/instantclient -> /oracle/client/10x_64/instantclient_10202

and change the soft link to point to 10204

ln -s /oracle/client/10x_64/instantclient_10204 /oracle/client/10x_64/instantclient

[Former Member]

Hi Ivan,

I would like to point out few important things about Oracle client update,

     1. Create soft link to point to new version (As Marius Burger suggested)

     2. Set proper environment variable for <sid>adm user (As RB suggested)

Why because files .dbenv.csh and .dbenv.sh in your <sid>adm home directory use a line “ADD=/oracle/client/10x_64/instantclient”

If you upgrade the Oracle client to 11X then you need to update those files(.dbenv.csh and .dbenv.sh) as well.

Restart of SAP system need after Oracle client upgrade .

According to the error you are facing now, Please follow as like of Marius Burger and RB suggested

Regarding , KK

[former_member427720]

Hello,

thank you for your reply, however, the error is still the same.

See TP.ECO log:

SAPup> Starting subprocess 25428128 at 20130809090718
ENV: DIR_LIBRARY=/usr/sap/scratch2/SBX/UPG/abap/exenew
ENV: LIBPATH=/usr/sap/scratch2/SBX/UPG/abap/exenew:/usr/sap/scratch2/SBX/UPG/jvm/jre/lib/ppc64/server:/usr/sap/scratch2/SBX/UPG/jvm/jre/lib/ppc64:/usr/sap/sc
ratch2/SBX/UPG/jvm/jre/../lib/ppc64:/usr/sap/scratch2/SBX/UPG/jvm/jre/lib/ppc64/jli:/usr/sap/scratch2/SBX/UPG/jvm/jre/../lib/ppc64/jli:/oracle/client/10x_64/
instantclient:/oracle/SBX/102_64/lib
ENV: NLS_LANG=AMERICAN_AMERICA.US7ASCII
ENV: ORACLE_BASE=/oracle
ENV: ORACLE_HOME=/oracle/SBX/102_64
ENV: ORACLE_PSRV=SBX
ENV: ORACLE_SID=SBX
ENV: PATH=/usr/sap/scratch2/SBX/UPG/abap/exenew:/oracle/SBX/102_64/bin:.:/home/sbxadm:/usr/sap/SBX/SYS/exe/run:/usr/sap/toolbox/bin:/usr/sap/toolbox/lib/perl
5/site_perl:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/dt/bin:/usr/bin/X11:/sbin:/usr/java5/jre/bin:/usr/java5/bin
ENV: SAPSYSTEMNAME=SBX
ENV: auth_shadow_upgrade=0
ENV: dbms_type=ORA
ENV: dbs_ora_schema=SAPR3
ENV: dbs_ora_tnsname=SBX

EXECUTING /usr/sap/scratch2/SBX/UPG/abap/exenew/tp (/usr/sap/scratch2/SBX/UPG/abap/exenew/tp) ntaction ntop=chkant SBX pf=/usr/sap/scratch2/SBX/UPG/abap/bin/
DEFAULT.TPP srctt=DDNTT srctf=DDNTF dsttt=DDNTT dsttf=DDNTF intdel=YES -Dntact_timecmp=no protyear=40
This is /usr/sap/scratch2/SBX/UPG/abap/exenew/tp version 380.03.82 (release 720)
ERROR: No connect due to DbSl Load Lib Failure
ERROR: The values set for DIR_LIBRARY ('/usr/sap/scratch2/SBX/UPG/abap/exenew') or dbms_type ('ORA') are invalid.
ERROR: Connect to SBX failed (20130809090718).
Warning: Parameter INTERRUPT is no longer used.
Warning: Parameter DAYLIGHT_SHUTDOWN is no longer used.
Warning: Parameter WITH_TACOB is no longer used.
Warning: Parameter IMPDP_BY_EVENT is no longer used.
Warning: Parameter INTERRUPT is no longer used.
Warning: Parameter DAYLIGHT_SHUTDOWN is no longer used.
Warning: Parameter WITH_TACOB is no longer used.
Warning: Parameter IMPDP_BY_EVENT is no longer used.
Warning: Parameter DBCONFPATH is no longer used.
TRACE-INFO: 1:  [     dev trc,00000]  Fri Aug  9 09:07:18 2013                                                        72068  0.072068
TRACE-INFO: 2:  [    dlux.c  ,00000]  *** ERROR => DlLoadLib()==DLENOACCESS - dlopen("/usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o") FAILED
TRACE-INFO: 3:                          "Symbol resolution failed for /usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o because:
TRACE-INFO: 4:                          Symbol strcat (number 150) is not exported from dependent                         13  0.072081
TRACE-INFO: 5:                            module /oracle/client/10x_64/instantclient/libclnt"  (errno=8,Exec format error)

tp returncode summary:

TOOLS: Highest return code of single steps was: 0
ERRORS: Highest tp internal error was: 0249
tp finished with return code: 249
meaning:
  connect failed due to DbSL load lib failure
Process with ID 25428128 terminated with status 249

Rgs,

Ivan

[Reagan]

Hello

This is /usr/sap/scratch2/SBX/UPG/abap/exenew/tp version 380.03.82 (release 720)
ERROR: No connect due to DbSl Load Lib Failure
ERROR: The values set for DIR_LIBRARY ('/usr/sap/scratch2/SBX/UPG/abap/exenew') or dbms_type ('ORA') are invalid.

The problem you have is with the target kernel version.

I suggest you to update first try with a different version of the dbsl files at the target kernel location.

If that fails then try again with a different version of the 720 release kernel.

If none of these helps then raise an OSS.

I had suggested you to update the Oracle Instant Client to version 10.2.0.5 in one of my previous replies. Did you try that ?

Good Luck

RB

[Reagan]

Hello

Please provide the output of these commands

cd /oracle/client/10x_64/instantclient

./genezi -v

ldd /usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o

Regards

RB

[former_member427720]

Hello,

here is the output.

af6hg036:sbxadm 13> cd /oracle/client/10x_64/instantclient_10204
af6hg036:sbxadm 14> ./genezi -v
Client Shared Library 64-bit - 10.2.0.4.0

System name:    AIX
Release:        1
Version:        6
Machine:        00CB96254C00

af6hg036:sbxadm 15> ldd /usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o
/usr/sap/scratch2/SBX/UPG/abap/exenew/dboraslib.o needs:
         /usr/lib/threads/libc.a(shr_64.o)
         /oracle/client/10x_64/instantclient_10204/libclntsh.a(shr.o)
         /unix
         /usr/lib/libcrypt.a(shr_64.o)
         /usr/lib/libpthreads.a(shr_xpg5_64.o)
         /usr/lib/libdl.a(shr_64.o)
         /usr/lib/libodm.a(shr_64.o)
         /usr/lib/libperfstat.a(shr_64.o)
         /usr/lib/libcfg.a(shr_64.o)
         /usr/lib/liblvm.a(shr_64.o)
         /usr/lib/libcorcfg.a(shr_64.o)
         /usr/lib/libsrc.a(shr_64.o)

Rgs,

Ivan

[Reagan]

Hello

af6hg036:sbxadm 13> cd /oracle/client/10x_64/instantclient_10204
af6hg036:sbxadm 14> ./genezi -v
Client Shared Library 64-bit - 10.2.0.4.0

I see you have 10.2.0.4.0 version of the Oracle Instant Client

I suggest you to use the Oracle Client 10.2.0.5

Most probably this should fix else contact SAP for a solution.

Follow this SAP note and set the environment variables accrodingly

Note 819829 - Oracle Instant Client Installation and Configuration on Unix

Regards

RB

[Reagan]

Hello

In addition to what Marius suggested below, I suggest you to delete or move all the old version or unused clients from the /oracle/client location and  correct set the environment variables according to the SAP note I have mentioned above.

Most probably the issue can be fixed by changing the client path to /oracle/client/10x_64/instantclient.

Create the symbolic link as well.

Make sure you restart the SUM instead or repeating the phase.

If the error is still there then provide the logs.

Regards

RB

[former_member427720]

This message was moderated.

[former_member206552]

Hi Ivan

It seems like you have an issue with the Oracle client, so I recommend you to review your installation as per below mentioned Note

819829 - Oracle Instant Client Installation and Configuration on Unix.

Also refer below mentioned notes:

SAP Note No. 1547947:  Oracle Database 11g: Preparations for SAP Upgrade

SAP Note No. 602843 - Environment settings for R/3/Oracle on UNIX

source: http://scn.sap.com/thread/3287662

Hope it helps

Marius

[former_member427720]

Hi,

we have already checked the mentioned SAP note, even the SAP has recommend that in the SAP msg, but we have still the same error.

Rgs,

Ivan

[former_member206552]

Hi Ivan,

can you please just double check the links and maybe paste

ls -l /oracle/client

ls -l /oracle/client/10x_64

ls -l /oracle/client/10x_64/instantclient

also maybe check http://scn.sap.com/community/aix/blog/2008/05/05/sap-r3trans-error-on-aix-with-async-io

Best Regards

Marius

[former_member427720]

Hello Marius,

here are the checks:

af6hg036:sbxadm 4> ls -l /oracle/client
total 40
drwxrwxr-x    5 orasbx   dba            4096 Apr 29 2011  10x_64
drwxrwxr-x    4 dsmadm   sapsys         4096 Jul  5 08:41 11x_64
drwxr-xr-x    8 orargx   dba            4096 Oct  7 2009  9207
drwxr-xr-x    8 orargx   dba            4096 Oct  8 2009  9208
drwxr-xr-x    5 oraprd   dba            4096 Jul  9 2002  92x_64

af6hg036:sbxadm 6> ls -l /oracle/client/10x_64
total 32
-rwxrwxr-x    1 orasbx   dba             101 Mar 27 2006  dbclient.lst
lrwxrwxrwx    1 orasbx   dba              41 Feb 29 2008  instantclient -> /oracle/client/10x_64/instantclient_10202
drwxrwxr-x    2 orasbx   dba            4096 May 19 2006  instantclient_10201
drwxrwxr-x    2 orasbx   dba            4096 Mar 27 2006  instantclient_10202
drwxrwxr-x    2 orasbx   dba            4096 Apr 29 2011  instantclient_10204

af6hg036:sbxadm 7> ls -l /oracle/client/10x_64/instantclient
lrwxrwxrwx    1 orasbx   dba              41 Feb 29 2008  /oracle/client/10x_64/instantclient -> /oracle/client/10x_64/instantclient_10202
af6hg036:sbxadm 8> ls -l /oracle/client/10x_64/instantclient_10202
total 302240
-rwxrwxr-x    1 orasbx   dba         1593906 Feb 25 2006  classes12.jar
-rwxrwxr-x    1 orasbx   dba        24077164 Feb 25 2006  libclntsh.a
-rwxrwxr-x    1 orasbx   dba        23503095 Mar 27 2006  libclntsh.so
-rwxrwxr-x    1 orasbx   dba         4796980 Feb 25 2006  libnnz10.so
-rwxrwxr-x    1 orasbx   dba         5972886 Feb 25 2006  libocci.a
-rwxrwxr-x    1 orasbx   dba        70110326 Feb 25 2006  libociei.so
-rwxrwxr-x    1 orasbx   dba        22795273 Feb 25 2006  libocijdbc10.so
-rwxrwxr-x    1 orasbx   dba         1540330 Feb 25 2006  ojdbc14.jar

Thank you!

Rgs,

Ivan

[Former Member]

Hi,

SAP Note - 819829 - Update your Oracle Instant Client accordingly.

Also check

Installation Guide SAP ERP 6.0 Including SAP Enhancement Package 5 - Technical Usage “Central Applications” ABAP on AIX: Oracle - Page 9 - "The database must be Oracle 10.2.0.4 or 11.2.0, and higher"

Regards, KK

[former_member427720]

Hi,

we have currently:

af6hg036:sbxadm 1> sqlplus

SQL*Plus: Release 10.2.0.4.0 - Production on Wed Aug 7 08:40:08 2013

We have also checked the oraclie client and tried to install the latest one, but no success, still the same error.

Rgs,

Ivan

[Reagan]

Hello

Update the Oracle Instant Client and retry.

Regards

RB

[former_member427720]

Hello,

we have updated the system to latest oracle 10g client, but still the same error.

Any other thougths?

Thank you!

Rgs,

Ivan

[Former Member]

Hi,

did you get the latest kernel for TP and place it in below directory as per the note suggested.

• Replace the one in \SUM\abap\exe and \SUM\abap\exenew.

can you please run the below command as mentioned below with SIDADM.

sapcontrol -nr XX -function GetVersionInfo

thanks

Rishi Abrol

[former_member427720]

Hi,

yes we already change the kernel for latest TP and R3trans, no success.

The mentioned command is not working, since we have kernel below 7.20.

Rgs,

Ivan

[Former Member]

Hi Ivan,

Please check the below note.

1817743 - ERROR: No connect due to DbSl Load Lib Failure - With tp version 380.03.89

This will help you to fix your issue.

Thanks

Rishi Abrol

[former_member427720]

This message was moderated.