Dialogue instance not starting up after performing stack upgrade.

[Former Member]

Dear Experts,

We have performed stack upgrade on ERP 6.0 EHP5 system. In configuration phase we have not got the option to update dialog instances.

We proceeded without that option and completed the upgrade successfully but the dialog instance is not getting started.

Kernel level got upgraded in all instance with the same level. Below  is the work process log content of dialog instance.

Loading DB library '/usr/sap/MMD/D10/exe/dboraslib.so' ...

M  *** ERROR => DlLoadLib()==DLENOACCESS - dlopen("/usr/sap/MMD/D10/exe/dborasli

b.so") FAILED

  "I/O error mapping in '/usr/sap/MMD/D10/exe/dboraslib.so'."  [dlux.c       445

]

B  *** ERROR => Couldn't load library '/usr/sap/MMD/D10/exe/dboraslib.so'

[dbcon.c      5768]

M  ***LOG R19=> ThInit, db_connect ( DB-Connect 008192) [thxxhead.c   1519]

M  in_ThErrHandle: 1

M  *** ERROR => ThInit: db_connect (step 1, th_errno 13, action 3, level 1) [thx

xhead.c   11532]

As per SUM guide we need to uninstall the dialog instances before downtime and re-install the dialog instances after upgrade but we didn't uninstalled dialog instances before downtime.

Kindly let me know if there is any other process start up the dialog instance with out going for re-installing it.

Also suggest on re-installing the dialog instance since we didn't uninstalled dialog instances before downtime.

Thanks,

Bharath.

[Former Member]

Hi,

Can you check if all the dependent libraries are available?

>>ldd /usr/sap/MMD/D10/exe/dboraslib.so

Did you perform any OS patch/upgrade on dialog server?

Thanks

SG

[Former Member]

Dear Surojit,

The loaded libraries are existing under specified location.

Yes we have performed OS patch upgrade as per SAP Note 837670 in all instances.

Thanks,

Bharath.

[Reagan]

What is the target level of the system ?

Are you sure that the installation of the dialog instance is applicable for the release you have ?

If the dialog instance doesn't start then supply the trace files from work directory of the application servers.

Bear in mind that there is no start profile in the new SAP systems. During an upgrade or an EHP update the start profile gets merged with the instance profile.

[Former Member]

Dear Reagan,

Source release is EHP5 FOR SAP ERP 6.0 SPS 9.

Target release is EHP5 FOR SAP ERP 6.0 SPS 10.

Below is the work process log information of dialog instance.

Dispatcher log error

DpStartStopMsg: start msg sent

CCMS: SemInMgt: Semaphore Management initialized by AlAttachShm_Doublestack.

CCMS: SemInit: Semaphore 38 initialized by AlAttachShm_Doublestack.

CCMS: start to initalize 3.X shared alert area (first segment).

*** ERROR => DpHdlDeadWp: W0 (pid 266) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=266) exited with exit code 255

*** ERROR => DpHdlDeadWp: W2 (pid 268) died (severity=0, status=0) [dpxxwp.c     1737]

*** ERROR => DpHdlDeadWp: W4 (pid 270) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=270) exited with exit code 255

*** ERROR => DpHdlDeadWp: W6 (pid 272) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=272) exited with exit code 255

*** ERROR => DpHdlDeadWp: W8 (pid 274) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=274) exited with exit code 255

*** ERROR => DpHdlDeadWp: W14 (pid 280) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=280) exited with exit code 255

*** ERROR => DpHdlDeadWp: W20 (pid 286) died (severity=0, status=0) [dpxxwp.c     1737]

*** ERROR => DpHdlDeadWp: W21 (pid 287) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=287) exited with exit code 255

*** ERROR => DpHdlDeadWp: W22 (pid 288) died (severity=0, status=65280) [dpxxwp.c     1737]

DpTraceWpStatus: child (pid=288) exited with exit code 255

DpMBufHwIdSet: set Hardware-ID

Work process log error

Loading DB library '/usr/sap/MMD/D10/exe/dboraslib.so' ...

M  *** ERROR => DlLoadLib()==DLENOACCESS - dlopen("/usr/sap/MMD/D10/exe/dborasli

b.so") FAILED

  "I/O error mapping in '/usr/sap/MMD/D10/exe/dboraslib.so'."  [dlux.c       445

]

B  *** ERROR => Couldn't load library '/usr/sap/MMD/D10/exe/dboraslib.so'

[dbcon.c      5768]

M  ***LOG R19=> ThInit, db_connect ( DB-Connect 008192) [thxxhead.c   1519]

M  in_ThErrHandle: 1

M  *** ERROR => ThInit: db_connect (step 1, th_errno 13, action 3, level 1) [thx

xhead.c   11532]

Kindly suggest on weather our system is supported for dialog instance upgrade using SUM.

Thanks,

Bharath.

[willi_eimler]

Hi,

could you please test if

/usr/sap/MMD/D10/exe/dboraslib.so

exists? And if yes the permissions should be 775 and should belong to mmdadm:sapsys.

The stack update chenged the kernel. maby this procedure resulted the error.

Did you use the right kernel (unicode or non unicode)?

Best regards

Willi Eimler

[Former Member]

Dear Willi,

I already checked that file exists with 755 with owner:group as sidadm:sapsys.

Kernel is Unicode and patch level is same as that in central instance.

Kindly suggest.

Thanks,

Bharath.

[divyanshu_srivastava3]

Kindly suggest on weather our system is supported for dialog instance upgrade using SUM.

What do you mean by this ?

[Former Member]

Dear Divyanshu,

As per Reagan reply "Are you sure that the installation of the dialog instance is applicable for the release you have ?"

Our  Source release is EHP5 FOR SAP ERP 6.0 SPS 9 & Target release is EHP5 FOR SAP ERP 6.0 SPS 10.

I asked for the suggestion on whether our system is supported for dialog instance upgrade using SUM or not?

Thanks,

Bharath.

[divyanshu_srivastava3]

Hi Bharath,

Share the last 4 logs file which are generated in work directory - along with dev_w0.

Also, copy the kernel from CI and replace the old files of DI from copied kernel(if OS are same) or update the kernel of DI to match CI.

Regards,

[Former Member]

Dear Divyanshu,

Please find the work process log of dialog instance.

--------------------------------------------------

trc file: "dev_w0", trc level: 1, release: "721"

---------------------------------------------------

*

*  ACTIVE TRACE LEVEL           1

*  ACTIVE TRACE COMPONENTS      all, MJ

*

M sysno      04

M sid        MMD

M systemid   274 (HP (IA-64) with HP-UX)

M relno      7210

M patchlevel 0

M patchno    300

M intno      20020600

M make       single threaded, Unicode, 64 bit, optimized

M profile    /sapmnt/MMD/profile/MMD_D10_msdprd

M pid        266

M

M 

M Fri Oct  3 02:46:33 2014

M  kernel runs with dp version 139000(ext=121000) (@(#) DPLIB-INT-VERSION-139000-UC)

M  length of sys_adm_ext is 592 bytes

M  ThStart: taskhandler started

M  ThTHPInit: thp initialized

M  ThInit: initializing DIA work process W0

M  ***LOG Q01=> ThInit, WPStart (Workp. 0 1 266) [thxxhead.c   1349]

M  ThInit: running on host msdprd

M  calling db_connect ...

B  Loading DB library '/usr/sap/MMD/D10/exe/dboraslib.so' ...

M  *** ERROR => DlLoadLib()==DLENOACCESS - dlopen("/usr/sap/MMD/D10/exe/dboraslib.so") FAILED

  "I/O error mapping in '/usr/sap/MMD/D10/exe/dboraslib.so'."  [dlux.c       445]

B  *** ERROR => Couldn't load library '/usr/sap/MMD/D10/exe/dboraslib.so'

[dbcon.c      5768]

M  ***LOG R19=> ThInit, db_connect ( DB-Connect 008192) [thxxhead.c   1519]

M  in_ThErrHandle: 1

M  *** ERROR => ThInit: db_connect (step 1, th_errno 13, action 3, level 1) [thxxhead.c   11532]

M 

M  Info for wp 0

M 

M    pid = 266

M    severity = 0

M    status = 0

M    stat = WP_NEW

M    waiting_for = NO_WAITING

M    reqtype = DP_RQ_DIAWP

M    act_reqtype = NO_REQTYPE

M    req.req_info =

M    req.tid = -1

M    req.uid = 4294967295

M    req.mode = 255

M    req.len = 0

M    req.rq_id = 65535

M    req.rq_source =

M    last_tid = 0

M    last_uid = 0

M    last_mode = 0

M    locked semaphores (), locking ()

M    act_cs_count = 1

M    csTrack = 0

M    csTrackRwExcl = 0

M    csTrackRwShrd = 0

M    mode_cleaned_counter = 0

M    control_flag = 0

M    int_checked_resource(RFC) = 0

M    ext_checked_resource(RFC) = 0

M    int_checked_resource(HTTP) = 0

M    ext_checked_resource(HTTP) = 0

M    report = >                                        <

M    action = 0

M    tab_name = >                              <

M    attachedVm = no VM

M 

M  ThIErrHandle: current request:

M 

M  -IN-- sender_id ?                 tid  -1    wp_ca_blk   -1      wp_id -1

M  -IN-- action    -                 uid  -1    appc_ca_blk -1      type  -   

M  -IN-- new_stat  NO_CHANGE         mode 255   len         0       rq_id -1

M  PfStatDisconnect: disconnect statistics

M  Entering TH_CALLHOOKS

M  ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP

M  *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c   867]

M  *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c  302]

sapcpe.log of dialog instance

2014/10/03 02:45:39

sapcpe version UC: @(#) $Id: //bas/721_REL/src/ccm/sapstart/sapxxcpe.c#21 $ SAP

Command line:

  0: /usr/sap/MMD/SYS/exe/run/sapcpe

  1: pf=/sapmnt/MMD/profile/MMD_D10_msdprd

Trace level (profile): 1

** INFO:    Try to find file table sapcpeft (default:sapcpeft)

** INFO:    Try to find file table in the central directory /usr/sap/MMD/SYS/exe/run/sapcpeft

** INFO:    Found file table: /usr/sap/MMD/SYS/exe/run/sapcpeft

*************************************************

source: /usr/sap/MMD/SYS/exe/run

target: /usr/sap/MMD/D10/exe

*************************************************

Checking the files in the file table for copy type

** INFO:    file /usr/sap/MMD/SYS/exe/run/db4cncl is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/dbdb2slib.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/dbdb2slib9.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/dbdb2slibx.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/dbdb4slib.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/dbinfslib.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/dbsldb4ccm.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/libsecude.so is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/setdb4pwd is not existing and status -copy when file exists- is set

** INFO:    file /usr/sap/MMD/SYS/exe/run/xdnxdaapi.so is not existing and status -copy when file exists- is set

** INFO:    no extern File-List defined. Look for the default File-List(s).

Checking File-List(s).

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/instance.lst" exists.

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/instancedb.lst" exists.

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/tools.lst" does not exist.

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/inhouse.lst" does not exist.

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/dbclient.lst" does not exist.

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/igsexe.lst" exists.

** INFO:    File-List "/usr/sap/MMD/SYS/exe/run/sapcrypto.lst" does not exist.

*************************************************

Current list file: see table below

*************************************************

ActualizeLocalFileOrDir: instance.lst (FPM:0x0)

ActualizeLocalFile: instance.lst

ActualizeLocalFileOrDir: sapmanifest.mf (FPM:0x0)

ActualizeLocalFile: sapmanifest.mf

ActualizeLocalFileOrDir: saproot.sh (FPM:0x0)

ActualizeLocalFile: saproot.sh

ActualizeLocalFileOrDir: scsclient.lst (FPM:0x0)

ActualizeLocalFile: scsclient.lst

ActualizeLocalFileOrDir: j2eeinst.lst (FPM:0x0)

ActualizeLocalFile: j2eeinst.lst

ActualizeLocalFileOrDir: sapmanifestj2ee.mf (FPM:0x0)

ActualizeLocalFile: sapmanifestj2ee.mf

ActualizeLocalFileOrDir: scs.lst (FPM:0x0)

ActualizeLocalFile: scs.lst

ActualizeLocalFileOrDir: webdispinst.lst (FPM:0x0)

ActualizeLocalFile: webdispinst.lst

ActualizeLocalFileOrDir: gateway.lst (FPM:0x0)

ActualizeLocalFile: gateway.lst

ActualizeLocalFileOrDir: disp+work (FPM:0x1)

ActualizeLocalFile: disp+work

*** ERROR: CopyFile: write <dst> failed: -1 <> 1048576, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/disp+work -> /usr/sap/MMD/D10/exe/disp+work (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: rsyn.bin (FPM:0x1)

ActualizeLocalFile: rsyn.bin

ActualizeLocalFileOrDir: ABAP.pad (FPM:0x1)

ActualizeLocalFile: ABAP.pad

ActualizeLocalFileOrDir: aci_ABAP.pad (FPM:0x1)

ActualizeLocalFile: aci_ABAP.pad

ActualizeLocalFileOrDir: aci_ABAP.gram (FPM:0x1)

ActualizeLocalFile: aci_ABAP.gram

ActualizeLocalFileOrDir: R3trans (FPM:0x1)

ActualizeLocalFile: R3trans

*** ERROR: CopyFile: write <dst> failed: -1 <> 1030984, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/R3trans -> /usr/sap/MMD/D10/exe/R3trans (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: R3ta (FPM:0x0)

ActualizeLocalFile: R3ta

*** ERROR: CopyFile: write <dst> failed: -1 <> 1048576, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/R3ta -> /usr/sap/MMD/D10/exe/R3ta (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: R3check (FPM:0x0)

ActualizeLocalFile: R3check

*** ERROR: CopyFile: write <dst> failed: -1 <> 704256, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/R3check -> /usr/sap/MMD/D10/exe/R3check (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: tp (FPM:0x0)

ActualizeLocalFile: tp

ActualizeLocalFileOrDir: dipgntab (FPM:0x0)

ActualizeLocalFile: dipgntab

*** ERROR: CopyFile: write <dst> failed: -1 <> 1048576, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/dipgntab -> /usr/sap/MMD/D10/exe/dipgntab (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: saplicense (FPM:0x1)

ActualizeLocalFile: saplicense

*** ERROR: CopyFile: write <dst> failed: -1 <> 1048576, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/saplicense -> /usr/sap/MMD/D10/exe/saplicense (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: saplikey (FPM:0x1)

ActualizeLocalFile: saplikey

*** ERROR: CopyFile: write <dst> failed: -1 <> 620472, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/saplikey -> /usr/sap/MMD/D10/exe/saplikey (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: sapcpeft (FPM:0x0)

ActualizeLocalFile: sapcpeft

ActualizeLocalFileOrDir: startsap (FPM:0x0)

ActualizeLocalFile: startsap

*** ERROR: CopyFile: write <dst> failed: -1 <> 92600, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/startsap -> /usr/sap/MMD/D10/exe/startsap (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: stopsap (FPM:0x0)

ActualizeLocalFile: stopsap

*** ERROR: CopyFile: write <dst> failed: -1 <> 92600, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/stopsap -> /usr/sap/MMD/D10/exe/stopsap (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: jdbcconnect.jar (FPM:0x0)

ActualizeLocalFile: jdbcconnect.jar

ActualizeLocalFileOrDir: SAPCAR (FPM:0x0)

ActualizeLocalFile: SAPCAR

*** ERROR: CopyFile: write <dst> failed: -1 <> 1048576, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/SAPCAR -> /usr/sap/MMD/D10/exe/SAPCAR (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: rslgsend (FPM:0x1)

ActualizeLocalFile: rslgsend

*** ERROR: CopyFile: write <dst> failed: -1 <> 249984, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/rslgsend -> /usr/sap/MMD/D10/exe/rslgsend (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: rslgcoll (FPM:0x1)

ActualizeLocalFile: rslgcoll

*** ERROR: CopyFile: write <dst> failed: -1 <> 324120, 28, No space left on device, [sapxxcpe.c:2714]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/rslgcoll -> /usr/sap/MMD/D10/exe/rslgcoll (rc = 2), 28, No space left on device, [sapxxcpe.c:2528]

ActualizeLocalFileOrDir: gwrd (FPM:0x1)

Kindly suggest.

Thanks,

Bharath.

[divyanshu_srivastava3]

*** ERROR: ActualizeLocalFile: FAILED /usr/sap/MMD/SYS/exe/run/rslgcoll -> /usr/sap/MMD/D10/exe/rslgcoll (rc = 2), 28, No space left on device, 

Check

File systems are correctly mounted.

You have enough space for running applications.

Once above 2 points are checked and corrected, run disp+work from exe directory of CI and DI host and share the version.

Also, update DBSL on both CI and DI and try again.

Regards,