cancel
Showing results for 
Search instead for 
Did you mean: 

Work processes of shadow instance (SAP upgrade) not starting

Former Member
0 Kudos

HI,

I'm in the process of upgrading our system to EhP4. I'm now in MAIN_SHADOW/START_SHDI_FIRST phase. Now the shadow instance is not starting up and here's the contents of the dev_w0 of the shadow instance work directory:

M  kernel runs with dp version 241(ext=110) (@(#) DPLIB-INT-VERSION-241)
M  length of sys_adm_ext is 364 bytes
M  ***LOG Q01=> ThInit, WPStart (Workproc 0 1 27066) [thxxhead.c   1290]
M  ThInit: running on host a65od007
M  calling db_connect ...
C  Oracle Client Version: '10.2.0.4.0'
C  Client NLS setting (OCINlsGetInfo): 'AMERICAN_AMERICA.WE8DEC'
C  Logon as OPS$-user to get SAPP01SHD's password
C  Connecting as /@Q03 on connection 0 (nls_hdl 0) ... (dbsl 700 151208)
C  Nls CharacterSet                 NationalCharSet              C      EnvHp      ErrHp ErrHpBatch
C    0 WE8DEC                                                    0  0x467e180  0x4685c60  0x46895c8
C  Attaching to DB Server Q03 (con_hdl=0,svchp=0x46894f8,srvhp=0x468bc08)
M  ------------------ C-STACK ----------------------
(CTrcStack2+0x82)[0x6c5852]
(SigIGenAction+0x2a2)[0x1938d62]
/lib64/libpthread.so.0[0x2b5b14e13d60]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(gslcgvg_GetValues+0x75)[0x2b5b1d3984c5]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(ora_ldap_get_values+0x71)[0x2b5b1d3839b9]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(ldap_get_values+0x44)[0x2b5b1d383936]
/lib64/libnss_ldap.so.2[0x2b5b1e9777fc]
/lib64/libnss_ldap.so.2[0x2b5b1e979c30]
/lib64/libnss_ldap.so.2[0x2b5b1e9773ab]
/lib64/libnss_ldap.so.2(_nss_ldap_gethostbyname2_r+0x6d)[0x2b5b1e97a10d]
/lib64/libnss_ldap.so.2(_nss_ldap_gethostbyname_r+0x66)[0x2b5b1e97a1b6]
/lib64/libc.so.6(gethostbyname_r+0x121)[0x2b5b14ff4371]
/lib64/libc.so.6(gethostbyname+0xd3)[0x2b5b14ff3c23]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(snlpcgthstbynm+0x42)[0x2b5b1d4e7f52]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(snlinGetAddrInfo+0x408)[0x2b5b1d4ebf00]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(nttbnd2addr+0x2b2)[0x2b5b1d558ce2]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(ntacbbnd2addr+0x1b8)[0x2b5b1d436ea6]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(ntacbnd2addr+0x4e)[0x2b5b1d436c62]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(nsc2addr+0x8a)[0x2b5b1d4b1ed6]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(nscall1+0x17c)[0x2b5b1d3fe1c2]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(nscall+0x332)[0x2b5b1d3fd5d8]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(niotns+0x6c2)[0x2b5b1d441efc]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(nigcall+0x3b)[0x2b5b1d4bfa53]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(osncon+0x444)[0x2b5b1d447442]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(kpuadef+0x88)[0x2b5b1d0f05e8]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(upiini+0x2ec)[0x2b5b1d2651ac]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(upiah0+0x72)[0x2b5b1d24224a]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(kpuatch+0x101a)[0x2b5b1d0effe8]
/oracle/Q03/102_64/lib/libclntsh.so.10.1(OCIServerAttach+0xa7)[0x2b5b1d2689d9]
/usr/sap/Q03/upg/abap/exe/dboraslib.so[0x2b5b1cc94f99]
/usr/sap/Q03/upg/abap/exe/dboraslib.so(oci_logon+0xec)[0x2b5b1cc9a61c]
/usr/sap/Q03/upg/abap/exe/dboraslib.so[0x2b5b1ccab47a]
/usr/sap/Q03/upg/abap/exe/dboraslib.so[0x2b5b1ccab841]
/usr/sap/Q03/upg/abap/exe/dboraslib.so[0x2b5b1ccad0ee]
(open_con+0xf1)[0x129e631]
(supply_con+0x31d)[0x12a065d]
(dbsl_connect+0x11)[0x12acab1]
(db_connect+0x11e)[0x12a30ae]
(ThInit+0x977)[0x556de7]
(ThStart+0x11b)[0x5590db]
(DpMain+0x285)[0x4bd985]
/lib64/libc.so.6(__libc_start_main+0xf4)[0x2b5b14f3c304]
M  -------------------------------------------------
M
M Mon Jan 24 11:42:20 2011
M  ***LOG Q0E=> SigIGenAction, signal ( 11) [sigux.c      1067]
M  in_ThErrHandle: 1
M  ThSigHandler: signal (step 1, th_errno 11, action 4)
M
M  Info for wp 0
M
M    pid = 27066
M    severity = 0
M    status = 0
M    stat = WP_RUN
M    waiting_for = NO_WAITING
M    reqtype = DP_RQ_DIAWP
M    act_reqtype = NO_REQTYPE
M    rq_info = 0
M    tid = -1
M    mode = 255
M    len = -1
M    rq_id = 65535
M    rq_source =
M    last_tid = 0
M    last_mode = 0
M    semaphore = 0
M    act_cs_count = 0
M    csTrack = 0
M    csTrackRwExcl = 0
M    csTrackRwShrd = 0
M    mode_cleaned_counter = 0
M    control_flag = 0
M    int_checked_resource(RFC) = 0
M    ext_checked_resource(RFC) = 0
M    int_checked_resource(HTTP) = 0
M    ext_checked_resource(HTTP) = 0
M    report = >                                        <
M    action = 0
M    tab_name = >                              <
M    attachedVm = no VM
M  PfStatDisconnect: disconnect statistics
M  Entering TH_CALLHOOKS
M  ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP
M  *** ERROR => ThrSaveSPAFields: no valid thr_wpadm [thxxrun1.c   723]
M  *** ERROR => ThCallHooks: event handler ThrSaveSPAFields for event BEFORE_DUMP failed [thxxtool3.c  261]
M  ThCallHooks: call hook >BtcCallLgCl< for event BEFORE_DUMP
M  Entering ThSetStatError
M  ThIErrHandle: do not call ThrCoreInfo (no_core_info=0, in_dynp_env=0)
M  Entering ThReadDetachMode
M  ***LOG Q02=> wp_halt, WPStop (Workproc 0 27066) [dpuxtool.c   269] 

I have already tried to restart the shadow instance manually using SAP Note 525677 (with cleanipc) but the problem still persists

Edited by: Lee Hwei on Jan 24, 2011 6:22 AM

Edited by: Lee Hwei on Jan 24, 2011 6:27 AM

Edited by: Lee Hwei on Jan 24, 2011 6:30 AM

Accepted Solutions (0)

Answers (3)

Answers (3)

Former Member
0 Kudos

Hi,

Thank you all for responding. This is already solved. I changed the value of SAPLOCALHOSTFULL in all profiles of the shadow instance to the value of the virtual hostname (initially the value was the physical hostname)

nirmal_konchada
Active Contributor
0 Kudos

Thanks for enlighting us on this

sunny_pahuja2
Active Contributor
0 Kudos

Could you please also paste dev_disp and dev_ms logs from shadow instance directory.

Thanks

Sunny

Former Member
0 Kudos

>

> Could you please also paste dev_disp and dev_ms logs from shadow instance directory.

>

> Thanks

> Sunny

Hi,

here's dev_disp of shadow instance (is there a way here in SCN to paste this properly???)

-


trc file: "dev_disp.new", trc level: 1, release: "701"

-


sysno 12

sid Q03

systemid 390 (AMD/Intel x86_64 with Linux)

relno 7010

patchlevel 0

patchno 32

intno 20020600

make: single threaded, ASCII, 64 bit, optimized

pid 27006

Mon Jan 24 11:41:41 2011

kernel runs with dp version 241(ext=110) (@(#) DPLIB-INT-VERSION-241)

length of sys_adm_ext is 364 bytes

      • SWITCH TRC-HIDE on ***

***LOG Q00=> DpSapEnvInit, DPStart (12 27006) [dpxxdisp.c 1286]

shared lib "dw_xml.so" version 32 successfully loaded

shared lib "dw_xtc.so" version 32 successfully loaded

shared lib "dw_stl.so" version 32 successfully loaded

shared lib "dw_gui.so" version 32 successfully loaded

shared lib "dw_mdm.so" version 32 successfully loaded

rdisp/softcancel_sequence : -> 0,5,-1

Mon Jan 24 11:42:07 2011

      • WARNING => DpNetCheck: NiHostToAddr(www.doesnotexist0101.qqq.nxst) took 26 seconds

Mon Jan 24 11:42:17 2011

      • WARNING => DpNetCheck: NiAddrToHost(1.0.0.0) took 10 seconds

***LOG GZZ=> 2 possible network problems detected - check tracefile and adjust the DNS settings [dpxxtool2.c 5529]

MtxInit: 30000 0 0

DpSysAdmExtInit: ABAP is active

DpSysAdmExtInit: VMC (JAVA VM in WP) is not active

DpIPCInit2: start server >a65wd007_Q03_12 <

DpShMCreate: sizeof(wp_adm) 26928 (1224)

DpShMCreate: sizeof(tm_adm) 5351424 (26624)

DpShMCreate: sizeof(wp_ca_adm) 22912 (76)

DpShMCreate: sizeof(appc_ca_adm) 7600 (76)

DpCommTableSize: max/headSize/ftSize/tableSize=500/8/548040/548048

DpShMCreate: sizeof(comm_adm) 548048 (1088)

DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0

DpShMCreate: sizeof(slock_adm) 0 (104)

DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0

DpShMCreate: sizeof(file_adm) 0 (72)

DpShMCreate: sizeof(vmc_adm) 0 (1760)

DpShMCreate: sizeof(wall_adm) (22440/36712/72/104)

DpShMCreate: sizeof(gw_adm) 48

DpShMCreate: SHM_DP_ADM_KEY (addr: 0x2b5b1f96d000, size: 6023992)

DpShMCreate: allocated sys_adm at 0x2b5b1f96d000

DpShMCreate: allocated wp_adm at 0x2b5b1f96ed98

DpShMCreate: allocated tm_adm_list at 0x2b5b1f9756c8

DpShMCreate: allocated tm_adm at 0x2b5b1f975728

DpShMCreate: allocated wp_ca_adm at 0x2b5b1fe8ff28

DpShMCreate: allocated appc_ca_adm at 0x2b5b1fe958a8

DpShMCreate: allocated comm_adm at 0x2b5b1fe97658

DpShMCreate: system runs without slock table

DpShMCreate: system runs without file table

DpShMCreate: allocated vmc_adm_list at 0x2b5b1ff1d328

DpShMCreate: allocated gw_adm at 0x2b5b1ff1d3a8

DpShMCreate: system runs without vmc_adm

DpShMCreate: allocated ca_info at 0x2b5b1ff1d3d8

DpShMCreate: allocated wall_adm at 0x2b5b1ff1d3e0

MBUF state OFF

DpCommInitTable: init table for 500 entries

rdisp/queue_size_check_value : -> off

ThTaskStatus: rdisp/reset_online_during_debug 0

EmInit: MmSetImplementation( 2 ).

MM global diagnostic options set: 0

<ES> client 0 initializing ....

<ES> InitFreeList

<ES> block size is 4096 kByte.

Using implementation map

<EsMap> EsMapUnamFileMapInit: Initialize the memory 4096 MB

Linux: Kernel supports shared memory disclaiming

Linux: using madvise(<pointer>, <size>, 9).

Linux: disclaiming for shared memory enabled

<ES> 1021 blocks reserved for free list.

ES initialized.

      • WARNING => System running without ICM - check rdisp/start_icman [dpxxdisp.c 13169]

Mon Jan 24 11:42:19 2011

rdisp/http_min_wait_dia_wp : 1 -> 1

Mon Jan 24 11:42:20 2011

***LOG Q0K=> DpMsAttach, mscon ( a65wd007) [dpxxdisp.c 12483]

DpStartStopMsg: send start message (myname is >a65wd007_Q03_12 <)

DpStartStopMsg: start msg sent

CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.

CCMS: start to initalize 3.X shared alert area (first segment).

      • ERROR => DpWPCheck: W0 (pid 27066) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27066) killed with signal 11

my types changed after wp death/restart 0x1f --> 0x1d

      • ERROR => DpWPCheck: W1 (pid 27067) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27067) killed with signal 11

      • ERROR => DpWPCheck: W3 (pid 27069) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27069) killed with signal 11

      • ERROR => DpWPCheck: W4 (pid 27070) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27070) killed with signal 11

      • ERROR => DpWPCheck: W5 (pid 27071) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27071) killed with signal 11

      • ERROR => DpWPCheck: W7 (pid 27073) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27073) killed with signal 11

      • ERROR => DpWPCheck: W10 (pid 27076) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27076) killed with signal 11

DpMsgAdmin: Set release to 7010, patchlevel 0

MBUF state PREPARED

MBUF component UP

DpMBufHwIdSet: set Hardware-ID

***LOG Q1C=> DpMBufHwIdSet [dpxxmbuf.c 1048]

DpMsgAdmin: Set patchno for this platform to 32

Release check o.K.

Mon Jan 24 11:42:33 2011

      • ERROR => DpWPCheck: W0 (pid 27066) died (severity=0, status=0) [dpxxdisp.c 15662]

my types changed after wp death/restart 0x1d --> 0x0

      • ERROR => DpWPCheck: W1 (pid 27067) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W2 (pid 27068) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27068) killed with signal 11

      • ERROR => DpWPCheck: W3 (pid 27069) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W4 (pid 27070) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W5 (pid 27071) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W6 (pid 27072) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W7 (pid 27073) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W8 (pid 27074) died (severity=0, status=0) [dpxxdisp.c 15662]

      • ERROR => DpWPCheck: W9 (pid 27075) died (severity=0, status=11) [dpxxdisp.c 15662]

child (pid=27075) killed with signal 11

      • ERROR => DpWPCheck: W10 (pid 27076) died (severity=0, status=0) [dpxxdisp.c 15662]

nirmal_konchada
Active Contributor
0 Kudos

Hi,

Below note may help you.

Note 525677 - Problems when starting shadow instance

If this is not your Production System then you can shutdown your syste and repeat the phase.

Regards,

Nirmal.K

nirmal_konchada
Active Contributor
0 Kudos

In addition to my earlier reply, you can try this :

1) Stop upgrade tool and main instance.

2) use cleanipc command to remove semaphores.

cleanipc <instance number> remove. Remove from both shadow and main instance.

3) Start shadow instance.

4) Start upgrade tool and repeat the phase.

Regards,

Nirmal.k