SAP systems not starting

[Former Member]

Hello!<br>

<br>

We have SAP NetWeaver 7.0 EP high availability, Oracle DB, AIX.<br>

SAP systems not starting.<br>

<br>

"defaultTrace.17.trc"<br>

#1.5 #00215EBF64EA00000000000F014900E800049AE9BF384364#1296228633035#com.sap.engine.core.locking.AbstractLockingManagerImpl##com.sap.engine.core.locking.AbstractLockingManagerImpl.unlockAllGeneric(owner, asynchronous)#######Thread[Thread-1,5,main]##0#0#Error##Plain###com.sap.engine.frame.core.locking.TechnicalLockException: Cannot unlock all locks for owner ********************************************.......7325700; internal error (returncode = 8).<br>

at com.sap.engine.core.locking.impl3.LockingManagerImpl.unlockAllInternal(LockingManagerImpl.java:441)<br>

at com.sap.engine.core.locking.AbstractLockingManagerImpl.unlockAllGeneric(AbstractLockingManagerImpl.java:651)<br>

at com.sap.engine.core.locking.AbstractLockingManagerImpl.removeAllLocksForClusterId(AbstractLockingManagerImpl.java:965)<br>

at com.sap.engine.core.locking.AbstractLockingManagerImpl.init(AbstractLockingManagerImpl.java:183)<br>

at com.sap.engine.core.Framework.loadSingleManager(Framework.java:582)<br>

at com.sap.engine.core.Framework.loadManagers(Framework.java:258)<br>

at com.sap.engine.core.Framework.start(Framework.java:190)<br>

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)<br>

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)<br>

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:61)<br>

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60)<br>

at java.lang.reflect.Method.invoke(Method.java:391)<br>

at com.sap.engine.boot.FrameThread.run(FrameThread.java:61)<br>

at java.lang.Thread.run(Thread.java:770)<br>

Caused by: java.lang.Exception: class com.sap.bc.krn.enqu.EnquServerException reported: SERVER ERROR:<br>

Code...: 8<br>

Name...: ENQ_ERROR<br>

Text...: internal error, see log for details<br>

<br>

at com.sap.engine.core.locking.impl3.LockingManagerImpl.transformToExceptionForApplication(LockingManagerImpl.java:700)<br>

at com.sap.engine.core.locking.impl3.LockingManagerImpl.unlockAllInternal(LockingManagerImpl.java:440)<br>

... 13 more<br>

<br>

"dev_enqwork"<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmenq.h 1791]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmenq.h 2125]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmdeq.h 153]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmdeq.h 486]<br>

[Thr 515] ***LOG GEA=> [enxxmop.h 129]<br>

[Thr 515] ***LOG GZZ=> CleanupUser(); rtc=8 [enxxmop.h 130]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmdeq.h 486]<br>

[Thr 515] ***LOG GEA=> [enxxmop.h 129]<br>

[Thr 515] ***LOG GZZ=> CleanupUser(); rtc=8 [enxxmop.h 130]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmenq.h 1791]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmenq.h 2125]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmdeq.h 153]<br>

[Thr 515] *** ERROR => e=22 semop(1048592,(0,-1,4096),1) (22: Invalid argument) [semux_mt.c 1196]<br>

[Thr 515] ***LOG GEA=> SemRq( 26, 1, INFINITE_WAIT) rtc=1 [enxxmdeq.h 486]<br>

[Thr 515] ***LOG GEA=> [enxxmop.h 129]<br>

[Thr 515] ***LOG GZZ=> CleanupUser(); rtc=8 [enxxmop.h 130]<br>

<br>

"startsap_ERS15.log"<br>

Called command: /usr/sap/LPP/SYS/exe/run/startsap ERS15<br>

<br>

Starting SAP Instance ERS15<br>

-

---

<br>

Instance Service on host lpp02 started<br>

Unable to change to Directory /usr/sap/RPP/ERS15/work<br>

(24445010) Exiting with Return-Code -102. (Change Directory failed)<br>

Startup of Instance failed<br>

<br>

"std_server0.out"<br>

-

---

<br>

stdout/stderr redirect<br>

-

---

<br>

node name : server0<br>

pid : 25624694<br>

system name : LPP<br>

system nr. : 00<br>

started at : Fri Jan 28 10:23:37 2011<br>

<br>

<br>

[Thr 1] Fri Jan 28 10:23:38 2011<br>

[Thr 1] MtxInit: 10001 0 0<br>

<br>

<?xml version="1.0" ?><br>

<br>

<verbosegc version="200902_24"><br>

<br>

<br>

SAP J2EE Engine Version 7.00 PatchLevel 76340.450 is starting...<br>

<br>

Loading: LogManager ... 388 ms. <br>

Loading: PoolManager ... 2 ms. <br>

Loading: ApplicationThreadManager ... 1410 ms. <br>

Loading: ThreadManager ... 13 ms. <br>

Loading: IpVerificationManager ... 6 ms. <br>

Loading: ClassLoaderManager ... 9 ms. <br>

Loading: ClusterManager ... [Framework -> criticalShutdown] Error getting confirmation reply during attach to cluster.<br>

Jan 28, 2011 10:24:12... com.sap.engine.core.Framework [Thread[Thread-1,5,main]] Fatal: Critical shutdown was invoked. Reason is: Error getting confirmation reply during attach to cluster.<br>

</verbosegc><br>

<br>

I have tried solution from this thread:<br>

[;<br>

<br>

but i cannot connecting to Visual Administrator.<br>

<br>

Please help.<br>

Any comments would be appreciated.<br>

Edited by: Alexander Muravyev on Jan 28, 2011 7:28 PM

Edited by: Alexander Muravyev on Jan 28, 2011 7:30 PM

[Former Member]

Even though the original message was posted in 2011.

Please checkout this link for anyone else experiencing these types of issues:

http://www.sapfans.com/forums/viewtopic.php?f=3&t=57223

[Former Member]

Starting SAP Instance ERS15
-
Instance Service on host lpp02 started
Unable to change to Directory /usr/sap/RPP/ERS15/work
(24445010) Exiting with Return-Code -102. (Change Directory failed)

No further comment...

Cheers Michael

[Former Member]

I would recommend to stop SAP and Database if not not. Then do cleanipc.

Make sure not SAP is process is hanging.

Then bring up database and SAP.

This might help.

Thanks!

[Former Member]

Hi,

It is dual stack, or only abap system. check if dispatcher is not working menas IP or hostname change and check the services saposcol,

Thanks

BM

[Former Member]

Hi,

I'm not sure this is an OS issue. I'd be interested to know if the HA system has worked correctly in the past. From the errors it seems there may be a hanging lock on the enqueue server. You could probably confirm this by taking thread dumps at startup and checking for long running enqueue threads. If this is the case you may have to restart Central services to clear any locks? It might also be interesting to see if the system was placed in this state by a hard shutdown, perhaps directly from the OS. A hard shutdown might not have given the enqueue server enough time to release all its locks and caused the current issue.

Kind Regards,

John

[Former Member]

For me looks like OS memory issue, run with memory clean up command's.

Commands: ipcclean (SAP), ipcrm (OS)

Note: read restrictions of commands before use.

Regards,

Sreenivas