SfCJavaVm: exit hook is called. (rc = 2150)

[Former Member]

Hi Experts,

I am facing problem in starting java server in Portal system.I am attaching the log files from work directory. Please help me to resolve this.

Below is a section of dev_server0 log file.It seems memory issue to me.plz suggest.

********************************************************************************

J     :    [258]   10:00:25    ***Warning: Omitting automatic activation of GC history: Only 471.29 MB usable disk space available.

J  JVMX version - Nov  2 2012 03:57:47 - 61_REL - optU - aix ppc64 - 6 - bas2:182728 (mixed mode)

J  (CompilerOracle read from file /usr/sap/EPD/J01/exe/sapjvm_6/jre/.hotspot_compiler)

F  [Thr 258] *** LOG => SfCJavaVm: Java VM started.

F

F Tue Feb  4 10:00:26 2014

F  ********************************************************************************

F  Main method call:

F  com/sap/engine/boot/Start.main()

F  ********************************************************************************

J

J Tue Feb  4 10:00:29 2014

J  3.392: [GC 3.392: [ParNew: 343552K->3825K(381696K), 0.0668230 secs] 343558K->3832K(2059008K), 0.0671560 secs] [Times: user=0.00 sys=0.08, real=0.07 secs]

F

F [Thr 258] Tue Feb  4 10:00:29 2014

F  [Thr 258] *** LOG => State changed from 0 (Initial) to 1 (Waiting to start).

F  [Thr 258] *** LOG    state real time: 4.872 CPU time: 0.520 sys, 6.060 usr

F  [Thr 258] *** LOG    total real time: 4.872 CPU time: 0.520 sys, 6.060 usr

F  [Thr 258]

F  [Thr 258] *** LOG => State changed from 1 (Waiting to start) to 2 (Starting framework).

F  [Thr 258] *** LOG    state real time: 0.000 CPU time: 0.000 sys, 0.000 usr

F  [Thr 258] *** LOG    total real time: 4.873 CPU time: 0.520 sys, 6.060 usr

F  [Thr 258]

J

J Tue Feb  4 10:00:36 2014

J  11.103: [GC 11.103: [ParNewJ

J Tue Feb  4 10:00:37 2014

: 347377K->34828K(381696K), 0.1847490 secs] 347432K->34884K(2059008K), 0.1851530 secs] [Times: user=0.00 sys=0.26, real=0.19 secs]

J

J Tue Feb  4 10:00:45 2014

J  20.235: [GC 20.235: [ParNewJ

J Tue Feb  4 10:00:46 2014

: 378380K->38144K(381696K), 0.4181470 secs] 378599K->62251K(2059008K), 0.4184690 secs] [Times: user=0.01 sys=0.57, real=0.42 secs]

J

J Tue Feb  4 10:00:51 2014

J  26.261: [GC 26.261: [ParNewJ

J Tue Feb  4 10:01:56 2014

J  27.742: [GC 27.742: [ParNew: 381696K->38144K(381696K), 0.4228780 secs] 413672K->111577K(2059008K), 0.4232030 secs] [Times: user=0.00 sys=0.76, real=0.43 secs]

J

J Tue Feb  4 10:02:07 2014

J  39.265: [GC 39.265: [ParNewJ

J Tue Feb  4 10:02:08 2014

: 381696K->38144K(381696K), 0.4925380 secs] 455789K->147872K(2059008K), 0.4928750 secs] [Times: user=0.00 sys=0.72, real=0.49 secs]

J

J Tue Feb  4 10:02:10 2014

J  Heap

J   par new generation   reserved 419840K, committed 419840K, used 58667K [0x0000000040000000, 0x0000000059a00000, 0x0000000059a00000)

J    eden space 343552K,   5% used [0x0000000040000000, 0x000000004140ae68, 0x0000000054f80000)

J    from space 38144K, 100% used [0x00000000574c0000, 0x0000000059a00000, 0x0000000059a00000)

J    to   space 38144K,   0% used [0x0000000054f80000, 0x0000000054f80000, 0x00000000574c0000)

J   concurrent mark-sweep generation reserved 1677312K, committed 1677312K, used 109729K [0x0000000059a00000, 0x00000000c0000000, 0x00000000c0000000)

J   concurrent-mark-sweep perm gen reserved 1048576K, committed 1048576K, used 64534K [0x00000000c0000000, 0x0000000100000000, 0x0000000100000000)

F

F [Thr 1800] Tue Feb  4 10:02:10 2014

F  [Thr 1800] *** LOG => SfCJavaVm: exit hook is called. (rc = 2150)

F

F  ********************************************************************************

F  *** ERROR => Java node 'server0' terminated with exit code 2150.

F  ***

F  *** Please see section 'Java program exit codes'

F  *** in SAP Note 1316652 for additional information and trouble shooting advice.

F  ********************************************************************************

F

F  [Thr 1800] *** LOG => exiting (exitcode 2150, retcode 1).

M  [Thr 1800] CCMS: CCMS Monitoring Cleanup finished successfully.

Thanks

Aditya Roushan

dev_jstart.zip

dev_server0.zip

[Former Member · Accepted solution]

Hi All,

Sorry for delayed response.Actually I had gone on leave.I am mentioning the steps that helped me to resolve the issue (note 1827807).

1)Create Communication user SAPJSF (if it does not not exist) with appropriate role/profile.

2)Edit the UMEBackendConnection via the offlinecfgeditor (or via the configtool -> switch to configuration editor mode)

3)Then navigate to: Configurations => destinations => RFC

4)Change to edit mode then double click "Propertysheet  <UMEBackendConnection>

5) Maintain Following entries

jco.client.ashost = <Hostname of the backend system>

jco.client.client = <Client number of the backend system>

jco.client.passwd = <Communications users password>

jco.client.sysnr = <System number of the backend system>

jco.client.user = <Commuinication user (typically SAPJSF)>

jco.client.r3name = <SID of backend system>

6)Restart the cluster using jcmon pf=<profile>   => 10:cluster administration menu.

Thanks to RB and other participants for their guidance.

Thanks

Aditya Roushan

[Former Member]

Hi Aditya,

If your SAP server have connection issue with LDAP server, then the UME will not work. Also, you will be facing issue while starting up SAP application.

Regards,

ARUN

[Reagan]

Hello Aditya

Those Jco parameters are not the ones to be manually entered in the configtool.

com.sap.security.core.ume.service is a service under services part in the Configttol and that contains the UME configuration to the backend system.

The problem you face is most likely due to a a bad JCO configuration.

It was possible to modify the JCO using Visual Admin in the older version of NW systems.

I believe you have a newer NW release system.

In that case enable the SAP* user and restart the J2EE system.

Once the Java stack is up then check the JCO's configured.

Check this link

Performing Further Configuration Steps - PI (Process Integration) - SAP Library

Regards

RB

[Former Member]

Hi RB,

You are right.Some changes has been done by one of my colleague in the JCO confg of this development box.I have reverted all the changes in parametrs suggested above and reset SAP* pass to default and took restart but again it is not starting.

Thanks

Aditya

[Reagan]

Were you able to locate those Jco parameters after connecting to the system with SAP* user ?

Regards

RB

[Former Member]

Hi RB,

I checked in default trace file it is again looking for same JCO parameters. can we manually reset these parameters in DB tables( if yes then in which table do dey reside).I am not sure from where these parameters are being read. I had maintained the right credential in configtool but it not being read from there.Ny idea in this situation?

Thanks

Aditya

[Former Member]

modifying the parameter or any kind of activity not supported by SAP unless stated by SAP. If you Java is not coming then you must change required parameter in configtool. As you mentioned you have modified the parameter but still same error, hope you are modifying correct parameter.

[Former Member]

Hi,

Since you are not able to start AS Java instance, you must edit UME-ABAP connection parameters in offline mode using configtool.

Go to Global server configuration -> services -> com.sap.security.core.ume.service

Check these properties:

ume.r3.connection.master.ashost

ume.r3.connection.master.sysnr

ume.r3.connection.master.client

ume.r3.connection.master.user

ume.r3.connection.master.passwd

Unlock the user on the target ABAP stack, make sure that correct password is maintained in UME configuration.

Finally save changes and restart AS Java.

Good luck : )

[Former Member]

Hi Artur/RB,

I checked in default trace file and found JCO_ERROR_LOGON_FAILURE.It is using user credential and other details as mentioned by RB above.I could find jco.client.user , jco.client.lang ,jco.client.sysnr , jco.client.client, jco.client.ashost ,jco.client.destination parameters in Configtools and have maintained it there.I am not getting option to add other parameters(listed in default trace file and also mentioned by RB).Now when I am restarting the server it is again giving authentication error.When I tried to logon to backend R3 server using this user id it gave me warning msg saying multiple failed logon attempt(although user is not getting locked and I m able to logon to system with same credential I have maintained in configtool).For confirmation I took multiple restart and after every restart when I logon to R3 system I get same warning msg of "multiple failed logon attempt" for this user id.I think when I am saving the password for same user id in configtool it is not getting updated/effective even after system restart.Any idea to resolve this.

Thanks

Aditya

[Former Member]

Hi,

I already gave you parameters to be checked for failing service.

Did you do that?

[Former Member]

Hi Aditya,

Did you manage to solve your problem ?

[Former Member]

Hi AS,

Sorry my reply was not posted properly.Actually I have added the parameters and their custom value in configtool (as they were missing) and took restart but again UME failed to start.I have used the user id and pwd information which I got from default trace file (also identified by RB in above post).Plz suggest if u have ny work around.

Thanks

Aditya

[Former Member]

Can you check in configtool and tell us what value is set in property

ume.persistence.data_source_configuration

under path

cluster-data -> Global server configuration -> services -> com.sap.security.core.ume.service

[Former Member]

Hi Artur,

My bad.whenever I reply to ur post it doesnot get posted properly.I have checked above mentioned parameter in configtool and it "dataSourceConfiguration_abap.xml".

Thanks

Aditya

[Former Member]

Hi,

Thanks for confirmation. That tells me you use ABAP user source, therefore all these properties

ume.r3.connection.master.ashost

ume.r3.connection.master.sysnr

ume.r3.connection.master.client

ume.r3.connection.master.user

ume.r3.connection.master.passwd

should already be available in service com.sap.security.core.ume.service

Having said that I see some misunderstanding. Previously you wrote

I have added the parameters and their custom value in configtool (as they were missing

So where did you add them?

Make sure that ume.r3.connection.master.passwd is correctly set for ume.r3.connection.master.user user .

Make sure that this user is not locked in the ABAP source system.

Review also remaining properties from above list.

Alternatively you might switch ume.persistence.data_source_configuration property to dataSourceConfiguration_database_only.xml and try starting system up. As soon as it's running - switch back to dataSourceConfiguration_abap.xml in UME Configuration (host:port/useradmin).

[Reagan]

Any valid information in the latest default trace ?

Regards

RB

[Former Member]

Hi RB,

Can you please tell me the which log file u r referring to? Is it sapstart0.trc ???

sapstart0.trc

------------------------------------------------------------------------------

(19595282) New Child Process created.

(19595282) Starting local Command:

Command:  jc.sapEPD_J01

           pf=/usr/sap/EPD/SYS/profile/EPD_J01_essmssdev

--------------------------------------------------------------------------------

Thanks

Aditya

[Reagan]

Hello Aditya

Check the latest default trace logs present at /usr/sap/EPD/J01/j2ee/cluster/server0/logs

Regards

RB

[Former Member]

Hi RB,

Yesterday I checked this log.I t was saying "JCO_ERROR_LOGON_FAILURE: Password logon no longer possible - too many failed attempts on srmdev sysnr 00" where srmdev is corresponding ECC system.Hence [com.sap.security.core.ume.service] service is not getting up.Since Portal is down I can not see which user id is used for the JCO connectivity.However I unlocked the SAP* id and error message changed to "JCO_ERROR_LOGON_FAILURE: Name or password is incorrect (repeat logon) on srmdev sysnr 00 ". Since the error msg is very long and repetitive I am pasting a section of it.In case u need the whole file I will attach dat.

#2.#2014 02 04 17:15:35:285#+00#Fatal#com.sap.engine.core.Framework#
com.sap.ASJ.krn_frm.000019#BC-JAS-COR#kernel.sda#C000B0000B43000D0000001000CF009C#13342550000002722##com.sap.engine.core.Framework#####e64d07608dbf11e38a56001a64a803a2#
#0#Service Runner [com.sap.security.core.ume.service]#Plain##
Critical shutdown with exit code [2150] was requested due to [AS Java cannot be started; core service [com.sap.security.core.ume.service] failed to start]#

#2.#2014 02 04 17:16:09:580#+00#Warning#com.sap.engine.core.session.Manager#
com.sap.ASJ.ses.cf0023#BC-JAS-COR-SES#kernel.sda#C000B0000B4300000000000A00CF00A0###com.sap.engine.core.session.Manager########Thread[main,5,main]#Plain##
Could not register com.sap.engine.session.mbeans.DatashareMBean. Exception is: java.lang.InstantiationException: com.sap.engine.session.mbeans.MonitorSessionsOpenMBean#

#2.#2014 02 04 17:16:35:309#+00#Warning#com.sap.engine.services.deploy.server.utils.container.ContainerInfoValidator#
com.sap.ASJ-dpl.ds-000463#N/A#tc~je~dbcontainer~srv#C000B0000B43000A0000000200CF00A0#13342550000002682##com.sap.engine.services.deploy.server.utils.container.ContainerI
nfoValidator#####0dd734768dc011e39859001a64a803a2##0#Event Processor [tc~je~dbcontainer~srv]#Plain##

Warnings:
   The [dbschemacontainer] container reports that is registered from [null] service, but really it is registered by [tc~je~dbcontainer~srv].
Solution: Please contact the container holder to fix these warnings.#
Warnings:
   The [dbcontentcontainer] container reports that is registered from [null] service, but really it is registered by [tc~je~dbcontainer~srv].
Solution: Please contact the container holder to fix these warnings.#

#2.#2014 02 04 17:16:41:003#+00#Error#com.sap.security.core.persistence.datasource.imp.R3Persistence#
com.sap.ASJ.umecore.000019#BC-JAS-SEC-UME#com.sap.security.core.sda#C000B0000B43000C0000000200CF00A0#13342550000002717##com.sap.security.core.persistence.datasource.imp
.R3Persistence#####0dd734768dc011e39859001a64a803a2##0#Service Runner [com.sap.security.core.ume.service]#Plain##
User Management Engine failed to connect to the ABAP backend system. Check that connection data are correct and the backend system is available. Error message: "com.sap
.conn.jco.JCoException: (103) JCO_ERROR_LOGON_FAILURE: Name or password is incorrect (repeat logon) on srmdev sysnr 00". Connection description: "destination PRDCLNT600
with properties:{jco.destination.auth_type=CONFIGURED_USER, jco.client.user=161698, jco.client.lang=EN, propertiesProvider=com.sap.security.core.server.destinations.pr
ovider.DestinationsProviderFrame, jco.client.auth_type=CONFIGURED_USER, jco.client.type=3, jco.client.sysnr=00, jco.client.client=600, jco.client.gwserv=sapgw00, jco.cl
ient.destination=PRDCLNT600, jco.client.ashost=srmdev, jco.client.cpic_trace=-1, jco.destination.peak_limit=10, jco.client.gwhost=srmdev, jco.destination.pool_capacity=
5, jco.destination.max_get_client_time=10000, jco.destination.expiration_time=300000}".#

#2.#2014 02 04 17:16:41:007#+00#Error#com.sap.security.core.persistence.datasource.imp.R3Persistence#
#BC-JAS-SEC-UME#com.sap.security.core.sda#C000B0000B43000C0000000300CF00A0#13342550000002717##com.sap.security.core.persistence.datasource.imp.R3Persistence.[cf=com.sap
.security.core.persistence.datasource.imp.R3Persistence][md=init()][cl=null]#####0dd734768dc011e39859001a64a803a2##0#Service Runner [com.sap.security.core.ume.service]#
Plain##
An exception was thrown in the UME/ABAP user management connector. This exception might be caused by incorrect user input. It is written to the trace to assist incident
analysis by SAP. Only messages also appearing in the log require administrator assistance.
[EXCEPTION]
com.sap.conn.jco.JCoException: (103) JCO_ERROR_LOGON_FAILURE: Name or password is incorrect (repeat logon) on srmdev sysnr 00
        at com.sap.security.core.persistence.datasource.imp.r3persistence.R3JCo700Proxy$Client700.getAttributes(R3JCo700Proxy.java:350)
        at com.sap.security.core.persistence.datasource.imp.R3PersistenceBase.doInitRfc(R3PersistenceBase.java:717)
        at com.sap.security.core.persistence.datasource.imp.R3Persistence.localInitialization(R3Persistence.java:257)
        at com.sap.security.core.persistence.datasource.imp.R3PersistenceBase.init(R3PersistenceBase.java:483)
        at com.sap.security.core.persistence.imp.PrincipalDatabagFactoryInstance.<init>(PrincipalDatabagFactoryInstance.java:526)
        at com.sap.security.core.persistence.imp.PrincipalDatabagFactory.getNewInstance(PrincipalDatabagFactory.java:205)
        at com.sap.security.core.persistence.imp.PrincipalDatabagFactory.newInstance(PrincipalDatabagFactory.java:155)
        at com.sap.security.core.persistence.imp.PrincipalDatabagFactory.newInstance(PrincipalDatabagFactory.java:217)
        at com.sap.security.core.persistence.imp.PrincipalDatabagFactory.getInstance(PrincipalDatabagFactory.java:141)
        at com.sap.security.core.persistence.imp.PrincipalDatabagFactory.getInstance(PrincipalDatabagFactory.java:96)
        at com.sap.security.core.InternalUMFactory.initializeUME(InternalUMFactory.java:209)
        at com.sap.security.core.server.ume.service.UMEServiceFrame.start(UMEServiceFrame.java:288)
        at com.sap.engine.frame.ApplicationFrameAdaptor.start(ApplicationFrameAdaptor.java:31)
        at com.sap.engine.core.service630.container.ServiceRunner.startFrame(ServiceRunner.java:165)
        at com.sap.engine.core.service630.container.ServiceRunner.startService(ServiceRunner.java:118)
        at com.sap.engine.core.service630.container.ServiceRunner.run(ServiceRunner.java:65)
        at com.sap.engine.frame.core.thread.Task.run(Task.java:73)
        at com.sap.engine.core.thread.impl5.SingleThread.execute(SingleThread.java:178)
        at com.sap.engine.core.thread.impl5.SingleThread.run(SingleThread.java:316)
Caused by: com.sap.conn.jco.JCoException: (103) JCO_ERROR_LOGON_FAILURE: Name or password is incorrect (repeat logon) on srmdev sysnr 00

Thanks

Aditya

[Former Member]

My concern is I don't know the user id and password ,maintained in JCO RFC.Since portal is not up how can I change the pwd and maintained that in corresponding ECC server.

Thanks

Aditya

[Reagan]

Here is the problem

User Management Engine failed to connect to the ABAP backend system. Check that connection data are correct and the backend system is available. Error message: "com.sap
.conn.jco.JCoException: (103) JCO_ERROR_LOGON_FAILURE: Name or password is incorrect (repeat logon) on srmdev sysnr 00". Connection description: "destination PRDCLNT600
with properties:{jco.destination.auth_type=CONFIGURED_USER, jco.client.user=161698, jco.client.lang=EN, propertiesProvider=com.sap.security.core.server.destinations.pr
ovider.DestinationsProviderFrame, jco.client.auth_type=CONFIGURED_USER, jco.client.type=3, jco.client.sysnr=00, jco.client.client=600, jco.client.gwserv=sapgw00, jco.cl
ient.destination=PRDCLNT600, jco.client.ashost=srmdev, jco.client.cpic_trace=-1, jco.destination.peak_limit=10, jco.client.gwhost=srmdev, jco.destination.pool_capacity=
5, jco.destination.max_get_client_time=10000, jco.destination.expiration_time=300000}".

com.sap.conn.jco.JCoException: (103) JCO_ERROR_LOGON_FAILURE: Name or password is incorrect (repeat logon) on srmdev sysnr 00

Update the correct password for the connection to the ABAP stack in the configtool and restart the system.

Regards

RB

[Reagan]

My concern is I don't know the user id and password ,maintained in JCO RFC.Since portal is not up how can I change the pwd and maintained that in corresponding ECC server.

I believe there is an ABAP stack connected to the system.

Here are the connection details:

destination PRDCLNT600
with properties:{jco.destination.auth_type=CONFIGURED_USER, jco.client.user=161698, jco.client.lang=EN, propertiesProvider=com.sap.security.core.server.destinations.pr
ovider.DestinationsProviderFrame, jco.client.auth_type=CONFIGURED_USER, jco.client.type=3, jco.client.sysnr=00, jco.client.client=600, jco.client.gwserv=sapgw00, jco.cl
ient.destination=PRDCLNT600, jco.client.ashost=srmdev, jco.client.cpic_trace=-1, jco.destination.peak_limit=10, jco.client.gwhost=srmdev, jco.destination.pool_capacity=
5, jco.destination.max_get_client_time=10000, jco.destination.expiration_time=300000

Regards

RB

[Former Member]

Hi RB,

Thanx a lot for the help.I checked these parameters in configtool and found it missing.I have maintained these details (for all those parameter I cud find in confgtool) however I could not find some of the parameters listed below in configtool.Plz tell me the exact path.I tried searching them using global search but I cud not .I took restart again same error "SfCJavaVm: exit hook is called. (rc = 2150

jco.destination.auth_type=CONFIGURED_USER,

propertiesProvider=com.sap.security.core.server.destinations.provider.DestinationsProviderFrame

jco.client.auth_type=CONFIGURED_USER

jco.client.type=3

jco.client.gwserv=sapgw00,

jco.client.cpic_trace=-1

jco.destination.peak_limit=10

jco.client.gwhost=srmdev

jco.destination.pool_capacity=5

jco.destination.max_get_client_time=10000

jco.destination.expiration_time=300000"

Thanks

Aditya

[Former Member]

check note 1794176 - J2EE System fails with "exitcode 2150" or

1945047 - AS Java stops with exitcode 2150