8 Replies Latest reply: Jun 21, 2013 4:04 AM by Bill Ramos RSS

What do I do when HANA instance on AWS doesn't start correctly?

Bill Ramos
Currently Being Moderated

My AWS HANA was working just fine all last week and starting yesterday, the instance won't start.I've been stopping and starting the instance about twice a day. Rebooting the VM didn't seem to help.  Here are the symptoms as seen from the Linux shell after running the su - hdbadm command.

  • hdbsql - Running hdbsql -n imdbhdb -i 00 -u SYSTEM -p manager   - seems to work, but when I run a SQL statement, I get the error:

hdbsql=> select * from tpc.nation

* -10709: Connection failed (RTE:[89006] System call 'connect' failed, rc=111:Connection refused)

  • startsap - Running this command results in the following error -

imdbhdb:/usr/sap/HDB/HDB00/exe/python_support> startsap

    End program!

    Starting with NGAP 2.0 the startsap/stopsap scripts will be removed

and is no more available.

    For detailed information see Incompatible Change:

    https://wiki.wdf.sap.corp/wiki/display/LeanByD/IC_0152

  • SAP HANA Studio - I get the following error in the error log -

eclipse.buildId=unknown

java.version=1.7.0_09

java.vendor=Oracle Corporation

BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=en_US

Command-line arguments:  -os win32 -ws win32 -arch x86_64

Error

Tue Nov 06 18:42:01 PST 2012

Instance not reachable: SAPControl request failed

  com.sap.ndb.studio.sapcontrol.SAPControlException: SAPControl request failed

at com.sap.ndb.studio.sapcontrol.operations.SAPControlOperation.execute(SAPControlOperation.java:118)

at com.sap.ndb.studio.sapcontrol.SAPControl.getInstanceProperties(SAPControl.java:495)

at com.sap.ndb.studio.sapcontrol.SAPControl.collectProperties(SAPControl.java:522)

at com.sap.ndb.studio.sapcontrol.SAPControl.open(SAPControl.java:333)

at com.sap.ndb.studio.sapcontrol.SAPControlManager.getSAPControl(SAPControlManager.java:121)

at com.sap.ndb.studio.sapcontrol.SAPControlManager.getSAPControl(SAPControlManager.java:101)

at com.sap.ndb.studio.sapcontrol.SAPControlManager.getSAPControl(SAPControlManager.java:83)

at com.sap.ndb.studio.admin.model.system.impl.InstancesImpl.populate(InstancesImpl.java:401)

at com.sap.ndb.studio.admin.logfiles.jobs.CredentialsHelper.tryPopulateWithPasswordDialog(CredentialsHelper.java:133)

at com.sap.ndb.studio.admin.logfiles.jobs.SapstartsrvPrepareJob.populateEngineSapstartsrv(SapstartsrvPrepareJob.java:120)

at com.sap.ndb.studio.admin.logfiles.jobs.SapstartsrvPrepareJob.access$0(SapstartsrvPrepareJob.java:73)

at com.sap.ndb.studio.admin.logfiles.jobs.SapstartsrvPrepareJob$PopulateEngineCallable.call(SapstartsrvPrepareJob.java:35)

at com.sap.ndb.studio.admin.logfiles.jobs.SapstartsrvPrepareJob$PopulateEngineCallable.call(SapstartsrvPrepareJob.java:1)

at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)

at java.util.concurrent.FutureTask.run(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

at java.lang.Thread.run(Unknown Source)

Caused by: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Message send failed

at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.call(Unknown Source)

at com.sap.ndb.studio.sapcontrol.operations.SAPControlOperation.execute(SAPControlOperation.java:94)

... 17 more

Caused by: com.sun.xml.internal.messaging.saaj.SOAPExceptionImpl: Message send failed

at com.sun.xml.internal.messaging.saaj.client.p2p.HttpSOAPConnection.post(Unknown Source)

... 19 more

Caused by: java.net.UnknownHostException: imdbhdb

at java.net.AbstractPlainSocketImpl.connect(Unknown Source)

at java.net.PlainSocketImpl.connect(Unknown Source)

at java.net.SocksSocketImpl.connect(Unknown Source)

at java.net.Socket.connect(Unknown Source)

at java.net.Socket.connect(Unknown Source)

at sun.net.NetworkClient.doConnect(Unknown Source)

at sun.net.www.http.HttpClient.openServer(Unknown Source)

at sun.net.www.http.HttpClient.openServer(Unknown Source)

at sun.net.www.http.HttpClient.<init>(Unknown Source)

at sun.net.www.http.HttpClient.New(Unknown Source)

at sun.net.www.http.HttpClient.New(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)

at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)

... 20 more

 

Any help would be greatly appreciated.

Bill

  • Re: What do I do when HANA instance on AWS doesn't start correctly?
    Juergen Schmerder
    Currently Being Moderated

    Hi Bill,

     

    Try the following from the Linux shell:

     

    su - hdbadm

    ./HDB stop

    ./HDB start

     

    Let us know if that helps

    --Juergen

  • Re: What do I do when HANA instance on AWS doesn't start correctly?
    Jens Wachtel
    Currently Being Moderated

    I experience the same issues/error like Bill.

     

    However stop & start doesn't help in my case. I may have 'overloaded' the database with data on a data generation process. I misscalculated the available RAM space and I think this caused the indexsserver to crash.

    After ./HDB start I am seeing a very fast memory consumption for the indexserver on top statistics and suddenly all the memory is released again and this message shows up:

     

    ....

    StartWait

    FAIL: process hdbindexserver HDB Indexserver not running

     

     

    I suspect a specific table which I want to delete, but I am unable to access the server console nor HANA Studio.

    Is there any way to remove this manually on the harddisk and then reinitialize. I couldn't find anything about this. If this granularity is not possible, can I get rid of a complete namespace and then restart?

     

    Thanks in advance

    Jens

    • Re: What do I do when HANA instance on AWS doesn't start correctly?
      Bill Ramos
      Currently Being Moderated

      Hi Jens,

         I reported a similar issue a while back - see http://scn.sap.com/thread/3293248. Essentially I terminated the HANA One instance and created a new one - this time keeping track of the memory I was using. I also switched all my larger tables to use column versus row tables.

      Regards,

      Bill

      • Re: What do I do when HANA instance on AWS doesn't start correctly?
        Jens Wachtel
        Currently Being Moderated

        Your thread hits it.

         

        (Don't want to get too much Off topic. But the mentioned column vs. row table approach for very large tables (>100 mio rows.) is this a best practice? My use-case would rather fit to a row store.)

        Thanks

        • Re: What do I do when HANA instance on AWS doesn't start correctly?
          Bill Ramos
          Currently Being Moderated

          Hi Jens,

              My suggestion is to go with column table with that many rows. Remember - row tables need to completely fit into memory don't have any compression! You also saw the consequence of going too far . You also have to account for any indexes that you might have created for your memory consumption.

              HANA is fairly efficient at inserts and updates with column tables as long as you make sure you are doing the delta merge on a regular basis. If you run into memory pressure, HANA will page data back out to disk from memory. With HANA One on AWS, you can also optimize the log processing speed by creating a provisioned IOPS drive (solid state) and then mounting it to your HANA One instance. You can then configure HANA to use the new drive as the log drive. This is essentially how the HANA appliances are configured and the way SAP originally envisioned configuring HANA One. The problem is that a provisioned IOPS drive is much more expensive than a standard EBS Raid0 configuration and that SAP didn't want people to have sticker shock when they got their bill at the end of the month from AWS. But, that's the great thing about AWS - if you need the horse power - they can generally deliver.

          Regards,

          Bill

Actions