SAP Application does not switch to the Standby DB if the Primary DB is down.

[Former Member]

HI All,

I have installed ECC6 EHP6 (distributed system) on Windows server 2008 R2.

• CI + ASCS on host SYBSERVER1
• Primary DB on host SYBSERVER3
• Standby DB on host SYBSERVER2

SRS & DR agent are installed on SYBSERVER3 & SYBSERVER2 based on “Co-Located Replication Topology: Central Instance on a Separate Server” method described in the Installation Guide.

The replication scenario is working fine.

I have prepared test case to test the replication scenario. The test was successful.

Test scenario for Replication:

• User ‘Test1’ was created in Primary DB (SYBSERVER3).
• User existence was checked in Standby DB (SYBSERVER2) in USR02 table. User ‘Test1’ was found. [select * from SAPSR3.USR02 where BNAME='TEST1'and MANDT='000'3. SQL query is successfuly able to show the TEST1 user details from USR02 table.]

Now I am trying to test the DR scenario and SAP application fails to start using the standby DB, if the Primary DB is shutdown along with SRS & DR Agent.

As per thesap note 1891560 - Disaster Recovery Setup with SAP Replication Server, I have updated the environment variables for SIDADM (SW1ADM) in SYBSERVER1 & SYBSERVER3

• dbs_syb_ha =1
• dbs_syb_server= SYBSERVER3:SYBSERVER2
• dbs_syb_port= 4901

Test scenario for DR:

Shutdown the primary DB, SRS & DR Agent on SYBSERVER3,  followed by restart of the SAP Application on SYBSERVER1. Standby DB on SYBSERVER2 is online

• SHUTDOWN SYB_BACKUP

         

• sp_hadr_admin ‘deactivate’ , ‘10’, ‘SW1’              

              

• SHUTDOWN
• Stop the services for SRS & DR agent on SYBSERVER3
• Now the Sap application is shut down and started again. The ASCS instance is up and running but work processes of CI fail to start. SAP Application does not try to connect to Standby DB (SYBSERVER2) rather it still tries to connect to Primary DB (SYBSERVER3) that is unavailable.

Attached is the log of Work process '0' for reference.

Can someone please advise why the SAP application does not automatically switch to Standby DB after restart even after the parameter dbs_syb_ha, dbs_syb_server, dbs_syb_port are set?

thanks in advance.

regards

Sai

dev_w0.txt.zip

[crisnormand · Accepted solution]

Hello Sai,

The sp_hadr_admin deactivate just prepares the primary site for failover, it doesn't execute the failover. After that, you need to go through few more steps until finish the failover.

Note that ideally the sp_hadr_admin stored procedure should be used in specific situations where for some reason you have some issue using DR Agent command and you need to identify which step fails, so you need to execute them one by one manually.

That's why I strongly recommend you to use the DR Agent command sap_failover for your test. It will execute all needed steps to failover , executing by himself required commands in the ASE and in the SRS in background.

Besides, the DR Agent is aimed to be used as interface to manage your HADR environment.

Please check KBA 2165143  - SAP ASE and SAP Replication Server: How to perform a planned/unplanned failover for instructions about how to perform a planned/unplanned failover.

HTH

Regards,

Cris

[Former Member · Accepted solution]

Hi Cris,

Thanks for the KBA 2165143, i will go through and try to do a test scenario for DR Planed/unplanned.

Will update once done.

thanks

Sai

[Former Member]

Hi Cris,

Which log files for Primary ASE are you referring to ?(E:\sybase\SW1\log)

I did not manually disable a repagent during uninstall process.

SAP Installer had prompted during uninstall process that repsvr.exe & dbsvr12.exe were still running. So I killed the work process manually and replication server uninstalled successfully.

But it is during the Installation process that I manually disabled a repagent. I got below error when trying to install replication server on Primary DB.

After this step, the SAP Installer further proceeded and is now stuck in another error

As suggested, I will use SWPM SP8 instead of 70SWPM SP8.

Shall I uninstall the replication server on both sites along with Standby DB using SWPM SP8 followed by a fresh installation?

Let me know any checks that i can run on the Primary DB to ensure its clean and consistent.

thanks

Sai

[Former Member]

Thanks Cris,

I am using 70SWPM SP08 (latest version). I guess the screen capture you provided is that from SWPM SP08 (for Netweaver 7.1 and higher). As I installed ECC 6 EHP 6 i was using 70 SWPM from the beginning.

I tried to stop DR agent and restarted the Primary DB server. Manually started the services DR agent and Sap Replication service. Bot services started successfully.

Tried to resume the Sap Inst but still ran into an error.

So, I had uninstalled replication servers on Primary and secondary DB (4th option tear down) followed by Uninstalling of Standby DB.

I have now

1. installed Empy Ase DB on Standby DB successfully

2. Sap Replication Server on Standby DB successfully (without configuration checkbox enabled) successfully

3. Sap Replication Server on Primary DB (with configuration checkbox enabled) --> currently in progress in error

Installation of replication server on Primary DB ran into an error which i managed to resolve.

errors in the log:

hence manually disabled the replication agent for 'master' database and installation proceeded further.

But now, SAP Inst ran into a new error:

I have checked the info from following log files from Primary but cuold not find

sapinst_dev.log

SW1_REP_SYBPRIM.log

SW1_REP_SYBPRIM_RSSD.out

SW1_REP_SYBPRIM_RSSD_ra.out

Based on the info. from SW1_REP_SYBPRIM.log, looks like there is an issue related to last commit.

Can you advise on the current error.

thanks

Sai

SW1_REP_SYBPRIM_RSSD_ra.TXT.zip

SW1_REP_SYBPRIM_RSSD.TXT.zip

SW1_REP_SYBPRIM.TXT.zip

[crisnormand]

Hello Sai,

Error is coming from primary ASE, could you upload it?

Your images are unfortunately not visible (only blank sace with a cross is showed).

However if you had to manually disable a repagent, then the complete uninstall was not done. And I suspect the problems you have are directly related to the fact that your ASE primary is not 'clean' from previous installation attempts as it should.

Therefore I encourage you to use the last available SWPM SP8 for your installation. SAP installer has evolved for SRS installation and HADR setup and as explained the only options documented now are those you can see in my second image.

Besides, since a while the SRS installation option is independent of the system you have (as showed in lastest SAP installer) so it is not a problem to use the most recent SWPM SP to proceed to install and configure your HADR env.

Regards,

Cris

[Former Member]

Hi Cris,

I have used the Option 'Tear down disaster recovery environment including replication software and ASE replication configuration' to uninstall Sap replication server on both Primary and Standby DB.

Post which, I had installed the replication server successfully on Standby DB and now trying to install the replication server on Primary DB.

Attached are the logs:

SW1_REP_SYBSERVER3.txt [E:\sybase\SW1_REP\SW1_REP_SYBSERVER3\SW1_REP_SYBSERVER3.log] --> Primary

SW1_REP_SYBSERVER2.txt [E:\sybase\SW1_REP\SW1_REP_SYBSERVER3\SW1_REP_SYBSERVER2.log] --> Standby

Is there a wrong approach followed to uninstall replication server software using the option 'Tear down disaster recovery environment including replication software and ASE replication configuration' in Primary and Standby DB?

thanks

Sai

SW1_REP_SYBSERVER2.txt.zip

SW1_REP_SYBSERVER3.txt.zip

[crisnormand]

Hello Sai,

In both SRS logs we can see the same error:

F. 2015/07/01 17:45:49. FATAL ERROR #1030 GLOBAL RS(GLOBAL RS) - \errhand.c(1168)

Open Server error: Error: 16029, State: 0, Severity 20 -- 'Failed to start any network listeners'.

Could you check in both sites with netstat -an if the ports assigned to the SRS (check their sql.ini files to get the ports configured) are already in use?

Concerning the option selected, it looks fine. I assume you are using an old version of SAPinst as I didn't see these 4 options before.. with lastest versions we see only two options (which by the way are the documented options to uninstall completely or teardown an HADR env in the SAP Systems Based on SAP NetWeaver ABAP Installation Guide) and they are under option 'Generic Installation Options':

[Former Member]

Hi Cris,

I tried to install Sap replication Server on Primary DB. But the installation ran into a error.

On further analysis from the logs of SAPInst_dev.log and agent.log (E:\sybase\SW1_REP\SCC-3_2\log), the replication server agent is found not active.

I tried to manually start the services in services.msc, but the service does not start

further checks into the logs reveal that the replication server agent was stopped by the installation process to perform the action 'Set or Release secondary truncation point'.

Post this action, SAP Installer checks for  the replication connection to server 'SW1_SYBSERVER3' and database 'SW1' which fails.

Can you please advise here. Attached is the agent log for reference.

Thanks

Sai

agent.txt.zip

[crisnormand]

Hello Sai,

Did you uninstall completely the previous SRS installation or it was a teardown?

As the failing command is from SRS, could you please upload both SRS -primary and standby- logs?

(path:

Ÿ\sybase\<SID>_REP\<SID>_REP_primary\<SID>_REP_<primary>.log (on primary)

Ÿ\sybase\<SID>_REP\<SID>_REP_standby\<SID>_REP_<standby>.log (on standby)

)

Regards,

Cris

[Former Member]

Hi Cris,

Thanks for the clarification. I will follow the steps a per KBA 2165143.

Currently, I have uninstalled the entire setup and re installing the Standby DB, SRS on both Primary DB and Standby.

Once the setup is done, I will do a test scenario for the DR as per 2165143.

Will update you again.

Thanks

Sai

[Former Member]

Thanks Cris for the clarification on  dbs_syb_xxx environment variables.

I have gone through the KBA 2165143 and the associated Note 2089569.

There are contradicting statements in these notes on when exactly the rematerialization should occur.

Please see below.

Note 2165143 mentions to reconfigure the former primary as a new standby in step 4 and followed by rematerialization in step 7.

But Note 2089569 mentions to rematerialize the old primary before it is designated as valid standby.

This contradicts to the sequence of steps mentioned in the note 2165143

Any thoughts on these contradicting statements from the above notes?

thanks

Sai

[crisnormand]

Hello Sai,

Thank you for your feedback regarding the KBAs. I agree with you that the expression 'designated as a valid standby site' in KBA 2089569 could cause some confusion, we'll modify the text accordingly.

The KBA 2165143 has the correct sequence of steps, you should always rematerialize the old primary after an unplanned failover.

Regards,

Cris

[Former Member]

Cris,

I have  doubt regarding the parameters for <SIDADM>.

As per thesap note 1891560 - Disaster Recovery Setup with SAP Replication Server, I have updated the environment variables for SIDADM (SW1ADM) on CI + ASCS (SYBSERVER1) & Primry DB (SYBSERVER3)

Should these parameters be maintained for SIDADM (SW1ADM) in Standby DB (SYBSERVER2) too? In the Sap note 1891560 there is no specific mention of the systems where these parameters needs to be maintained.

• dbs_syb_ha =1
• dbs_syb_server= SYBSERVER3:SYBSERVER2
• dbs_syb_port= 4901

Please advise.

thanks

Sai

[crisnormand]

Hello Sai,

The SAP note 1891560 assumes you have:

=> a primary site - the SAP application (i.e the SAP ERP) running with SAP ASE -

=> and a standby site, where you have installed the SAP ASE, via System Copy, so you have a /sapmnt but no SAP application.

Setting the environment variables dbs_syb_xxx will allow the SAP application to re-direct the client connections to the former standby ASE after the failover.

So under the scenario described above, you don't need to set these environment variables at standby site as you are not supposed to have a SAP application there.

Regards,

Cris