SAP HANA on VMware vSphere

[erik_rieger1]

SCN Content has been Migrated to 1DX NEW !

All content of the SCN space will be migrated to the Virtualization and Cloud Infrastructure Community and to the Virtualization and Cloud Infrastructure Wiki.

Therefore, the content of this page has been migrated to SAP HANA on VMware vSphere Wiki.

SCN Content Only up-to-date until October 4th 2016

By running the SAP HANA platform virtualized on VMware vSphere, SAP customers can leverage a industry standard data center platform, optimized for agility, high availability, cost savings, and easy provisioning. SAP customers will not only gain the ability to provision instances of SAP HANA in virtual machines much faster, but also benefit from unique capabilities like:

• Increased security and SLAs, e.g. through NSX or DRS.
• Live migration of running SAP HANA instances, with VMware vSphere® vMotion®
• Standardized High Availability, based on VMware vSphere® High Availability (HA)
• Built-in multi-tenancy support, through system encapsulation in a virtual machine (VM)
• Abstraction of the hardware layer
• Higher hardware utilization rates

These and other advanced features - to an large extend found exclusively in virtualization - lower the total cost of ownership and ensures the best operational performance and availability.  

News

• Since May 2016, SAP supports also SAP HANA on vSphere 6 deployments for production workloads. See SAP Note 2315348 and blogs.vmware.com for more details on what’s supported, which deployment options can get used and summary of the best practices.
• The support for vSphere 6 allows customers to increase the RAM to up to 4 TB (4080 GB) of existing virtual SAP HANA systems when migrated to vSphere 6. Beside increased RAM sizes, vSphere 6 supports also more vCPUs: Up to 128 vCPUs may now be configured and used by a single SAP HANA VM.
• Supporting more physical compute resources inside a VM ultimately provides more “power” to a virtualized SAP HANA system – this alone is worth considering an upgrade from a vSphere 5.5 to a vSphere 6.0 based SAP HANA environment.

Support Information

• Customers running SAP HANA on VMware shall follow the Architecture Guidelines and Best Practices for Deployments of SAP HANA on VMware vSphere.
• We have lately seen some issues with VMs configured with large virtual hardware resources and memory-intensive workloads becoming temporally unresponsive after vMotion has completed. We recommend to read VMware KB 144984 and configure hosts correspondingly.
• Noticing a certain interest in our CPU hot-add feature, we would like to highlight that the use of this capability also requires the OS and the SAP HANA database itself to be capable of leveraging the additional resources instantly. Please be aware that the use of CPU hot-add feature disables vNUMA and hence may lead to performance degradation.

The below table states capabilities, supported deployment options and best practices, like minimal vCPU count or maximal vRAM sizes for SAP HANA VMs on VMware vSphere as of May 2016. For additional reference, see SAP Note 1788665 et. al.

Capability / Option	vSphere 5.5		vSphere 6.0
	Supported in Production	Supported in non-Production	Supported in Production	Supported in non-Production
SAP HANA Scale-Up VM ≤ 1024 GB	Yes
SAP HANA Scale-Up VM ≤ 4080 GB	No		Yes
SAP HANA Scale-Out VM ≤ 1024 GB [1]	Yes		No	Yes
SAP HANA Scale-Out VM ≤ 4080 GB [1]	No			Yes
SAP HANA Multi-VM [2]	Yes		Limited [3]	Yes
SAP HANA Version	SPS07 and above		SPS11 and above	SPS09 and above
VMware vSphere and SAP HANA HA and Operation Features [4]
VMware HA	Yes
SAP HANA Host Auto-Failover [5]	Yes
SAP HANA System Replication	Yes
VMware FT	No
VMware SRM	Yes
vSphere vMotion	Yes
VMware DRS [6]	Yes
Supported HW configurations [7]
Supported SAP HANA Systems for VMware virtualization	Only SAP HANA and VMware certified two-, four- and eight socket Intel E7 v2 (Ivy Bridge) and later Intel processor-based server systems, as well as Intel Xeon E5 v3 and v4 based two-socket single node SAP HANA entry level systems, with a minimum of eight cores per CPU are supported.
Intel Xeon E7 CPU Support	Ivy Bridge, Haswell		Ivy Bridge, Haswell, Brodwell[8]
NUMA Nodes per server	4	8
NUMA Node Sharing	No	Yes	No	Yes
Enable Hyperthreading	Yes
Maximal RAM installed in server[9]	4/6 TiB		6/12 TiB
Supported Storage Configuration
Supported SAP HANA Storage Systems for virtualisation [10]	All SAP HANA TDI and VMware certified / supported storage solutions can get used.
TDI Storage KPI requested	Yes, per HANA VM	No	Yes, per HANA VM	No
SAP HANA Virtual Machine Configuration [11]
Max. VM size #vCPU	64		128
Max. VM size #vRAM	1024 GB		4080 GB
Minimal VM size #vCPU	All threads of a single CPU socket	10	All threads of a single CPU socket	10
Minimal VM size #vRAM	RAM of CPU socket	As sized	RAM of CPU socket	As sized
CPU reservations [12]	No
RAM reservation	Yes
Supported OS for virtualized SAP HANA
SLES 11 and 12 [13]	Yes
RHEL 6 and 7 [12]	Yes

Table 1. SAP HANA on vSphere supported capabilities and options

1. SAP support for SAP HANA scale-out deployments of SAP Business Warehouse (BW), powered by SAP HANA, on VMware vSphere as outlined in the table in context of SAP BW, powered by SAP HANA. Further supported workloads of OLAP-type (HANA Data Mart, HANA Data Warehouse) generated by SAP and non-SAP tools:
   • Documentation and / or tool-support for table distribution, sizing, partitioning, scale-out and how to configure SAP and 3rd party products on SAP HANA has to be provided by SAP, partners or customers resp.
   • 3rd party scenarios will not be tested by SAP. In specific cases, SAP HANA support for specific products or scenarios might be excluded by SAP in the future. If tested, then these will be listed in the SAP HANA on vSphere support note.
   • 3rd party tool support can be checked at http://global.sap.com/community/ebook/2013_09_adpd/enEN/search.html (select a partner --> details --> SAP Certified Solutions)
2. The defined KPIs for Data Throughput and Latency for production SAP HANA systems has to be fulfilled for each VM. SAP has released a special tool, the SAP HANA HW Configuration Check Tool, to measure if the used storage is able to deliver the required IO capacity, see SAP Note 1943937. 
3. Just like with the vSphere 5.5 SAP HANA support release in the beginning 2014, vSphere 6 supports currently only one production level VM that may get co-deployed with non-production level SAP HANA VMs. No resource sharing with other VMs is supported for production level SAP HANA VMs.
4. Generally all VMware vSphere features, like distributed switches are supported, no support for VMware FT for SAP HANA, FT may get used with an (A)SCS instance. vMotion and DRS are explicitly listed since running SAP HANA VMs can get moved from one host to another host while the application is online and users are connected and any negative impact needs to get excluded.
5. Only available with specific SAP HANA TDI Storage Vendors, see SCN page DOC-60470.
6. VMware DRS should get configured in “manual mode”
7. Both SAP HANA appliance and SAP HANA Tailored Datacenter Integration (TDI) delivery methods are supported for SAP HANA on VMware vSphere. Where the SAP HANA system has either been delivered pre-configured on certified SAP HANA appliances, as listed in SAP HANA Product Availability Matrix (PAM), with VMware vSphere hypervisor installed by SAP HANA hardware partner or the SAP HANA installation was done by an SAP certified engineer, qualified as "SAP Certified Technology Specialist - SAP HANA Installation" on SAP HANA certified hardware. It is recommended to verify the configuration with the SAP HANA hardware configuration check tool prior the installation.
8. Brodwell CPU vSphere 6 support in validation, yet not supported. Please check SAP support release support note 2315348 for an updated status.
9. Maximal RAM installed in physical server as specified by SAP, in the Certified SAP HANA Hardware Directory. The listed memory figures are the maximal memory configurations supported by VMware vSphere. Up to 6TB is supported for ESXi 5.5 Update 2 and later and up to 12 TB is supported with vSphere 6.0 on specific OEM certified platforms.
10. For details please refer to the Certified SAP HANA Hardware Directory and go to “Certified Enterprise Storage” configurations.
11. The maximum size of a virtual SAP HANA instance is limited by the maximum size of a virtual machine of the VMware vSphere release. Each SAP HANA instance / virtual machine is sized according to the existing SAP HANA sizing guidelines and VMware recommendations. CPU and Memory overcommitting must not be used.
12. Since production level VMs have to run on dedicated NUMA nodes, there is no need in configuring CPU reservations. Configuring CPU reservations is only required when production level SAP HANA VMs would share a NUMA node, which is currently not supported. Non-prod. level VMs do not require CPU reservations.
13. Both, SUSE Linux Enterprise Server (SLES) 11 and 12 and Red Hat Enterprise Linux (RHEL) 6 are supported operating systems when virtualized. Due to different scope and test coverage during corresponding validation activities, the underlying server must be certified for corresponding operating system, however.

SAP HANA on vSphere Best Practices

In a nutshell: SAP HANA follows general published vSphere Best Practices for databases:

• Set Memory Reservations for SAP HANA Virtual Machines
• Configuring Paravirtual SCSI Controllers and Network Adapters
• Right sizing of SAP HANA VMs to ensure local NUMA node memory access and high 2nd level cache hit ratios
• No NUMA node sharing for production SAP HANA VMs
• Enable Hyper Threading on the ESXi host
• Use dedicated networks for vMotion, management and client network
• Use vMotion and VMware snapshots during non peak times

Untill the new “Architecture Guidelines and Best Practices for Deployments of SAP HANA on VMware vSphere guide will be available from www.vmware.com, see the best practices and especially the configuration settings as described in the Best Practices and Recommendations for Scale-Out Deployments of SAP HANA on VMware vSphere document (page 71-73) for more detailed guidance. In addition, please ensure that following configuration parameters are used in addition to achieve optimal performance for SAP HANA:

• halt_in_monitor = “TRUE”
• idleLoopSpinBeforeHalt = “TRUE”
• Lat. Sensitivity – normal

Comparison: Physical vs. virtual SAP HANA deployment (as state of February 2016)

Capability	Physical SAP HANA	Virtual SAP HANA
SAP HANA Single Node Memory per node	128 GB – 4 TB RAM with 8 sockets	≤ 1 TB RAM per VM (vSphere 6.0 ≤ 4** TB per VM possible)
SAP HANA Scale UP SoH Memory per node	768 GB - 12 TB RAM with 16 sockets	≤ 1 TB RAM per VM (vSphere 6.0 ≤ 4** TB per VM possible)
SAP HANA Scale Out BWoH Memory per node	256 GB - 4 TB RAM with 8 sockets	≤ 1 TB RAM per VM (vSphere 6.0* ≤ 4** TB per VM possible)
Appliance Model	Yes	possible***, normally a TDI deployment
Server Hardware	Certified Appliances	Certified Appliances or TDI
Storage Hardware	Certified Appliances or TDI	Certified Appliances or TDI
Scale Up  Configuration	Supported	Supported
Scale Out Configuration	Supported	Supported
CPU Sockets per server	2 to 16 CPU sockets	2 to 4 CPU sockets (8 with vSphere 6)
Flexible HANA sizing's	No, T-Shirt sizing's	Yes
HANA DB Live Migration	No, not possible	Yes
Standby-Node / System Replicaktion / VMware HA	Yes / Yes / No	Yes / Yes / Yes
Multi-Tenancy / full tenant isolation	Yes, since SPS09 / No	Yes / Yes

* vSphere 6 for non-prod. use cases **vRAM configuration is not allowed to exceed installed memory of a physical server system and must follow SAP HANA sizing guidelines. ***Some SAP HANA HW vendors offer VMware vSphere as part of their SAP HANA appliance system. This is not required and optional. The customer can choose between a pre-installed appliance or the certified server configuration and may perform the installation of vSphere on its own.

VMware SAP HANA  System Health Check

SAP and VMware are offering joint services to help you design, deploy, and implement your virtual SAP HANA platform.

Examples of services which may be offered are:

• Pre-Call to discuss PoC, goals, timeline and success factors (SAP, HW Partner, Customer and VMware)
• Architecture review and introduction to the SAP HANA on vSphere Configuration Guidelines and Requirements
• Introduction to SAP HANA Hardware Check Configuration Tool (HWCCT)
• Online review session to check prepared environment like:
  • Host including storage and network configuration
  • VM configuration
  • Linux OS configuration

For more information on SAP Consulting and VMware Professional Services, visit vmware.com/consulting or get in contact with your local VMware contact.