The SAP in-memory computing engine that resides at the heart of SAP HANA is an integrated database and calculation layer that allows the processing of massive quantities of real-time data in main memory to provide immediate results from analyses and transactions.
Like any standard database, the SAP in-memory computing engine supports industry standards such as SQL and MDX but also incorporates a high-performance calculation engine that embeds procedural language support directly into the database kernel. This approach is designed to eliminate the need to read data from the database, process it and then write data back to the database.
Technical Proof Points Show Unprecedented Results With Customer Data
The SAP in-memory computing engine delivers technical breakthroughs at the most fundamental levels such as CPU core utilization and massively parallel processing across nodes. “Working with large customer data sets from the beginning helped tremendously to speed up the development process,” said Dr. Hasso Plattner, co-founder of SAP and chairman of the SAP Supervisory Board. Through this work with customers during the SAP HANA pilot phase, SAP has been able to demonstrate fundamental improvement in three areas:
- Speed: The SAP in-memory computing engine uses an in-memory columnar store to achieve breakthrough performance in scan, grouping and aggregation operations that are the heart of analytics. It has the ability to scan 2 million records per millisecond per core and over 10 million complex aggregations calculated on the fly per second per core. These results were attained with real customer data running on standard Intel processors. This performance has the potential to transform business processes. For example, as part of a consumer product goods customer proof of concept, SAP has implemented a real-world scenario on SAP HANA that demonstrates the ability to perform arbitrarily complex queries on over 450 billion records in a matter of seconds. Update, insert and data load performance also shows dramatic improvement relative to traditional systems, as no aggregates or database indices are maintained and all runtime data operations are conducted in memory with only logging executed against high performance solid-state disk (SSD) drives. In insert- only mode, the data base is lock free and can run multiple insert threads. In addition to the columnar store, the included row-store provides high transactional performance for small, read-only tables and single-row inserts.
- Scalability: The core engine of SAP HANA has been designed ground-up around a multi-core architecture and implements dynamic parallelization and dynamic partitioning, for both OLAP and OLTP workloads, via adaptive, cache-aware algorithms. As a result, performance scales linearly across not only blades but the number of cores per blade. Current analyses indicate full parallelization at 1,000 cores and beyond. In addition, extensibility is much easier as new attributes can be added on the fly. The implications are clear: a future- proof technology that will continue to provide breakthrough price performance as server core density continues to increase exponentially.
- Compression: Lastly, the SAP in-memory computing engine employs advanced compression algorithms and data structures that minimize the memory footprint required to run the system while still maintaining full support for OLTP workloads. The same 450 billion record system referenced above was implemented on less than three terabytes of physical memory. In addition, several application concepts, such as ageing, can be used to achieve further compression in-memory.
In-Memory Computing Engine and Native In-Memory Applications to Transform Enterprise Software Industry
“In addition to providing breakthrough value in analytics, the in-memory technology in SAP HANA enables entirely new applications that solve previously unaddressed problems,” said Dr. Vishal Sikka, Executive board member, Technology and Innovation, SAP AG. “By combining new software development techniques and programming models with design-thinking, beautiful new products at unprecedented speeds are now possible.” SAP has used its market-leading understanding of how enterprise applications interact with the database to consolidate key parts of the application layer such as business logic and object frameworks. This optimization is reflected in further substantial performance gains. For example, during the SAP HANA pilot phase, SAP has implemented core application scenarios such as Dunning with a 1,200x improvement in performance. Many customer application scenarios that currently take over two or three hours are now running in less than two or three seconds.
The SAP in-memory computing engine contains an integrated programming environment that allows the easy creation, inclusion and extension of native business functions that can be defined in a variety of languages such as SQL scripting, C++ and in the near future, Project R and Java Script. The SAP in-memory computing engine offers a unified information modeling design environment and can access data from both non-SAP and SAP sources. Thus, the optimization and simplification the SAP in-memory computing engine provides does not come at the cost of flexibility.
For more information visit the in-memory homepage on SDN, and watch the following TechEd Live interviews:
Follow in-memory computing technology and SAP HANA on Twitter at @SAPInMemory, and join the conversation at #hana, #inmemory and #sapteched.
