AGL Data Intelligence Program SAP HANA Innovation Award 2014

[Former Member]

SAP HANA Innovation Award 2014

Project Name: Data Intelligence Program - SAP HANA

Description:

AGL launched SAP’s HANA in-memory computing technology to successfully deliver high performance analytics and reporting capabilities to support key AGL business operations.

The program is a result of AGL’s continued focus in bringing innovation into the marketplace, based on leading edge technology from SAP.

Project Contact Details

Contact Person:	Darrel Egan	Angela Catt	Babu Murugesan
Role:	Program Director	Head of Commercial Operations	Project Manager
Email:	degan@agl.com.au	acatt@agl.com.au	BMuruges@agl.com.au
Street:	120 Spencer St
Post-code:	3000
City:	Melbourne
Country:	Australia
Phone:	0410 471 030	0412 691 718	0449611139

Implementation Partner Details

Implementation Partner Details:

Partner Implemented

Implementer Name:

Tata Consultancy Services Limited (TCS)

Implementer Contact: Babu Murugesan

Implementer Title: Project Manager

Project Details

Implemented Solution: SAP HANA

Go live date: 30th May 2013

The main reasons that AGL selected SAP are:

• Existing strategic investment in the SAP platform;
• Leading edge data storage, processing & analytics technology;
• Innovative best in class functionality and capabilities.

Estimated number of SAP users:

Professional: Analyst & Call Center

HANA has been released to 60 business users and 900 plus call center users.

Total Project Budget: $16.85M

Implementation Overview

Business Issue:

The Electricity Industry in Australia embarked on a major change primarily due to the rollout of Advanced Metering Infrastructure (AMI) followed by implementation of the National Smart Metering Standards. This resulted in the rollout of the smart meters replacing the existing basic meters in the state of Victoria.

AGL, like most utility companies has a high dependency on quality and timely information to be delivered, distributed and processed by its key information systems. The operational performance, availability and integrity of these information systems are vital in ensuring continued commercial viability of AGL. With all the opportunities and benefits Smart Grids and Advanced Metering Infrastructure (AMI) provides, it also comes at a significant cost and introduces substantial risk as well; the most critical of these risks is the impact of managing large data volume. It is important to note that over 95% of AGL’s customer base had basic meters i.e. meters that are manually read or estimated every quarter; with AMI, these customer meters will be automatically read every 30 minutes and the meter reads transmitted daily. This will result to 17,520 reads per year and 4,400 times more data to store and process.

In addition, other time-series data needs to be captured so the processing impact will be much higher than just the customer consumption readings. It is imperative that a scalable and sustainable IT environment be provided to support AGL’s business operations.

Therefore, the business focus was to build a high performance analytics and reporting tool capable of handling this huge volume of incoming reads whilst also providing the following:

• Increased accuracy in load forecasting and settlements processes which lead to more accurate bidding in the wholesale market and financial forecasting.
• Enhanced customer insight into the market data and advanced capabilities like profiling and customer segmentation.
• Significantly reduced processing time of analytical algorithms.
• Ensured accuracy of data.

Project Justification:

AGL’s core business processes and supporting IT systems had to be enhanced to support and comply with these new regulatory/Industry standards, rules and procedures to sustain the core business and remain competitive in the market place.

AGL being one of the early adopters of SAP HANA, now has the ability to efficiently perform accurate load forecasting, predictive analysis on smart meter data and customer’s consumption pattern (profiling and segmentation) to come up with better insight into customer’s consumption behavior in order to provide most appropriate product offers. The customer profiling and segmentation features have been achieved using the SAP BI reporting suite.

Implementation Overview - Project Scope

The capabilities of SAP HANA has been utilised to import, validate, cleanse and manage this huge tsunami of reads in one central data store. HANA has also replaced the legacy application’s functionalities and provides rich, high performing analytics and reporting capabilities to support key AGL business operations, for example:

• Load forecasting.
• Settlements.
• Customer segmentation and profiling.
• Validation of network billing process.
• Financial forecasting.
• Operational and Analytical Reports to check data integrity between various sources including ISU billing and the market.
• Enhanced business intelligence.

Apart from the meter reads, SAP HANA also stores the following essential data and process entities:

• Market data like electricity meter information and settlements data available from market operator.
• Actual and forecasted weather information from external providers like BOM and Weatherzone.
• Customer contract information from source systems like SAP IS-U and CRM.
• Regional demand information (NEM Regional Demand) from AEMO.

TCS as the strategic IT partner of AGL was engaged to implement SAP HANA (High Performance Analytical Appliance) solution along with Data Services and SAP BI tools through the Data Intelligence (DI) program for managing the smart meter growth in an efficient way and build an efficient reporting solution.

Resources, Timeline, Architecture

The Data Intelligence (DI) program involved TCS as the System Integrator to deliver end-to-end solution and SAP as the product support lead. Resources were based in Melbourne (Australia) and Kolkata (India) with the project operating from 2 separate locations, involving a peak team size of 65.

The DI program included 6 streams of work (Core Development, People Change Management, Risks & Dependencies, Technical Architecture, Testing, deployment and decommissioning).

The DI Program ran from October 2011 to June 2013 with SAP Australia being an integral part of the DI program Governance.

The SAP HANA system has been integrated with other SAP products like SAP Data Services, SAP BOBJ/BI, and SAP BW & Solution Manager.

As part of this implementation, there were technical challenges such as dealing with high volume of data and transactions. Also, as the product was going through a maturity life-cycle the project team had to retrofit several product versions during the course of the project. These were efficiently managed through thorough management and architectural governance throughout the duration of the program.

Project Execution

Project approach involved the completion of the Proof of Technology to agree solution design and to confirm the project delivery cost. A successful project go-live in May 2013 was achieved.

AGL and TCS have been highly innovative by introducing and utilising AGILE methodology for effective project management delivering 94 user stories to the business. The AGILE delivery methodology was used by releasing the primary and secondary business features under the SCRUM framework. This framework was driven by the Scrum Master and the Product Owner. Not only did this ensure that the business expectations were captured and delivered, but also made sure that the project team worked as a coherent unit to deliver the expected outcome.

The project was a perfect example of “One Team” culture where all partners and AGL worked as a unit even under extreme pressure to deliver. AGL, TCS and SAP were completely aligned in various decisions throughout the duration of the program. The entire team blended well pertaining to their roles. The team deployed 15 sprints into the production environment using the SCRUM framework.

A Project Management Office (PMO) was created within the project to manage risks, issues, dependencies, and schedule as well as a weekly critical path analysis.

Based on the forecast, TCS also delivered a remediation initiative to ensure an optimal memory usage in HANA.

Change Management was an integral and key element of the program as the project was one of the largest cross divisional change program implemented by AGL.

Project Results and ROI

AGL was the first utility company in Australia to leverage in-memory analytics platform (SAP HANA).

Key Tangible Benefits (business value created)

1. 1.    Portfolio Aggregation – 50 fold performance improvement. To run a 6 month aggregation, the implemented HANA solution takes around 1 hour compared to 50 hours in the legacy applications. This enables quicker visibility into the forecasting results and provisions faster decision making as well as decision changing capability.

1. 2.    Contract Aggregation – 20 fold performance improvement. To run an aggregation of 6 months contracts for all portfolios, HANA solution takes around 2 minutes. The end result of this process is a key input to the portfolio aggregation process. The quicker run time of this process in turn enables faster run of the portfolio aggregation process.

1. 3.    Faster load forecasting (from 5 weeks to 4 hours). Providing more accurate load forecasts on a more regular basis means traders can understand the portfolio position and trade. For instance a 1% improvement in accuracy provides a $2 million reduction in hedging costs while a more regular forecast will manage risk more effectively.

1. 4.    Impact factor:  Has application to individuals at all levels across EPM (Energy Portfolio Management) with useful improvements and opportunity also flowing into settlements (Wholesale Risk), debt collection, Retail analytics and strategy development and Group Strategy.

1. 5.    Settlement Reconciliation reporting – 60 fold performance improvements. The Reconciliation reports can now be viewed instantly (<1 minutes) compared to long running batch process (60 minutes) in the earlier legacy applications. Huge improvement in the user experience as they don’t need to wait for an hour to view the report.

1. 6.    Data integrity Reporting – SAP HANA is now the source of truth of meter data and other market data for merchant business. Reporting out of HANA has enabled the business to track even minor data integrity issues at the source saving considerable time and effort for analysis of any issues in complex downstream processes like portfolio aggregation.

1. 7.    Complete and timely billing – data integrity reports identify more unbilled customers and enable identification in a timelier manner. Identifying an unbilled customer earlier in the process means less revenue leakage.

1. 8.    Improved Load forecasting - 50 fold performance improvement. Enhancement of load forecasting models leads to reduced load forecast errors, Operational and Hedging costs.

1. 9.    The solution has empowered AGL staff by freeing the data. A single source of truth means all business units can access, use common language and refer to the same data, analytics and recommendations. 

1. 10.  A large focus of the program was to enable End-user self-service pre-defined views where Analytic and Calculation views were created for real time aggregation of data (Portfolio, Contract aggregation). Business users were also given access (POWERUSER) and training to create Analytic and Calculation views to leverage HANA Analytical capabilities for their day to day activities/analysis.

1. 11.  The use of SAP Data Services to load HANA has eliminated data latency and facilitates real time replication of data to maintain the data currency.  In addition, HANA helped us gain performance improvement of more than 1,000 times for aggregation job which helped us in getting aggregation results in real time in place of scheduled batch processes.

1. 12.  Database performance tuning reduction    Re-Designed Database to remove Concatenated Indexes. More partitions reduces the likelihood that all partitions will be uncompressed at the same time.  Smaller partitions will take up less memory when individually uncompressed. Database Monitoring: Monitor database footprint for ongoing growth, Average record size, Average daily growth, Compressed data size, Concatenated index size.

1. 13.  Advanced data management challenges overcome by HANA include:

• Structured and unstructured data   
• HANA‘s performance capability have been utilized to accommodate high volume of structured market feeds which include smart & basic meter data, master data and market reconciliation logs. Data has been replicated from different RDBMS (Oracle, SQL server ),BW and various file types (XML,CSV/txt)
• Data is loaded continuously into HANA through Data Services, which reduces the latency of data ensuring the most current data is available for analysis. 
• Connectors/API’s consolidated ……   
• SAP BODS have been used to handle high volume data influx in HANA. Close coupling features like Bulk loading, pushdown of BODS had been achieved in connection with HANA.
• Data types in HANA are Meter reads, Market data like electricity meter information and settlements data available from market operator. Actual and forecasted weather information from external providers like BOM and Weatherzone. Customer contract information from source systems like SAP IS-U and CRM. Regional demand information (NEM Regional Demand) from AEMO.
• Complex data analysis including low to high level aggregations, market reconciliation, segmentation & Revenue leakage are few of the advanced analytics that had been achieved

1. 14.  More Accurate Pricing - Ability to price tariffs based on critical peak pricing. By providing more competitive pricing, AGL will be able to acquire/retain approximately 10,000 additional customers per annum.

1. 15.  HANA has become a central storage of customer meter reads, providing a single source of truth. This has “freed” the data and enables all Business Units to talk on a common basis. 

1. 16.  HANA has enabled AGL to consolidation / decommission legacy systems thus speeding up business processes and reducing costs.

1. 17. The innovation delivered through the use of SAP HANA has enabled AGL to provide prescriptive analysis in addition to the descriptive and predictive analysis previously provided. Enabling AGL to act with foresight to new market signals.

1. 18.  Ability to pinpoint all data integrity issues, saving a lot of effort in analysis.

1. 19.  70% higher rate of reconciliation in monthly payments and receivables.

1. 20.  As all the data resides in RAM this results in faster report generation for a better user experience.

1. 21.  Now meter data is centralized, providing a single source of truth. Faster, more detailed reporting is improving forecasting and helping identify millions in potential revenue and unbilled services. Plus, greater pricing accuracy allows AGL to be more competitive, helping secure roughly 10,000 additional customers each year.

1. 22.  AGL has utilised HANA’s excellent data compression function to reduce the overall data volumes.  Compressing ratios of between 2 and 7 times have been achieved.   

Intangible Benefits

1. 1.    “Single source of Truth” across all AGL customers resulting in improved business processes, data quality, reporting, forecasting and budgeting. This has resulted in improved business efficiency and a more consistent and improved customer experience.

1. 2.    Much Improved data accuracy.

1. 3.    HANA data has now become the benchmark to SAP SMA (Smart Meter Analytics) in AGL.

1. 4.    Improved Master User Interface with wider scope and added security.

1. 5.    Data Integrity Reports to provide better insight into the key source data (Standing data, Contract data & Meter data)

1. 6.    HANA has been released to 60 business users and 900 plus potential users.

1. 7.    Advanced analytical capabilities for AGL business operations.

Trailblazer – The entry should explain how the solution uses SAP HANA to transform the business to empower the workforce and customers.  These improvements should be substantial rather than incremental improvements

AGL being one of the early adopters of SAP HANA, now has the ability to efficiently perform predictive and prescriptive analysis on smart meter data and customer’s consumption pattern (profiling and segmentation) identifying insights into customer’s consumption behavior. Customers are empowered to view their own consumption patterns as well as AGL segmenting and understanding the customer better.

Also accurate and more timely demand forecasting results have been achieved for wholesale operations using HANA capabilities in conjunction with tools like MATLAB.

Please see attached document for further details