1 2 Previous Next

Josh's BI Ravings

17 Posts

I recently blogged and shared a video walkthrough of deploying an SAP Analytic Content Bundle, covering the Insurance vertical. If you haven't had a chance to read it yet, you can take a look here. In that video, I promised to record a second episode which covered the Telco Analytic Content, which was more in-depth and based on an SAP PowerDesigner model.


Well, here for your viewing pleasure, is Episode 2 - Telco & Utilities Analytic Content.



The steps I undertook before the video were:


  • Install a trial of PowerDesigner (available here)
  • Install a trial Sybase IQ (available here)
  • Download the Telco Analytic Content (available here)


As a reminder, these Analytic Content bundles are pre-built industry content are free, and provide an excellent starting platform as a data warehouse.  Let me know what you think of the Analytic Content bundles in the comments, and any feedback on the video.

Last week I had the opportunity to participate in and present at SAPInsider's BI2013 event, held in Amsterdam 11th to 13th June 2013. It was a big event with five sub-events all located together - Financials, HR, CRM, GRC, and Business Intelligence. With 1,500 people in attendance total, and representing 30 countries - it was definitely a great learning and networking opportunity! The conference was held at the RAI Conference Center, which was an interesting sight when I first arrived.




In this post I want to share some of the key things I took away from the conference, and for a different perspective you can also see my 'stream of consciousness' thanks to Twitter and Storify here: http://storify.com/josh_fletcher/bi2013-amsterdam. Without further ado...


Keynote: Driving Business Transformation through Technology Innovation - Steve Lucas, SAP


Steve (@nstevenlucas) kicked off the event with an engaging and humorous keynote. His message was on SAP enabling the connected enterprise, as a key part of the internet of everything. With so many manual processes in organisations, Steve discussed how these manual processes, with people as key decision makers in the middle, can hinder automation and the improvement of these processes. Of course not all processes can be fully automated, but Steve communicated the SAP vision of working towards a 'connected enterprise'.


Some of the key things I noted from the keynote:

  • Steve highlighted how much SAP has transformed, noting that 'Database & Technology together with Analytics now accounts for 50% of SAP's annual revenue!'
  • Steve asked Anders Reinhardt, Global BI Manager of VELUX, to provide an overview of their vision for a connected enterprise. VELUX manufacture roof windows and sell them globally, and are moving towards smart roof windows that will be connected to other parts of the house to assist with managing the internal climate of houses.
  • Also highlighted was the example of Disney, who now deliver real-time user feedback using smart wrist bands for patrons, which let Disney crowd-control (for example, if a ride is broken, they can redirect patrons to other rides which are similar to their previous rides) and provide a better user experience - again part of a connected enterprise.




British American Tobacco's Lessons from it's global implementation of BI4 and BW7.3 - Mark Cooper, BAT


Mark Cooper (@mcbobj) from British American Tobacco (BAT) delivered a fantastic case study on a global implementation of BI4 and BW7.3. Unfortunately I missed the first few minutes introducing the history up to the implementation, however Mark was comprehensive in their tool choices and implementation strategies.


Highlights included:

  • BI requirements were separated into Analysis, Formatted Reporting and Dashboards, and BAT respectively selected Analysis for OLAP, Crystal Reports For Enterprise and Xcelsius to deliver against these in majority of deliverables.
  • Mark recommended always keeping one eye on the SAP roadmap and keep up-to-date, as this may impact a technology choice sooner than expected - an example that Mark gave was Zen now named Design Studio, which they will move towards over the next few years.
  • One of the biggest barriers that BAT have faced in respect to user adoption has been the different User Interfaces that the business experiences when having to switch between the above three tools.
  • After discussing some challenges, Mark also spoke on their positive experience with Analysis OLAP, mentioning that the product always worked throughout patch challenges, and was very robust for a 1.0 product. Feedback from BAT developers, finance and operations users were all positive, and Mark summarised this in a statement 'The "Analysis" family of products is a good choice for SAP NetWeaver BW customers in BAT's opinion.'
  • Commentary was a hard and fast requirement for BAT, and together with SAP Consulting, a solution was developed using Knowledge Management and Web DynPro. This met BAT's requirements and enabled them to deliver commentary throughout the BI solution.




BI Template Best Practices - Simon Stieper, VELUX


Simon (@simonstieper) is the BI Manager at VELUX and went into great detail on VELUX's approach to BI using BusinessObjects.


Simon shared VELUX's observation of how the typical BI user at VELUX has changed over the last decade, and I found this part very interesting:

  • 2005 - typical BI user was a 'controller', an SAP expert who used Excel extracts to do reporting and analysis on one process
  • 2010 - typical BI user was an 'analyst', who used web-based tools to analyse multiple sources
  • 2013 - typical BI user is now focused on 'self-service', and they are not afraid to learn multiple tools and expect high performance


Simon then went on to explain VELUX's approach to BI and the principles they have chosen to support self-service BI:

  • Data Transparency
  • External Data
  • Reporting Flexibility
  • Multiple Tools


The BusinessObjects tools that VELUX selected are:

  • Web Intelligence - used for day-to-day reporting
  • Analysis Office - for analytical requirements
  • Xcelsius - for management reporting
  • Business Warehouse - main data warehouse
  • Data Services - for integration of external datasets


VELUX have also implemented HANA, and will be rolling out SAP Predictive Analysis shortly as well. There was lots more in this presentation, but the other thing that caught my eye was VELUX utilising an internally-developed Web Intelligence Design Guide, which is based around a Web Intelligence template. The Design Guide itself is 50 pages, and covers typography, naming conventions, formats, layouts and use of variables and report settings.




Confessions of a BW Architect - How DuPont Leverages the Common Semantic Layer - Jason Kehl, DuPont


Jason (@jasonmkehl) had the opportunity to close out the conference's last session with a bang, and he delivered. I really enjoyed his presentation on a high priority reporting scenario that DuPont encountered and needed to resolve.


DuPont identified a requirement to provide real-time reporting and analysis on top of 9 disparate plant systems (running on Oracle), with a dataset integrated back to BW. The business allowed a whole 2 WEEKS for IT to deliver this, which Jason and the team managed to do.


The solution delivered really highlighted the new common semantic layer and it's embedded Data Federator engine. Firstly the required BW InfoProvider was mapped into a multi-source universe. Next, the 18 different Oracle connections (9x plants with both Current and Historical databases) were added, and two super-large Derived Tables were added for the Current and Historical datasets.


The universe was finalised, but initially performance was terrible. Jason then related that, with the help of an external consultant, they were able to leverage the Data Federation Administration Tool to highly tune the multi-source universe.


The result? A smart universe which knew to choose 1 connection out of 19 to query based on the report selection parameters, achieving sub-second performance for those reports. The universe also supported much larger report queries which combined data from all 9 plants (previously impossible) with BW data.






There were several other sessions I attended that were of great value, but unfortunately this blog post would have grew way too large to fit it all in.  I also enjoyed:

  • R and SAP Predictive Analysis - John MacGregor, SAP
  • Three Reports, Three Connectors for BOBJ and BW - Jason Kehl, DuPont
  • Information Steward Tips - Don Loden, Decision First
  • Design Studio Overview - Ingo Hilgefort, SAP


Lastly, the SAP Analytics team did a great job of running a Social Media Lounge at the conference, and plenty of interviews took place. You can view them all here: http://www.youtube.com/sapbusinessobjects.


Below are the two interviews I managed to squeeze in, with Jason Kehl from DuPont and Don Loden from Decision First (@donloden).



Recently the Diversified Semantic Layer crew recorded a podcast with Jayne Landry (VP for Solution Management in the Analytics Team) focusing on the newly released Analytic Bundles.  If you haven't had a chance, head over to DSLayer.net to listen to the podcast.  These bundles are pre-built industry content using BusinessObjects BI4, and provide an excellent starting platform as a data warehouse, universe, as well as dashboards and reports for your business.


Why else should you care?  If you are an existing BusinessObjects customer, and running (or thinking of upgrading to) BI4 - guess what?  These Analytic Bundles are provided FREE.  Even if you aren't a good fit for one of the industries, these provide a sample set of universes and reports that showcase what is possible with the tool.  You may even be able to reach out to Jayne and discuss if there is an opportunity to develop other industry-specific bundles.


As an extension of this topic, I wanted to do a walkthrough of the process to deploy an Analytic Bundle (preview: it's super quick and easy!!), what it looks like from a deployment model, but most importantly - how it can easily be extended.


Without further ado, here is the walkthrough video for your enjoyment.



In our next episode, we will examine the Analytics Data Model for Telco & Utility, which leverages SAP Sybase PowerDesigner.


To look at demos of each of the Analytic Bundles (and/or download them), you can do so here.

We are now only two weeks out from BI2013 in Amsterdam, and I hope you are getting as excited as I am!


I have the privilege to deliver three presentations whilst in Amsterdam.  As always, I'll be active on Twitter so please feel free to engage me in person or on-line. Below you can find a preview of what my sessions will contain, including some useful tips!


An Introductory Guide to SAP's Enterprise Information Management (EIM) Solution Portfolio

Wednesday June 12th at 12:15pm


SAP's Enterprise Information Management, or EIM, portfolio is quite extensive and underpins SAP's Real-Time Data Platform. In this presentation, I focus on two of the key applications, Data Services and Information Steward, which are used for data integration, cleansing and data quality management - plus lots more!


This presentation will help introduce these tools to attendees who may not be familiar with them, so will also include lots of demonstrations of key functionality, like data profiling, view impact & lineage analysis and transforming data for loading into HANA.


As for tips & tricks, Data Services can be highly tuned.  Some Data Services developers aren't aware of the Degrees of Parallelism setting, which allows you to specify how many CPU cores the application can use for a particular dataflow.  This one setting can create a large performance increase in your ETL jobs.


Come along to my presentation for several more tips & tricks.


Best Practices for Designing Universes using the Information Design Tool (IDT)

Wednesday June 12th at 2:30pm


The Information Design Tool is still relatively new for many organisations, and so this presentation will focus on what's new to the Information Design Tool, as well as new features of the unx universe.


But more importantly, I discuss best practices in universe design, which apply to both the legacy unv universes, and the new unx universes. In my opinion, the universe designer has one of the most important jobs in a BusinessObjects team. A poorly-designed universe, the interface to all their information, can turn users off using the system. In contrast, a best-practice designed universe answers all business queries, is easy to understand for non-technical users, and performs very well.


Learn how to design universes with best practices in mind in this presentation.


Proven Design Techniques to Improve Formatting, Drill-down Navigation, and Performance of SAP Crystal Reports

Thursday June 13th at 10:15am


The title is certainly a mouthful, but this presentation is packed full of really useful Crystal Reports techniques that I've come across and learnt over my 12 years with Crystal Reports. We will cover techniques such as form layouts, dynamic suppression and advanced formulas. 


Did you know that when developing reports, the new Crystal Reports for Enterprise is very close to feature parity with Crystal Reports 2011? My presentation will use Crystal Reports for Enterprise for several demonstrations, and we will walk through each advanced technique.


Crystal Reports sometimes can be forgotten among all the new more interactive tools, however it is still the most powerful report development application in the BusinessObjects suite.  If you haven't used it before, I encourage you to give it a try!


See you in two weeks!

Re-posted from my original blog post in 2009 from Geek2Live.net


I recently had to provide a list of best practices in Universe Design – partly compiled from the SAP BusinessObjects training material, as well as my own experiences.  I believe that the universe is the foundation of the BusinessObjects system.  With poor universe design, required reports are often difficult or even impossible in some cases.


Poor universe design can also lead to wrong data, non-optimal SQL generation, incompatible objects and/or Cartesian products (or very wrong data).


I've categorised the best practices into two groups, End-User and Cosmetic methods, as well as Functional or Optimisation methods.


End-User / Cosmetic

  • The universe should be created from the end-user perspective, not from a developer’s perspective. This should be kept in mind whether designing new universes, or modifying existing universes.
  • All metadata (class names, object names and descriptions) should use business terminology. This allows the end users to easily use the universe without understanding the underlying database structure. It also speeds up development time, as header cells do not need to be edited. Users can then also search the universe when building a report to quickly find an object. For example, instead of calling a dimension object ‘Commit Cuml’, it should be called ‘Cumulative Commitment’.
  • All development objects should be removed before promoting to Production. Universes should not contain old, legacy or development objects (and classes) – these should be removed before promoting to Production. If absolutely necessary, these should be hidden and placed in a ‘Legacy Objects’ class so no mistake can be made regarding their status.
  • Objects should not be both dimensions and measures unless absolutely necessary. This is confusing to the user, and unnecessary. Providing row-level data is possible by introducing a unique object, like the ID.
  • Object formatting should always be specified, especially with measures. Formatting like dates, currency and so forth is important to provide objects that the user doesn't need to format themselves.
  • Date fields should be expanded using SQL functions, and split into separate objects like Calendar Year, Calendar Month, Calendar Month Name etc. A library of date functions for both T-SQL and PL-SQL should be built up and re-used where possible.
  • Classes and objects should be based on user requirements, not table-centric. Classes might contain objects from many different tables, and should not be based on the columns within a table.
  • Custom hierarchies should always be defined and done according to the end-user’s requirements. This prevents users from drilling using default hierarchies, and allows control over what users can drill into.
  • Pre-defined conditions should be used to accelerate report development and encode difficult query filters into an object that the user can use easily.
  • Object names should be unique, and never repeated, regardless of the object classification.


Functional / Optimisation

  • SQL aggregation functions should always be present in measures. All measures should have both a SQL aggregation function (ie SUM, COUNT and AVERAGE), then a projection aggregation function assigned (respectively SUM, SUM and NONE). This ensures that GROUP BYs are used and the processing power of the database is leveraged, returning fewer rows quicker. This also has additional performance benefits at the data provider level, where less RAM will be used on the server, and lower network traffic of data.
  • Tables should not be unconnected. This will generate a Cartesian product for the end-user, and leads to very confusing universes. If tables have no way of being joined at all, they should be placed in separate universes.
  • Left outer joins should be avoided where possible. This is sometimes impossible with transactional schemas, but left outer joins lead to NULLs and slower SQL queries. If left outer joins are necessary, universe objects should contain ISNULL functions or similar so that users aren't presented with NULL values when using objects.
  • Cardinalities on joins should always be set to allow for loop and SQL trap detection, this includes self-joins that should be marked as 1-1. Always set cardinalities manually, not automatically.
  • Contexts should be used, and if too complex, the universe should be split up. This allows for path navigation and allows a universe to have more functionality.
  • Loops and SQL traps should always be resolved to avoid Cartesian products and incorrect data. This is a very important component of universe design, especially with transactional schemas.
  • Integrity checks and refreshes of the database structure should be done regularly to ensure the universe has the up-to-date view of the database.
  • Views should be used in preference to derived tables, and kept to a minimum. This ensures optimal SQL generation.
  • When aliasing tables, the original should be placed in the top left of the schema window and noted with ‘Aliased Table – Do Not Remove’. This will ensure that an original table isn't removed by mistake, which will also remove all aliases of the table as well.
  • The query limits should always be set, to ensure that runaway queries or Cartesian product queries don’t bring down the database. Both length of query time and number of rows should be set. The limits will differ per database.
  • Multiple SQL statements for each measure should be turned off, to generate optimum SQL.
  • Index awareness should be used where possible, to generate optimal SQL.
  • List of values should be disabled for all dimension and detail objects that don’t require LOVs, to avoid users doing queries like all IDs for a given table.



I received quite a few comments on additional best practices, so I've included them here.

  • Lock the universe when editing to prevent other users from editing the same universe.  This can be controlled with the security model, so only the user who locked the universe can unlock it.  However, beware that using the Import Wizard still allows you to overwrite a universe that is locked (Jansi).
  • Use universe parameters to control how the SQL is generated.  You can control SQL generation so it is ANSI-92 compliant, as well as use JOIN_BY_SQL to allow separate sub-queries to be combined at the query level instead of in the report (Andreas).
  • All ratios should be set as database delegated/smart measures.  This ensures that the ratio isn't calculated incorrectly (ie average of an average) and is calculated back at the database (Andreas).
  • Generic time objects for Today, Yesterday, This Month, Last Month etc. should be provided (as dimension objects as well as filters).  These dimension objects can be used in filtering objects (Some Date Object = Today Object).  Using filters may depend on the number of dates present in your data, however you could still create filters for the most common date objects (Andreas).
  • As a rule of thumb do not use the WHERE clause in any measure, dimension, or detail object. Use CASE WHEN … SQL syntax instead.  Combining two objects that have WHERE clauses filtering the same column(s) will return zero results, which the user won’t expect (Andreas).
  • I also recently stumbled across the same kind of post over at the Business Intelligence blog.  There are some differences in our lists, so you may wish to refer to that post as well.


Updated 22/07/2009.


- Josh

Joshua Fletcher

BI2013 in Amsterdam

Posted by Joshua Fletcher Apr 8, 2013

Last year I had a fantastic time attending SAPinsider's BI2012 conference in Milan, Italy.  The event was held over three days in June 2012, and topics ranged from Netweaver BW 7.3, BusinessObjects BI4.0 (which was still quite new!) and HANA (also quite new).




Some of the aspects that I really enjoyed at BI2012 were:


  • Ask-the-Experts - I was asked to be one of the experts, and these one-on-one sessions allowed conference attendees to seek out experts in their field and get detailed answers to their questions.  I was able to meet some great people this way, and hopefully point them in the right direction!
  • Customer Case Studies - there were many presentations from customers on how they are using the SAP technologies in their companies, such as McKesson (both Gabe Orthous and Ross Wilson were presenters), or local organisations such as City of Hamburg and Allianz
  • SAP Product Management - there were also specific areas set aside where you could schedule meetings with SAP's Product Management teams.  I was able to meet Karsten Ruf, who manages the SAP Mobile BI portfolio, and give Karsten some direct feedback from the customers I was working with. Karsten also gave me a demonstration of what features were coming in the next version
  • Community - as part of the DSLayered crew, I interviewed lots of local and international attendees to the conference, and got to meet some new friends at the conference and also out to dinner.





This year I'm looking forward to once again presenting at BI2013, but this time in Amsterdam.


I will be presenting on the following three subjects:

  1. Introduction to SAP's Enterprise Information Management Portfolio - this presentation will be an introduction to Data Services and Information Steward, and how the two products integrate with SAP BusinessObjects
  2. Best Practices for Universe Design - focusing on the Information Design Tool new features, as well as best practices for universe design including the old debated topic of aliases vs contexts (hint: use both where required!), and
  3. Advanced Crystal Reports Design Techniques - lots of demonstrations of Crystal Report functionality in the new Crystal Reports for Enterprise


If you are thinking of attending, I hope to see you there - I'm looking forward to meeting lots more people in our broad community :-)


For more details, you can check out the agenda for BI2013 here http://sapbi2013.com/Amsterdam/.

This week I had the opportunity to attend some of the Mastering SAP Technologies 2013 sessions, held in Melbourne Australia, as well as the accompanying SAP Inside Track.


You can find some great coverage of the SITMEL and SAP Tech conference by Custodio de Oliveira here and by Simon Kemp here.


As part of the conference, the keynote was presented by Irfan Khan, who is the Head of Solution Management for Database and Data Warehousing as well as the Chief Technology Officer for the SAP Database & Technologies pillar.


Irfan's keynote focused on 'What Is Driving Technology at SAP?', and ranged from discussion on programming for cache misses at the hardware level 20 years ago, to allelles and genome sequencing with HANA.


Later that day, Irfan gave me some of his time to sit down for an interview about his keynote and the Real-Time Data Platform (RTDP), which you can watch below.


Here are the key points in the video:


01:15   Key messages in Mastering SAP Technologies 2013 keynote

01:55   Overview of the RTDP

03:35   Timeframe to implement the RTDP

05:30   Foundational technologies underlying the RTDP

08:10   Common unified management and governance tool for RTDP

09:10   Implications for new application development with the RTDP

11:15   Implications of the RTDP for BI and DW professionals


Hi everyone,


Welcome to the sixth and final episode in our Diversified Semantic Layer podcast series about Business Warehouse for the BusinessObjects community.  Once again I'm joined by Ethan Jewett, a fellow SAP Mentor.


In this episode, we will examine how we can extend our data mart with new data from a different source system, the methods available to do this, and what benefits this approach brings to Enterprise Data Warehousing.


We hope you enjoy watching.


Joshua Fletcher


Hi everyone,


Welcome to the next episode in our Diversified Semantic Layer podcast series about Business Warehouse for the BusinessObjects community.  Once again I'm joined by Ethan Jewett, a fellow SAP Mentor.


In this episode, we will examine the improvements made to the Bex query in Episode 4, such as navigational attributes, hierarchies and units of measure.


Our next episode will see us bringing in a second data source to recreate the requirements of an Enterprise Data Warehouse.


We hope you enjoy watching.


Joshua Fletcher


Hi everyone,


Welcome to the next episode in our Diversified Semantic Layer podcast series about Business Warehouse for the BusinessObjects community.  Once again I'm joined by Ethan Jewett, a fellow SAP Mentor.


In this episode, we connect to our new Bex query using many of the BusinessObjects tools, including Web Intelligence, Crystal Reports and Dashboard Designer (Xcelsius).  We finish up by walking through creating a multi-source universe on top of the Multiprovider, which will allow us to query BW from Visual Intelligence and Explorer.


Our next episode will continue on optimising and explaining different features of BW.


We hope you enjoy watching.


Joshua Fletcher


Hi everyone,


Welcome to the next episode in our Diversified Semantic Layer podcast series about Business Warehouse for the BusinessObjects community.  Once again I'm joined by Ethan Jewett, a fellow SAP Mentor.


In this episode, we create a MultiProvider on top of the Sales DSO, and then create a simple Bex query on top of the MultiProvider.  We also discuss some specific BW features that don't exist in the BOBJ portfolio which are particularly beneficial in data warehousing.  We finish the podcast by creating an OLAP connection in the CMC, and connect SAP BusinessObjects Analysis for OLAP to our new Bex query.


Our next episode will cover connecting additional BusinessObjects tools to the Bex query.


We hope you enjoy watching.



Joshua Fletcher


In our regular Diversified Semantic Layer interview series helping you get to know people in our community, also known as 'Unstructured Geek Analytics', I took a chance to sit down with:


  • Jayne Landry - Global Vice President & Solution Manager for SAP Analytics
  • Saskia Battersby - General Manager BI Solution Management - Classic BOBJ
  • Blair Wheadon - General Manager BI Solution Management - SAP ERP & BW Customers


We were together in Melbourne, Australia, for a Q&A panel about Business Intelligence, with SAP customers Officeworks and KPMG and fellow SAP Mentor Paul Hawking.  You can listen to the interview here http://dslayer.net/ugeeks/view/171-jayne-saskia-blair, or feel free to subscribe to the podcast on iTunes.


Key questions in the interview:


1:00 - What interesting thing did you take away from the panel discussion?

2:21 - What current area of the BI portfolio is exciting for you?

4:38 - What's your favourite buzz word right now?

6:58 - What's your biggest screw-up at work that you are allowed to tell us about?

10:00 - What do you do for fun in your spare time?


Thanks again to Jayne, Saskia and Blair for giving me some of their time to be interviewed.

Hi everyone,


Welcome to the next episode in our Diversified Semantic Layer podcast series about Business Warehouse for the BusinessObjects community.  Once again I'm joined by Ethan Jewett, a fellow SAP Mentor from Wisconsin.


In this episode, we have connected Data Services 4.0 with Business Warehouse 7.03 using an RFC call, and load sales data into a Persistent Staging Area (PSA) using Data Services jobs.  We also create InfoObjects for the relevant characteristics and key figures in our sales data, which we then load into a DataStore Object (DSO).  Along the way, we (Ethan most of the time) discuss where BW has good and not-so-good features.


Our next episode will cover creating a Bex query on our DSO, connecting the BusinessObjects tools to the Bex query, and then starting to load in our second disparate data source, to begin the process of building an Enterprise Data Warehouse.


We hope you enjoy watching.


Joshua Fletcher


Hi everyone,


Welcome to another Diversified Semantic Layer podcast series.


I'm excited to be joined by Ethan Jewett, a fellow SAP Mentor from Wisconsin.  This podcast series focuses on explaining SAP Business Warehouse for the classic BusinessObjects community, and aims to show by example how Business Warehouse can be utilised as an Enterprise Data Warehouse, even without a dependent SAP ERP system.


Over several podcast episodes, Ethan and myself will build a BW system with two separate data sources, and begin building the foundations for an Enterprise Data Warehouse.  We will leverage BusinessObjects technologies such as Data Services, and the BusinessObjects Enterprise platform, to show how the BW and BusinessObjects can work together.  Lastly, we will discuss how BW has specific functionality which can add many benefits to data warehouse projects.


We hope you enjoy the series, and please feel free to comment and ask questions which we can incorporate into the series.


Joshua Fletcher



Below are the links that Ethan references in the podcast:

A key thing that stood out for me was a recent tweet by friend and fellow Mentor Clint Vosloo - 'There is so much opportunity for SAP in the EDW [Enterprise Data Warehouse] space right now' - and I couldn't have agreed more.  This statement was meant for non-SAP ERP customers, not for classic SAP ERP customers.  I want to drill into this statement and why I agree with this blog post.

On the recent DSLayer podcast with John Appleby, Ethan Jewett and Steve Lucas, Steve talked a bit about the traditional gap between SAP BI (Business Warehouse / BW) people and BusinessObjects (BOBJ) people.  For a lot of BOBJ people, the universe is the universe - this has sometimes meant that some BOBJ people don't understand the true value of BW.

Universe developers can build a semantic layer on any datasource, and don't really care if it is an EDW, a data mart, a 3NF Inmon DW or a transactional OLTP system.  And this is definitely true, and one of the pieces of magic in the BOBJ platform.  Being able to hide the complexity of an OLTP system and still expose a business-focussed layer that is easy to use is a powerful tool, no wonder it is called the universe.

However, my background is not only BOBJ, but also building data warehouses using the Data Services platform, modelling my own Kimball star schemas, designing the ETL and once that is done, building the BI platform.  This experience has given me visibility of how BOBJ is made doubly powerful when combined with a solid EDW and ETL platform.  Thus, my interest in how SAP will shape their recent acquisitions and newly released technology - as combined, there is a huge potential to deliver massive business value.

One of the concerns from my perspective that has added fuel to the debate about BW, HANA and BOBJ is the duplication of functionality across different technologies in the SAP portfolio. So what technology pieces currently are available from SAP?  Let's break it down by functionality (and please excuse me if this isn't 100% correct or exhaustive - as you can see from my background, parts of my understanding are high-level only):


Functionality'Classic' SAPBusinessObjectsSybase'New' SAP
ModellingSAP BW
Data Integration / ETLSAP BWData ServicesReplication Server
Semantic LayerSAP BWUniverse
Data GovernanceSAP BWInformation Steward

Database Engine

Sybase IQ & Sybase ASEHANA
ERP Standard ContentSAP BWRapid Marts
BI & ReportingSAP BW (Bex)BusinessObjects

PlanningBPC (BW backend)

Predictive Analytics
Predictive AnalysisSybase IQHANA (R)



That is a lot of duplicate functionality, in my humble opinion.  You can also see why there is such a huge gulf between BW and BOBJ people.  BW and BOBJ, prior to the BOBJ acquisition, did pretty much the exact same thing.  There might be technical differences (physical cube vs virtual cube ala universe), and quite significant additional capability from Netweaver BW as a platform, like planning capability, but all in all a lot of the same capability is duplicated.  No wonder there is so much discussion and interest from both communities!

So why I do agree with Clint that there is so much opportunity in the non-ERP EDW space?  I look at the above table and see huge opportunities, opportunities for customers to leverage best-of-breed technology in ETL, BI, data governance, modelling and database engines (both IQ and HANA).  My work sees me working with a lot of greenfield customers, or customers who have existing business systems but very limited BI or DW capability.  For these customers, I see big opportunities, being able to leverage not only BOBJ and Data Services, but now having access to database technology like HANA or IQ.

By combining PowerDesigner, Data Services, Sybase IQ, BOBJ, Information Steward and Predictive Analytics, that is a very powerful EDW platform.  Throw in HANA for specific in-memory and/or realtime requirements (supplemented with Replication Server), and the value is even higher.  And all without touching SAP BW if a customer decides not to.

Switching track to SAP BW now, I completely agree with many of the sentiments already posted online - if customers have BW already, HANA provides a fantastic performance boost - there is no reason why BW customers shouldn't move to HANA.  Significant investments exist in the BW platform, and by leveraging HANA and BOBJ, these customers have a great platform going forward.

But when it comes to non-ERP data, I'm not yet sold that BW is the EDW platform you should use.  I've heard comments that it shouldn't be the chosen EDW platform if you don't run SAP ERP, that it is difficult to get non-ERP data in, and to combine ERP and non-ERP data.  I can't comment on that part, but what I can do is express the ease of use that I have experienced by using Data Services, Rapid Marts (if SAP ERP is part of the picture), and a standard Kimball process.  Sure there are always challenges in any EDW project, but with an agile methodology I personally have experienced great results.

Still, there are also huge opportunities to improve the SAP BW platform by integrating the new tech that has been acquired and/or released.  Replace the ETL engine with Data Services.  Make it easier to load in non-SAP data, or enable more agility in mash-ups.

What's the point of all my ramblings above?  There are big opportunities for SAP.  They have purchased some great tech, and HANA is super-exciting.  But I would love to see SAP take all of this great tech and build something that is worth more than all the current non-integrated parts.  What would this look like?  I don't know, and smarter minds than mine should decide.

And what about all this BW vs BOBJ?  I believe there are several things to keep in mind.  One, there are benefits and drawbacks to both platforms, and I definitely don't intend for this post to favour the BOBJ side.  I'm super-envious of how good hierarchies are handled in BW, of multi-currency conversions, of the BPC capability - and there is much more.  I want that functionality for my customers, ERP and non-ERP alike.

And likewise, I believe there are several things that the non-BW technology does really well.  I've heard comments from BW developers that they wish they could use Data Services to load ERP data into BW, because of how powerful it is (and yes this is technically possible nowadays - but how many customers are doing it?).  Likewise Sybase IQ is such a high-performant database engine, and does meet many requirements that HANA does not.  Such as big BIG *BIG* data  - like a one petabyte data warehouse!

Secondly, by keeping two separate EDW solutions (by which I mean 1-BW, and 2-most everything else), we are seeing  very different workflow develop in the BOBJ platforms.  If you use BW, use Analysis Office, Analysis Web, BICS connector for all your reporting tools, but some functions are not supported - try getting flattened hierarchial data in Webi than mashing it with non-BW data.

If you don't use BW and instead use a universe, hey guess what sorry, you can't use the Analysis tools, and hierarchy functionality works completely differently.  This creates confusion in the developer community, as is evidenced by the huge amount of BW-BOBJ integration questions raised during every BOBJ webcast and presentation.  If the gap isn't pulled together, I worry that it will only reinforce the gap between BW and BOBJ people, and lead to bigger problems long-term.

What I want to draw attention to by this post, is that I'm anxious to see SAP create something truly amazing, to take their great investments and exciting new tech and provide a clear and visible roadmap that uses all these discrete pieces.  Some ideas of what this could look like have been floating in my head, and please contribute some more:


  • A next-generation semantic layer which pulls together the best functionality from BW, the universe and PowerDesigner.  This could allow both logical and physical modelling, true hierarchy creation that is virtual only, and which also allows business data to be captured and stored adhoc (instead of using Excel for all those pesky lookups that aren't stored anywhere else)
  • HANA and IQ for near-line storage, but automated and easily maintainable - advances have already been made in this direction by the capabilities now built into Data Services 4.1, allowing many source tables to be replicated with minimal direction
  • Standard content 'standardised', available out of SAP, available in the universe, available in ETL, available in HANA - whichever way you want to consume



Lastly, please don't think that I don't appreciate how monumental a task this is for SAP.  A common theme is 'innovation without disruption'.  That is a big ask of any software vendor, and for SAP in the Database & Technology space, there is a lot of discrete technology and many platforms.  But from my perspective, there is fantastic opportunities by integrating and taking this technology to the next level. There are also so many smart people in our community - if anyone can do it, SAP can. I for one look forward to what happens next.


PS - I really hope this post doesn't offend anyone, and I'd love to have some great discussion about where I am wrong - just be nice :-)


[Update 1 - added RDS to ERP Standard Content in table]