Welcome to my second blog about the BW Layered Scalable Architecture (LSA) the reference architecture for large scale BW DWHs/ EDWs. In this blog we will have a look at the LSA Building Blocks, the cornerstones of your future BW architecture.
What we discussed so far
In my blog ‘SAP NetWeaver BW: What is the BW Layered Scalable Architecture (LSA) all about?’ we highlighted that the design of large scale BW Data Warehouses (EDWs) should follow architecture principles like we can observe them constructing large buildings: standardized, scalable, no squiggles, efficient usage of materials, earth quake save.
Large scale BW Data Warehouses (EDWs) should be designed like the building shown in this picture – highly standardized. The BW Layered Scalable Architecture (LSA) describes these design principles.With buildings we have a standardized supporting structure (pillars & ceilings). The LSA calls these architecture cornerstones ‘Building Blocks’.
LSA Building Blocks - The Architecture CornerstonesThe LSA Building Blocks
- are the cornerstones of your future architecture thus having a decisive influence on the overall success of your future BW EDW
- describe the general BW EDW layout independent from concrete BI applications and BI projects.
The LSA has the intention to provide rapid value without losing the complete picture. Delivering rapid value means that already after a short brainstorming and investigation phase a first but sustainable draft of the new Customer LSA is available and new BI projects and/ or migration projects can start implementing on it (s. my blog ‘SAP NetWeaver BW: What is the BW Layered Scalable Architecture (LSA) all about? ').
A rapid roll-out of your Customer LSA is achieved dividing the Building Blocks into two categories (s. picture)
- Landmark Building Blocks and
- Assistant Building Blocks
Landmark Building Blocks deal with architecture areas that need fundamental decisions before doing implementations - they play the same role like the supporting structures (pillars & ceilings) of buildings.
The Reference LSA helps you making initial landmark building blocks definitions without closing the door for future expansions and refinements of your architecture framework.
Assistant Building Blocks deal with architecture areas that are normally less prior with respect to the point in time you should decide and roll out corporate standards.
When and whether we have to decide on Assistant Building Blocks depends very much on customer conditions.
In general making decisions on Assistant Building Blocks means also deciding on investment what takes time. Taking this time into consideration, it is nevertheless important to start brainstorming on these architecture topics at an early point in time despite the fact that we in general do not need guidelines on these topics to get started.
Let's have a look at the LSA Landmark Building Blocks.
LSA Landmark Building Blocks
Your future BW EDW needs a stable, flexible framework. This framework is defined by the LSA Landmark Building Blocks: The Data Layers, the Data Domains and the Date Model
- The Data Layers
Standardize the management and definition of persistent data containers (Data Store Objects (DSOs) and InfoCubes) from extraction to delivery and how data are transferred from one layer to the next one. The Layers apply throughout the entire BW EDW for all data flows.
- The Data Domains
section the data in your data warehouse by a stable, customer-defined criterion. A geographical criterion often fits. A simple one for a global BW EDW could be ‘continent'. Then the Domains defined by continent result in a sectioning of data such that for instance Asian sales orders and European ones are not hosted in the same data container (DSO, InfoCube) but in particular ones.
For transparency reasons I recommend to section all (transactional) data in the same way applying Domains throughout the entire BW EDW.
- The BW data model
is defined by the BW InfoObjects. A proper data model strategy is important to keep your architecture flexible and consistent.
You can think of InfoObjects as a BW-wide shell that forms the basis for
- § defining consistent InfoProviders (Data Store Objects (DSOs), InfoCubes) across your layer, domain grid
- § storing master data
- § enabling cross InfoProvider reporting via BW MultiProvider being the ‘shared dimensions' of the data marts
The data model becomes all the more important the less integrated the master data are at customer side. For large scale BW EDWs we typically have to provide cross reporting for corporate needs and for local sites reporting on tactical or even operational level. That means that our data model strategy must take into account to host local not-harmonized master data and integrated corporate master data.
This is nothing new. The BW-experienced reader knows and a lot of customers use the BW features to deal with this challenge (InfoObject compounding or concatenation) but you have to remember it and to decide before you start implementing.
LSA Assistant Building Blocks
Now let's have a short look at the Assistant Building Blocks:
- ETL (Extraction, Transformation, Load)
Do you expect extensively data from non-SAP sources? If the answer is ‘yes', it is meaningful for you investigating an ETL-tool like SAP Data Integrator. If SAP systems are the initial and primary sources for your future BW EDW, you just don't care. May be later on.
- Data Quality
Do you have considerable data quality issues? If the answer is ‘yes', it makes sense for you thinking about a Data Quality tool. Again, if integrated SAP systems are the initial and primary sources for your future BW EDW, you normally don't care. May be later on.
often reduced to ‘Do I need a single or a multiple BW instance' landscape. This topic has become more and more an assistant one, because of the arrival of new technologies and the transparent support by BW (BW Accelerator, Near Line Storage s. below ‘Storage').
The landscape architecture comes into focus
- if we have to support mission critical BI or to observe legal restrictions or with other customer specific requirements
- if it comes to agile BI and local autonomy (federated landscapes, SAP Newton appliances and BW EDW)
But these kinds of landscape requirements do not prevent a rapid implementation start based on initial customer guidelines on Landmark Building Blocks.
I intend to offer a dedicated blog talking on ‘LSA and BI/ DWH Landscape'.
yes, this sounds a little bit abstract. What I mean with ‘storage' is the following: traditionally all data of a BW DWH are hosted by an RDBMS. This is for large scale BW EDWs not adequate (also having a smaller BW you should rethink this strategy):
Near Line Storage (NLS) & Data Life Cycle Management - In all DWHs there is a high amount of data, which are hosted ‘just in case' e.g. for recovery purposes. In BW EDW we expect even more rarely used data (complete history, ‘support the unknown' concept). Rarely used data should be hosted by a Near Line Storage (NLS) tool. NLS tools compress your data offering space reduction up 95% (e.g. SAND) without destroying your Service Level Agreements (SLAs). The nice thing is: BW transparently supports NLS, allowing you to move rarely used data to cheaper storage. The LSA standardizes the usage of NLS thus the life cycle management of your data across your BW EDW.
You should keep in your mind:
- NLS keeps large BW EDWs manageable and reduces TCO.
- NLS must be part of the architecture. This is the more true the larger the expected data volume.
But NLS does not need to be in place at the very beginning. Introducing NLS means of course that you first have to invest.
I will discuss NLS and LSA in a subsequent blog.
BW Accelerator (BWA) & In Memory Technology - You probably know from your own experience that RDBMS are week when it comes to random access as we have it with ad hoc reporting - no index or aggregate means bad response time. Also with standard reporting we need a bunch of aggregates to keep our service levels what means high costs.
The BW Accelerator (BWA) resolves these issues and must be part of the overall architecture.
- Ownership/ Organization
Designing, implementing and operating a BW EDW need strong governance. A BI Competency Center (BICC) should be established if not existing yet. You can find a lot of postings in the net about BICCs.
- Development Concept / BI Deployment
Most SAP customers follow two targets establishing a BW EDW: enabling integrated cross organizational reporting & analytics and standardizing local (organizational) reporting & analytics. Both directions are normally addressed creating a central developed offering of BI applications based on the Customer LSA (central template).
Such a central maintained BI template solution has obvious advantages but there are also limitations. As it delivers a standard it can and will never cover all specific local or personal BI needs. So the question must be discussed ‘how to make BI agile' without corrupting the LSA BW/ EDW achievements? There are many, many areas joining the game if we want to define a customer strategy for this topic:
- Organization - BI Competency Center (BI CC)
- Processes - defining the scope of the central BI template
- Landscape & BW technology
federated architectures (e.g. BW EDW & Newton Appliances)
meta data management and deployment
Making BI agile is crucial for the overall success of your BI strategy.
But independent of this fact establishing a solid foundation for your corporate BI strategy - a LSA founded BW EDW - is a necessary condition for this overall success.
LSA Lessons Learned - Wrap UP
The sun is going down and it's time for a short wrap up. The sum of topics related to BW EDW architecture seems overwhelming and indeed it is and each customer is unique and puts new architecture relevant topics on the table.
The most important advice I can give you is: focus on the important things first like the Landmark Building Blocks. Follow always the 80:20 rule meaning don't lose yourself or allow others to direct you into details and exceptions at an early stage.
And: Standardize, Standardize, Standardize :-)
More details in my Teched 2009 presentation: The Layered Scalable Architecture for SAP NetWeaver BW on a Corporate Scale’ – lecture BI302
May be a more general presentation from SAPSKILLS 2008 is also of value for you: SAP NetWeaver BI: Data Infrastructure und BI-Maturity – Data Marts, Data Warehouses (DWH), Enterprise Data Warehouses (EDW), Appliances
(Don't become confused only the first two slides are in German :-))