In my previous article “An adventure in automation world” , I explained to build automatically ERD and Multi-dimensional models.
In BI, there is a third modeling technique named “Data Vault” which is based on three basic table types: Hubs, Links and Satellite. You’ll find mode information on this site.
This technique uses massive load processing technique and requires adequate infrastructure but, in the other hand, permit to build a solid foundation layer for enterprise warehouse.
We decided to create new service based on this approach with the following questions in mind (a service and not a tool because code is not protected inside current extended models):
Using these goals, we decided to build a tool to support our services with PowerDesigner. We named it “Extreme” because we automatically
build our targets in few minutes instead of spending weeks or months with manual stuff.
Here’s an example of what we want to do:
Here’s a small sample database schema you can have
Additionally we want pre-optimized data models and complete Information
Liquidity models giving all basic instructions to elaborate ETL flows.
Well, as you can see our goals are very complete. Here’re steps we follow to build a solution:
The main extended model has only few attributes. This is dramatically simple to determine parameters before launching the model production:
The complexity is located inside the code (nearly 3000 lines of code), impossible to explain here.
The logic followed is quite “simple”:
Let’s examine results:
First, prepare parameters in the source model and determine Business Keys.
Next launch the Data Vault Analyzer and, after few minutes, you’ll get:
1.All data models and Information liquidity models
2.With full traceability
3.With also all transformations tasks to load target tables through models
4.And the cherry on the cake, how to orchestrate everything:
As you can see, using automation techniques can help providing "extreme" services with high and repetitive quality for your customer.
You'll find also a white paper regarding this article here. Feel free to share your comments !
Kr,
Thierry
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
37 | |
10 | |
5 | |
4 | |
4 | |
3 | |
3 | |
3 | |
2 | |
2 |