One of the greatest statements in 2010, from my side, was a Gartner statement made by Mark P. Donald, when he stated 'without business in business intelligence, BI is dead’. He condensed four years of my research brilliantly into a single sentence. By 2006 Gartner started reporting that BI projects were not meeting business expectations. Then in 2011 Gartner once again reported that the world spent $11.2 billion on BI in a fuzzy background that less than 50% of these projects would meet business expectations. Then in April 2012 ZDNet reported that there is a reported wastage of $6.2 trillion though it remains largely unverified by reliable resources. However, one thing is certain that BI projects are consistently failing to meet business expectations and with HBR publishing that good data alone does not assure good information all the pieces better fall into place.
Over my 15 years of BI experience I can state with fair confidence that there are few companies that do not feel their BI is in a deep crisis. Each executive and CIO is wondering whether it is the HW, SW or their implementation partner that is at fault. If you ask me the answer is both yes and No.
In order to find this solution we need to go back two decades when it took two people four to six weeks to prepare six reports for the board meeting and there was no questioning whatever the executives got once or twice a year. Fast forward to 2012 and we have invented hardware that can compute millions of time faster, and softwares that can perform technical miracles when compared to what it could do in 1987. In 1987 we needed two resources to provide corporate and executive reports, by 2012 we need a team of twenty to thirty experts to keep the complex technology humming. We are all specialist now and hold a lot of information about a small part of the whole solution. We have unfortunately created teams of Technocratic workers who have little idea of the forest as they continue to clip leaves on a tree. What we need today is business value workers who are explorers and not technocratic cowboys.
Spend a little to plan and save big time in error fixing later: Our research indicates that the current technocratic designs mean that more than 50% of BI project live in a state of comatose existence, over 60% of our data warehouses are designed to strategically fail, and that ovr 65% of technical doctors that go to take care of sick data warehouses do not end with an improvement to the health of the information delivery capability. The writing is clear on the wall that we need business value workers for two reasons, the first being the unmanageable costs of BI projects and the second the low business value being derived from them. In all this we must not forget that when we go from a world of PC's and two resources working on Lotus 123 for six weeks to prepare six reports to one were we generate hundreds of reports from tens of terabytes of data the costs cannot possibly remain the same.
The differences: In 1987 we used to deliver reports on what had happened last year, quarter or month and our management was only reactive. By 2002-3 we started delivering analytics where we started analyzing trends and moving more towards performance analytics that was summarized automatically. We efficiently merged planning data with global actual numbers or conducted spend and vendor performance. By 2009 we commenced on a path to predictive analytics and entered the world of informatics and bioinformatics. By this time though the technology had matured the systems to design it had not yet become scientifically driven. We could now summarize global data into daily views for our executives and came to the new world of predictive analytics. Now we did not wait for events to happen but could actually predict it in near real-time. For example it was then possible to to visually see the full global supply chain with connections to supplier systems and see a process weaken and immediately ripple across the supply chain. Companies were able to now react to the future. By 2011 we entered the world of true-real-real time. I use the double real-time for we had misused it in the past. Now we can conduct CO-PA analytics and see the situation of a campaign or trade promotion as it stands right now. We can see shelf space utilization as it stands right now across thousands of retail outlets across the state, country or planet. But unfortunately we still continue speeding the technology path where we deploy BI project successfully but they often do not deliver business expectations or values. This is where BVA, or Business Value Attainment, principles based on scientific standards and processes comes in.
Our solutions come from something as simple as data, pure empirical data. When we reviewed all the complexities in the data we suddenly realized that the most expensive BI projects did not result in the best information and vice versa, i.e. the best BI projects were not the most expensive ones.
What this means is that there is Hope for each and every one of us, because if only the most expensive projects were most successful then it would indeed be very disheartening for all but the largest corporation - but thankfully this is not the case.
When we analyzed the positive deviance a little further we found that the ones that were most successful looked more and more like scientific systems. All the evidence prove that simply having the most expensive technocratic components is never enough. In fact it turns out to be a big disadvantage as these projects then gedt driven by technocratic arrogance, based on an unbending faith that ‘technology is king’. These costly projects end up with a pile of expensive components that do not run like a well-oiled system and each issue has each group of experts pointing their fingers the other way.
Our research also established that scientific systems have inbuilt checks and measures.
Skill 1: Scientific systems have an ability to identify success and failure. The issue is that specialized technocrats can only see the small section of their specialization, and super specialists can see even less and both these workers can never do any form of predictive analytics – i.e. what will the impact of this step be 2 or 3 years from today to the enterprise information capabilities
Skill 2: Scientific systems devise solutions by eliminating defects. Tradition technocratic solution to any problem is to either buy a bigger HW, a
new SW or send their resources for more specialized training but when we viewed the DW and BI ecosystem all we found were more and more experts, who actually did not solve but exasperated the problem.
Skill 3: Scientific Systems eliminate defects with Checklists: When we analyzed how critical components were managed we found an abundance of checklists. Pilots use them, engineers use them and today even cowboys use them. Checklists do not tell pilots how to fly a plane but it is a reminder of critical things that are often forgotten or get missed and that can potentially lead to a disaster.
There is no clear recipe for success but there we did manage to build a checklist for eliminating failures. Over the last 6 years we have implemented these methodologies, that we call BVA or Business Value Attainment in 4 projects and achieved exceptionally high success. Each of our projects scored in the upper 90% project satisfaction in week 1 and week 30. I state this for one of our research indicated that ’98% of BI projects are declared successful in week 1, and less than 50% of them remain successful by week 10’ Gartner and BI Valuenomics research.
The last skill we collectively need is the ability to implement these established scientific methodologies. Despite its established success we found it surprisingly slow to implement. Read my blog on Moneyball and BI’ as it explains this dilemma. These new 'scientific principles' (due for publishing in Q3 of 2012 in a book) challenge the status quo and current value systems. It challenges things the technocrats have been doing for too many years. It requires us to work with with humility and replace our technical arrogance, it requires us to work with Business and IT as a single team and replace technocratic isolation, and it requires us to work with scientific principles and checklists where we though we knew everything and where a 50% failure rate was a done thing.
Today establishing scientific principles in everything is our greatest path forward in order to enhance quality and lower costs via a process of defect elimination. Taylor did it in the 1911, Ford carried it to the next level by 1921 and the flame was carried by Demings and then when Drucker coined the knowledge worker. We have now come to an infliction point where we all need to evolve from 'knowledge workers' to becoming 'Business Value Workers', i.e. become better Business Value owners and less of technocratic dictators
that march only to our own drum beats. The writings are clear it now only depends on who all are ready to read it.
- SAP Managed Tags:
