Business Intelligence has evolved over time, and has gained more prominence. Even in the current economy, BI projects are generally alive and well. So what is the path forward? Of course, I do not have a crystal ball to gaze and predict. Nor have I conducted a scientific study of any sort. However, what I have done is that I have discussed this with several people in the field, read what others have written on it and have thought about it in some detail. So here is what I think would unfold - for whatever it is worth 🙂
1. Stand alone BI will become a thing of past
We are already seeing BI becoming an integrated part of business processes - with CRM applications possibly leading the way. I am very certain that this will be the rule and not the exception in the future.
2. Speed of analytics will increase manifold
Consider SAP BW and the accelerator appliance. The bottleneck now is BW itself, and I am sure more of the OLAP engine will move to BWA pretty soon. So why have this extra appliance? Can't we have this high speed analytics available in standard BW itself? I used BW as an example - I am expecting all leading BI products to increase speed many times in next few years. Here, the improvements in hardware related technologies is key - processor speeds have consistently improved, blade servers have become common place, and virtualization is not wishful thinking any more. Hard disk is a lot cheaper than in the past. All of these will continue to improve so I am guessing we should see dramatic improvements in medium term.
3. One system that does it all
By extension of the above, I suspect that we should start seeing more and more of BI moving into OLTP systems. I am not a computer science major, but my impression is that the database theories that we used over past several years that lead us to develop different models for OLTP and OLAP were built for making best use of the available limited processor power and expensive hard disk space. With the advances in hardware, it should lead to some convergence of OLTP and OLAP into one system. Same should hold true for planning applications - instead of periodic and "after the fact" comparisons of plan vs actual data, it should become integrated into all business processes at real time. Planning was once a part of OLTP systems, including SAP. Then it was moved out of it (like the BW based SEM in SAP's case). My theory is that it will come back and become part of the combined OLTP/OLAP system.
Considering the huge investments that have been done on separate OLTP and OLAP systems around us today, I don't think that this will be the case any time soon. So what would happen in the interim? That is where the next point could make an impact.
4. Federated datawarehouses
This is certainly not a new idea - we have been talking about this for a few years now. However, it hasn't quite become the rage yet. Why is that? well, interoperability is still wishful thinking for the most part in the BI world. SOA is a potential way out, but then the marketing of SOA ruined it for us. I expect that we have to wait for the hype to die down and let some early adopters take the plunge and show us what is possible. I had an initial confusion in my mind whether "one system that does it all" and "federated datawarehouses" contradict each other. However, I have concluded that they don't - for a few reasons. For one, even if OLTP and OLAP systems merge, several such systems could still co-exist in a distributed landscape. And if cloud computing does take off in a big way - big advances will happen in federation techniques.
5. More analytics will happen on unstructured data
The primary reason that analytics needs to become very good in dealing with unstructured data is the gaining popularity of social networks and media and its impact on business. Pretty soon, executives should be asking for dashboards that tell them what is happening in the social media that impacts them. This will probably start in an "after the fact" mode, like most of traditional BI, but pretty soon - I expect this to become very neal real time. Also, it will become imperative that unstructured data can be combined intelligently with structured data.
6. Competition amongst vendors will increase, but may be - so will co-operation
If federated data warehouses are the future, then I don't think there will be many generic data warehousing solutions left in the market. Each vendor should then aim to satisfy highly specialized niches of the BI market. But I also expect a lot of co-operation to go along with the competition - since customers will demand that these disparate solutions can some how talk smoothly wih each other to make the federated system work efficiently.
7.Visualization will rule
Visualization is serious tool, but I have a serious gripe on how it gets positioned in BI projects. As a result, a lot of executives get an impression that it is not useful, and that it is nothing more than a pretty marketing picture. In future, I also expect BI tools to be able to suggest intelligent visual representations of the data with an ability to switch to different visuals, and the ability to drill down as needed.
8.Predictive analytics will turn from half-fiction to reality
Traditional analytics presents data to a human decision maker, who then takes some decisions based on it, and some rules and his or her experience. A good part of these decisions can be made by the machine as predictive analysis improves its abilities. Even if we don't reach sci-fi type scenarios, it is not unrealistic to expect BI to give us options with some pros and cons in near future. This should be of immense use to people like scientists and insurance underwriters and eventually to every process owner.
Before I finish, let me also state that I don't have unrealistic expectations on how soon all these changes will happen. When I look back at how BI has evolved since I started in the field, technology has improved manifold, but the requirements from business users have not changed that drastically. I still hear requirements from my clients that are very similar to what I heard in my first project in the nineties. People still primarily need summarized information with the ability to slice and dice as needed. They also need canned reports, and very few do their own reports. Charts have not replaced line item data, and most of my users need the ability to use excel spreadsheets in tandem with BI solutions. What has indeed changed is my ability to deliver solutions for these requirements - due to better technology, tool features and the experience I have gained over a period of time.
So, what do you folks think? I am eagerly waiting to hear your thoughts and ideas.
