Dennis Howlett suggested that I do a post on a conversation we were involved with on Twitter recently. Basically, the conversation had started out as a discussion about how companies will need data scientists to be successful at analytics. The argument was basically that analytics were too complex and you need specialists to get real value. I chimed in that I thought that was a cop out. My belief is that business intelligence/analytic applications have not been easy enough or valuable enough to the layperson to gain wide adoption.
I used the analogy of TurboTax and tax filing. Unless you had very simple taxes, until TurboTax came along, you needed an accountant to help you prepare your taxes. TurboTax cannot help for every use case of tax preparation needs. However, it significantly increased the number of use cases where you did not need a professional to help you prepare taxes. I do not think we have seen the equivalent of TurboTax for business intelligence/analytic applications - yet. There will still be a role for data scientist to deal with the really hard and challenging use cases for analytics, just like accountants are still needed for complex tax filing scenarios. However, with the right tools, mere business people should be able to do meaningful analysis that drives better decision-making.
I think the biggest issue is the lack of intelligence in the tools themselves. One of my greatest pet peeves as an analyst at Gartner was watching demos of reporting tools that showed how you could drill-down to find the data or exception you needed to know or act on. The demo person would effortlessly drill-down four or five levels and get to the result. Most business leaders are not going to take the time to do that kind of exploration (because they may not know where to look to find this nugget like the demo person does). However, those business leaders would be quite interested in the results of that exploration.
Now, they could give the assignment to a business analyst or equivalent to do the exploration, but that is not really solving the problem (but it is what most frequently happens in organizations today). What if instead you had intelligence in the system that would interrogate all of the drill-down paths and report back interesting findings based on your role. That to me would be real business intelligence. Some of the findings may not be useful (and hopefully a system such as this would learn from what was found to be useful or not). Some of the findings may be worth more exploration (but there would be something definitive to look for). Some findings may call for immediate action or decisions to be made.
This is just one class of use cases for analytics. It is one that has been around for quite awhile. With the rush to focus on all the opportunities that Big Data affords (and that do require the know-how of data scientists to exploit), it has taken the attention away from the more basic needs of planning and analysis that were not well-addressed previously for the lay business person. That is a shame.
Disclaimer: These are my personal views and not the views of my employer
Good post and does the topic more justice than the limited Twitter conversation.
I still think though you oversimplify analytics and measure it by the wrong analogy - as I posted here:
http://enswmu.blogspot.com/#!/2013/02/why-is-analytics-so-hard-or-holy-grail.html
Posted by: HolgerM | February 13, 2013 at 12:57 PM
We will have to agree to disagree. You may also want to look at the follow up post - http://blogerp.typepad.com/hcm_research/2013/02/putting-more-intelligence-in-business-intelligence.html
I think you will see I do not think it is simplistic at all. Big data and associated tools do not necessarily solve the usefulness issue. Also, I think the analogy holds in terms of building domain expertise/intelligence into the tools themselves.
Posted by: Jim | February 13, 2013 at 01:52 PM