My views on BI after one year in the trenches
After having worked for about a year as a Business Intelligence consultant, I’d like to explain my views on the subject. At know.bi, we mainly work with the commercial open source BI platform Pentaho, so I’ll use that as a reference, but this post should apply to BI in general and is not meant to be limited to any given platform.
What do you want to do?
First things first: what do you want to do?
Business Intelligence, according to Wikipedia, means:
“ A way to comprise the strategies and technologies used by enterprises for the data analysis of business information“
A whole lot of words to say BI helps you to understand your business. The idea is to know more about your business by looking at the numbers. Let’s say you own a bakery. At the end of the month, you obviously know how much you spend on flour, milk or other ingredients to make your products. You also know how much you sold of each product. Imagine that your customers have a client card that holds their personal information. If you were to combine the customer information with the products you sell to that particular customer, you could improve your relationship with such customer and improve your sales.
To be more illustrative: Mark buys a bread every Wednesday, on Sunday he also buys pancakes. If you were to keep track of all Mark’s purchases and add up the amount of bread he buys you can give out a promotion of free pancakes with every 10th bread he buys because you know he likes pancakes.
Business Intelligence allows you to keep track off all this information thus making it possible for you to create a more personalized approach to marketing.
Which brings us to the next question: What do you want to do? Do you want to improve your marketing? Do you prefer to see the evolution of your stockpile? Or do you want to do both?
Based on previous information, you could even start to make predictions about the future: e.g. based on the increase in cake sales over the Holiday period for the last couple of years, how much extra cake do I need to plan for this year’s Holiday period.
Understand the problem!
BI is a cyclic process. In every iteration of the process and with every step of the data exploration, you will learn more and will be able to better define what you want. It’s often easier to ask a person what they want and then show them what can be done with the available data than the other way around.
Creating a solution for your ‘wants’ depends on the data you have. To bring up our previous example: If the bakery didn’t have a customer card, then they wouldn’t be able to keep track of Mark’s purchases and thus wouldn’t be able to determine the proper promotion.
Now don’t get me wrong: having the correct information isn’t enough to create the ‘what’ you want. You also have to understand it. It is important to understand the problem and the underlying data in order to find out what can be done with the available data. We always aim for a “think big, start small” approach, because this allows us to start working on small, manageable problems, without losing sight of the bigger picture.
Evaluation - fail fast, fail often
With a clear understanding of the business problem, chopped up into small, manageable tasks, work can start.
However, not all business problems are clearly understood from the beginning, not even by the business stakeholders or process owners.
It therefore makes sense to evaluate quickly and often (the agile project “fail fast, fail often” methodology). As soon as you realize a problem was not understood completely or correctly, it is easier to correct as early in the process as possible.
This iterative process of continuous improvement almost always provides better results in the end.
Worrying about the price tag...
Once you know exactly what you want and actually need from your data, you can start looking at the required tools.
Some of the required tools are a data warehouse to store your data, a tool to process and transform your data and a visualization platform.
Although BI can be expensive if not tackled properly, costs can be scaled and minimized! For example, storing your data in a cloud with auto-scaling (dynamically scaling out to your needs) can be a lot cheaper than physical, on-premise servers. The market leader for cloud (BI) services is AWS (Amazon Web Services).
Another way of tackling the cost is using an integrated platform that can process, transform and visualize your data. We use the commercial open source platform Pentaho, because it covers the entire BI lifecycle and offers customers a clear, predictable cost structure.