The real challenge of BI System

There is a lot of attention in BI projects on planning dashboard and KPI and a great emphasis on customer experience,

This is very important and I don’t want to make you think that you should take it lightly, this is the front of the project, and the way you present your data is what encourages users to start using it. But this is not where the real challenge lies, and this is not the real purpose of BI Projects.

It's just like going to a restaurant. A very tasty food served in unattractive representation will not encourage you to taste it. But, bad food with beautiful presentation is much worse; you will taste it, and get so disappointed that you will not enter this restaurant again. You can forgive a bad presentation if the food is very tasty, but you won't forgive food that is not tasty even if it served in a beautiful costume.

What make the food great is the quality and combination of the ingredients, and the way that the cook integrates it all to perfection. A good chef will choose the ingredients carefully and handle them with care. The presentation is the last touch, it's the grand finale, but this is not the essence.

It’s the same in BI projects. The ingredients of this dish are the data, integrating complex data is the big challenge. Once the data is integrated confirmed and correct it can be presents in any visuals and colors, in any presentation tool that you like and as long as the data is good, many business insights can be derived from it and turn into business decisions.

You should never forget, BI is a decision support system, it only purpose is help decision makers make good business decisions. Good business decisions are depending on good analysis and good analysis depends on the quality of the inputs, means the quality of the data.

So one may say well, what is the problem? Suppose I'll give you access to all my databases and you can select anything that you need.

Well, it is not that simple. It is not that simple for different reasons, all combined together which make it more complex:

• Data that comes from different sources have different point of view, different terminology, so how do we integrate this data into a single complete 360 degrees view with unified terminology?

• Information systems are based on workflow between different units/systems, and during this workflow data is been duplicated, and each copy has its own life and it splits into different and sometimes contradicted angles. How do we decide what is the real business point of view?

• Sometimes the same data duplicates into several units, each unit needs different part of the data, and it update only the part that is important for its mission. Other parts of the data are not updated or worse, only partial updated. So how can we know which part of the data in which system is the right and up to date data?

• Other issue is “garbage”. A lot of times you will find in databases data that nobody can explain where it came from. It can be technical data like control characters or real data that accidently enter the wrong field and nobody noticed. This can happen for many reasons; some information that was changes in other system that was not relevant for the current process, sometimes “garbage” can be residuals of an old systems, and there may be many other unexplained reason. So what can we do with "garbage" data and how can we differentiateit from valid data?

The main challenge in any analytical platform is getting the data, the real data, the data that describes business reality, the data that is agreed upon everyone in the organization as the real data, the data that is the single version of the truth.