It’s human nature to hoard stuff. I, for example, have a habit of buying tools specific to the hobby project that I’m working on. My behaviour usually follows the same pattern: I’m in a midst of a challenging task when I realize that a specialized tool would make my job easier. I then do a quick run to the local Home Depot to pick up what’s needed. When the task is complete, I find the right storage space for my new purchase. But despite of trying to stay on top of what I own, I’m periodically confronted with a garage full of gadgets that serve a similar purpose.
Most often, one of these two things happens when we’re faced with clutter: we lose track of what we have and resort to using only the things from the top of the pile; or, like me, we keep getting new tools, only to realize that we own a lot of similar items that take up space but add little value.
The same principles apply to business intelligence and analytics. Most organizations that have BI programs in place, at some point struggle with duplicate reports, messy BI storage or just plain old lack of regular planning.
Luckily, as with other types of mess, BI can too be organized.
In fact, organizations that make a conscious effort to maintain order and declutter from time to time, reap the benefits of timely analytics that meet business needs without sacrificing performance.
Decluttering Business Intelligence
Setting up a successful Business Intelligence (BI) program requires planning. In the early stages of getting business analytics off the ground, we tend to spend a lot of time deliberating the size, scope and scale of the program. However, once the Business Intelligence program is up and running, new problems emerge making organizations question their original approach. Just as in keeping a tidy house, the key to an efficient BI program lies in an ongoing rationalization and decluttering of various BI components.
How to tell if my Business Intelligence needs reorganizing?
If:
- You’ve got business intelligence reports or visualizations, but they don’t fully address your current needs. You find yourself creating completely new reports based on new data sets.
- You’ve got many data sets in a centralized location, but you aren’t entirely sure where to find the right data.
- You’re doubling or tripling the effort and/or data.
.. you might be experiencing the common symptoms of a cluttered BI.
On the surface, the above issues seem easy to address. Create a couple of catalogs, decommission old BI elements and the BI platform will be clean. Unfortunately, it is rarely that easy…
Consider this example:
Suppose you have a BI report which shows which products were sold in different regions and at what price. To get this report, source data was extracted from several siloed systems in each of your regions including production systems, financial systems, and ERP systems. After extraction, your data was cleansed, transformed and consolidated in the dimensional Enterprise Data Warehouse. Then, the required dimensions and facts were extracted to a Data Mart and your report was created.
However, the market situation has changed, and you need to analyse how different types of clients consume your product. You reach out to the Data Analysts from the business unit, but they have a hard time locating required industry information in the consolidated BI storage. When you turn to the team responsible for maintaining BI storage, they offer little insight into the data content. You realize that there are no data owners since data sets in the BI Storage are all based on consolidated data. Now, to get the results you’re looking for, you will need to start a new project, extract the entire data set again and create a new report. Often, at the very end of it, you will realize you’ve created a duplicate data set.
Take steps to organize your Business Intelligence
It’s easy to imagine what your business intelligence will look like after repetitively generating BI products without addressing the underlaying issues. At some point, BI rationalization (decluttering) will be required.
While there are many variations of BI implementations from an organizational and technical standpoint, at a high level the fundamental elements and data flow process remain the same:
1. Start by reviewing your Data Sources
We often see that one of the leading causes of BI clutter is ingesting more data from the sources than what’s needed. To get organized, begin by identifying the current and future use cases for your business intelligence. For starters, every extracted data set should be associated with a Data Owner (also called Data Steward or Data Custodian) who can provide insights into this data set.
Once the data is extracted from its sources, it’s often transformed, normalized (restructured and modeled) and merged or joined based on the BI Data Architecture. This is a point where things can go awry. To avoid future complications, always opt to retain metadata lineage which could track the data back to its sources.
2. Take a good look at your BI Storage
Business requirements and data consumption performance are the two major drivers that affect decisions on how data is originally joined, merged, and structured. However, in virtually all cases, business needs will change over time. When they do, the newly created BI products (dashboards, reports) and added sources of data will affect the structure of your BI storage sometimes leading to unnecessary complexity and poor performance.
There are several ways to reduce this complexity, which all boil down to planning and organizing the storage regardless of whether you’re using dimensional or non-dimensional data stores for your program. It’s self-evident that modeling is the most important clutter limiting measure for dimensional databases, as it enables future development and assures ultimate performance. However, what less often understood is that non-dimensional BI stores also require a good dose of organizing. Any non-dimensional BI storage can benefit from clearly defining, maintaining and enabling all data types and linking them back to their sources. By also associating data with a specific owner and documenting it in a catalogue you can ensure that your BI storage will be ready for further analysis.
3. Analyse your BI Consumption
Numerous organizations struggle with efficient intelligence analysis because their BI products don’t meet the changing organizational needs. As the enterprise grows and evolves, its dashboards and reports should follow suit. One of the best ways to ensure that the insights you receive are relevant and timely is by cataloguing BI products and attaching them to current business requirements.
You will notice that when data lineage is preserved and ready to handle the full BI data lifecycle, decluttering will become simpler and a lot more efficient.
4. Consider the organizational approach to Business Intelligence
As the last step, take a look at the big picture. For some companies, the source of BI clutter may be rooted in organizational approach rather than the BI process itself. For example, when BI elements are handled by several distinct groups within the organization, there is a significant risk that the approach to data isn’t always consistent.
In other organizations, what’s needed is a revamped project methodology. For instance, BI processes, strategy, and delivery are rarely effective when following the waterfall approach. Conversely, agile methodology can offer greater benefits, especially when paired with a defined Information Strategy and governance.
Finally, remember that you’re not alone. In fact, the industry has been struggling with disorganized BI for a long time. What helps set the leaders on track is a holistic, organization-wide change that comes with Implementation of Architecture Practices and defined governance that enable timely, efficient and structured BI.
Do you see any red flags in your BI program? Give us a call. Together we can ensure that your organization has the right insights for accurate decision-making.