Business Analysis & Data Visualization

Shorten the distance between information and action.

Oly Oyales · Oct 4, 2020

Data management is key to enabling business analysis

A s Head of the Data Visualization & Analytics function of the Technology Business Management Office at DB, I was tasked to ensure the financial and operational data were properly integrated with our Apptio Cost Transparency solution.  

It was a tall order.  

There were over 100 "inventory" files of different shapes and forms for compute, database, end user services, network, voice, mobile devices -- pretty much anything related to the cost of running Technology at DB.   I saw an opportunity to create a system to store and process the huge variety of datasets from the Product Owners.  

I learned from my stint as a Senior Product Manager that by setting up a good structure along the dimensions of People, Process, Tools & Technology our team would be able to address the challenge and scale up to meet future demands.  

I set out to build our technical capabilities.   Among those:

  • Microsoft SQL Server - to stage and store the inventories.
  • Web page - to let Product Owners upload their inventories.
  • PHP, Cygwin, Shell Scripts, and Python - to inspect the raw inventories and let them know if we detected an issue.
  • SFTP - to stage and transmit the inventories to Apptio via Datadrop.
  • and many more

My team brought these tech goodies together and created a solution that allowed our partners to upload the inventories through a web interface instead of having them send those to us via email as they did previously.

Data integration

One of the key tools we used in our data management operations is Microsoft's SQL Server Integration Services (SSIS).   My team practically built the data wrangling practice around it along with Cygwin, PHP, Perl, Unix, and Windows shell scripts.   We were able to conveniently connect to our data sources, tap in to a library of business rules in the form of stored procedures and functions in our SQL server.   We also implemented data classification routines in our web backend using Codeigniter and Python to determine the types and forms of the raw data before we ingest them.   We built a Data Governance model to ensure quality, timeliness, and completeness of the raw data.   We also set up baseline markers to ensure that inventories with high variance against the norms were detected and corrected before they became an issue.   Finally, we leveraged Apptio's REST APIs to pull the calculated data back on-premise to power our visual analytics and share the insights on risks and opportunities around Technology cost to our senior leaders.

Visual Analytics

I've used many data visualization solutions over the years, among those include Qlikview, Tableau, popular Open Source packages such as d3.js, and charting packages in Python (matplotlib, seaborn, etc).

I used Tableau to visualize the Apptio data because of the size and scope of the financial and operational dataset.   I was actually hesitant to use Tableau in the beginning because I still got the hangover from the Qlikview Kool-Aid, but I worked with great partners in the company and in the process I learned what a good data analytics practice looks like: starting with a good roadmap, building the core capabilities, and leading a community of practice.  

I settled on the Tableau groove and created a number of visual analytics to present the analyis of Technology cost that stakeholders at several levels would understand.   Those analysis included the Consumed Cost, Owned Cost, Value-to-Recover, Total Cost, Unit Cost, Volume, Trend and other related insights that inspired the senior leaders and partners to ask the "why" questions.

My team published the Tableau workbooks to our enterprise Tableau server to share the insights to our stakeholders.   I implemented access control policies and mechanisms to ensure that the appropriate staff received the appropriate content.   My team would periodically check who accessed what content and how often to find out whether we need to update them.  

The word quickly got around and more people asked for cost data for their own purpose.   After I trained my colleagues on Tableau and shared the best practices with them on Business Intelligence and Visual Analytics, we started building the momentun and eventually, the velocity to deliver the insights to a wider audience.   We were able to establish a consistent cadence on delivering trusted and governed BI content from development to test to production.  

Life-long learner

If you Google the word "big data landscape" or "data visualization" or "business intelligence" or "AI landscape" you'll find a dizzying amount of materials that look like multiple periodic table of elements.   For example, there are mentions of Oracle Analytics Cloud and Adobe Analytics Cloud and others that seem to offer augmented, all-in-one solutions.   Perhaps, many companies are performing their analysis in the cloud to cope with scale around large datasets and streams.   I think these are wonderful solutions that cater to many use cases, and if I have all the time and money, I would love to try them all. :)  

Back on earth, I recently added Python and Alteryx into my tool collection.   I found the Pandas package in Python to be simple, yet extremely powerful while Alteryx is incredibly easy to use and appropriate for a number of data transformation situations especially in the Finance domain.   I've also evaluated Power BI, and have begun exploring Big Data analytics solutions like Databricks, Google BigQuery, and Dataiku.   I'll write about the experience on these topics as I learn more, so stay tuned!