When Business Moves Faster than BI
Face the Facts – IT Does Interfere With Business Agility
It’s a rare organization where the business staff doesn’t complain that IT is holding them back. This is, in fact, one of the key reasons why there is friction between IT and the users they support. Rather than just write this off to whiney users who don’t understand IT, let’s give this complaint its due – IT frequently does get in the way of business agility. Hell, it’s hard to be an agile businessman when your computer guys keep saying they need six to twelve months to support what you need.
Well hell, as an IT guy myself, I know it’s impossible to, at a moment’s notice, deliver the systems the business needs while ensuring that the data is clean, correct, and complete. So, there is bound to be friction between business and IT. But, here are some thoughts on how to deliver quicker and limit the friction.
Problem: Business Moves at the Speed of Thought, IT Moves at the Speed of Architecture
So, by way of example, we’re working with a client right now that has a number of operational systems as well as a data warehouse. This client’s marketing group just kicked off a marketing program with the help of an external company. As part of this program, this venture will deliver important information about new, high value prospects. The business needs to capture and analyze this data and would love to tie it to the data they already have in their operational systems and their data warehouse.
Now, from an IT perspective, handling this data properly will take a good deal of work. To start with, some basic questions must be answered. Questions like, Where will this data land when it comes into our company? and How will it make it’s way into the data warehouse? and How will it tie to the other data in our warehouse? and What structural changes must we make to the warehouse to support this data? and How must our BI tools be configured to maximize the value of this data? and…
Given all this, and all the other projects on their plate, there is no way that IT is going to be ready to fully support this need when the business is ready to move, in about a month according to their latest schedule.
IT’s Natural Inclination
So, what does IT naturally do in this situation? Whether we like to admit it or not, in many cases IT’s natural inclination involves liberal doses of complaining combined with embarrassing efforts to get the business to slow down. I see this at many of our clients; IT tells the business, “We won’t be ready to support you for 6 – 12 months so, wait for us or you’re on your own. And, by the way, we won’t let you be on your own (i.e. just wait and suck it up!).”
Of course, let’s cut IT some slack. Yes, almost anyone can load data into an Access database or Excel spreadsheet and start playing with it. IT’s job, however, is ensuring that data is correct, timely, accessible, well structured for growth, easily understood, and a whole bunch of other things that, like it or not, take time.
A Better Approach
The problem here is that we, in IT, get caught up in the need for nearly perfect, clean data. And, that is of course, necessary. However, that also takes time. In the short term, though, the business might be satisfied with quick, albeit imperfect, data. So, to satisfy both business and IT, I propose that you take two steps:
Step 1: Allow the business to temporarily capture their data in a flexible data store. A sandbox area could be great for this. And, while I hate to admit it, a tool like Microsoft Access might also suffice.
Step 2: Immediately start developing the longer term data structures, likely in your data warehouse, for this data. As part of this effort, set a target date for when the temporary area will be shut down.
This approach will provide flexibility along with structure and control. It can, though, backfire. To protect against this, here are some rules to follow:
- Recipients of data from the temporary area must always be aware that this data is not certified by IT and, therefore, should be used with caution (look for a post on certified vs uncertified data soon)
- Be disciplined about moving this data into certified databases. If the temporary solution works well, we all know it will be easy for folks to back burner the step where the data is moved into official, corporate data structures. However, doing so will lead to long term problems. Remember your long term data plan, your data topology (more on this in a later post, too), and stick to it!
- Make sure the folks loading and extracting temporary data have the technical skills to do so. If they don’t, supplement them with technology experts. In large organizations, you can envision a small department dedicated to shepherding new data sources from temporary to long term data storage.
So, let’s admit it – we in IT do sometimes get in the way. But, at the same time, we’re just doing our job. The trick to satisfying a fast moving business is to come up with temporary fixes to support it, without selling our data quality souls to the devil.
A quick note to friends of Dataspace: Folks have asked why they haven’t seen blog posts from me for a few months. Well, it’s been a while since I’ve been able to write but it’s all good! Dataspace has kept me quite busy lately – in addition to enterprise data management and BI planning for a few clients, we’ve been asked to perform a few executive searches for high-level IT staff and to provide temporary DW / BI staffing for others. Needless to say, it’s kept us on our toes. Still, I’ve always got time for clients, friends and those with an interest in BI. Drop me a line sometime!