BI Predictions for 2015 (2 of 4): Cloud-based Data Warehouse

While different people will point to different times for the birth of business intelligence, I’ve always argued that BI started with the invention of writing.  Writing was invented, in part, as an accounting device.  How do I know how many cows I have?  How many did I have last year?  Write it down, report it and, bingo – business intelligence!

Regardless of when it started, like just about everything else in the world, BI has changed a lot – and the pace of change is accelerating.  With that in mind, here is the second of four predictions for business intelligence in 2015, and beyond.

More Companies Move to Cloud-based Data Warehouse Storage

The concept of focusing on core competencies is not new.  Since the beginning of trade, organizations have sought to focus on their core competencies and outsource those things that others can do better.  Examples?

  • Want to publish a newspaper (back when people did such things)?  You didn’t build the press, you bought it from someone who specialized in this.
  • Are you a logistics company that needs a delivery truck?  You don’t build it, you purchase it.
  • Need a word processor?  You don’t build it, you buy Microsoft Word or Google Docs.

It’s not that companies couldn’t do all these things themselves.  They can clearly be done or even the vendors couldn’t do them.  But, it doesn’t make business sense to do them internally.

So, let’s apply this to IT infrastructure.  Unless you’re an infrastructure company, running data centers is a complex task that’s probably not your forte.  So, like so much else in the world, why not outsource it to someone who’s forte it is?  This is driving – and will to continue to drive – companies to move more and more of their data center operations to hosting providers.

Depending on the customer and their needs, this service is available at varying levels in the IT stack.  You can, for example, just move all your systems to servers running to a vendor’s hosted servers – companies have been doing this for years.  But, a rather compelling idea is to  move components to the cloud (i.e. a platform as a service model).  In this case, the data warehouses database components.

As of a few months ago, Redshift was Amazon’s fastest growing database storage platform.  Why?  Because it provides a powerhouse combination of capabilities that you just can’t replicate with in-house solutions:

  • A data warehouse-optimized column store database
  • A variety of storage options ranging from hard disk drives for huge data sets to solid state drives for blazing performance
  • The ability to instantly spool capacity up and down
  • Inherent backup and restore
  • Low cost 

Yes, your team will still need to develop some Redshift DBA skills, but only so deep.  They won’t have to worry about what kinds of servers to purchase, or where to store them, or how to install the software on them, or how to make sure that they’re getting backed up, or if the backups will even be restorable, etc.

One more point seems important here: All of the major hardware providers have recently announced multi-billion dollar investments in cloud data centers(IBM, HP, …).  Why?  Well, in part because they see the future.  They realize that companies are going to buy less and less hardware and shift their infrastructures to cloud services.

RECOMMENDATION: Cloud-based data storage is just a matter of time.  If your organization is ready for it, start out using it for proof of concept and development.  Once the comfort level is there, port over as makes sense for your unique situation.

PROVISO: Of course, IT and company management are going to have to get comfortable with the idea of storing their data outside their firewall (BTW, cloud providers have some options for keeping all of this within secure boundaries).  Security concerns are valid and must be addressed.  However, it’s a bit farcical to believe that in-house data centers don’t suffer from some of the same issues.  For example, if your data center is in another location, it’s probably already sharing data over the internet.

Read BI Predictions 3 of 4

Let us know your thoughts on our BI Prediction 2 of 4 and check “Notify me of new posts by email” below to make sure you don’t miss the next two predictions.  Add your comments below, feel free to email me at, and, of course, have a great holiday season!


0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.