Why We Moved to Google Cloud Platform, in Two Slides
Six months after taking our shiny new enterprise health data warehouse live on a “safe bet,” traditional, on-premises data warehousing platform, we were mired in performance, stability, and maintenance issues. In desperation we kicked off a stealth-mode pilot project to measure the capabilities of Google Cloud Platform — especially BigQuery — against the on-premises system.
Three months later we walked away from our original investment, completed a full migration to Google Cloud, and have never looked back.
I get asked all the time how we did it. If you’d like an in-person deep dive, it will be the subject of my speaking session at Google Cloud Next this month. Or you can listen to the whole story here as I told it a couple years ago. There’s drama. Politics. Steak (or not). It’s a wonder I didn’t lose my job.
But over time I’ve come to realize that much of the why came down to the data on just two slides:
The core capability of any data warehousing system is performance. If it can’t keep up with demand, there is literally no other benefit that will make it a worthwhile investment. The left chart shows the 50% performance improvement we found in comparing performance for the daily slog of loading and integrating data from source systems. The right chart shows a particular analytical process — fuzzy matching millions of patient records to look for duplicates — and how it went from taking 8 hours to about 15 minutes.
I’m not kidding about this. We’ve operated this system in production for two years now and the performance holds up. If anything, it’s better.
I care a great deal about elegant architectures, scalability, innovative product development, etc. But it turns out a lot of other people whose opinions matter care much more about money. Particularly where we were walking away from a major initial investment, we knew the “money slide” had to be compelling. And it was.
With the savings we’d generate by moving to Google Cloud, we could complete the full migration and relaunch on the new platform within our original budget. We could eliminate our performance and maintenance headaches, improve our security posture, position ourselves for future innovation, and do it all without asking for more money.
That’s why we did it.
Later I’ll write about how, including how we convinced a team of security, legal, and compliance officers that their data was safer on the cloud; how important it was to rethink architecture to maximize benefits and minimize costs; and how liberating it feels to free yourself from infrastructure headaches and focus instead on innovation and delivering value.
Follow me on Twitter.
📝 Read this story later in Journal.
👩💻 Wake up every Sunday morning to the week’s most noteworthy stories in Tech waiting in your inbox. Read the Noteworthy in Tech newsletter.
