Joe Waller, Chief Product & Technology Officer, Finder
Cloud computing has been around for over a decade, and I have been a follower, admirer, and user for much of this time. Recent advancements are extremely interesting to me in what they enable in the business landscape. I will go into this in more detail later in this article.
Before cloud computing
Before the advent of Amazon Web Services (AWS), it was a big investment for a business to scale globally. In the 2000s, I worked for Betfair, the world’s largest sports exchange platform. When Betfair needed to scale its presence around the globe, it was the data center by the data center. This meant sending a team of infrastructure engineers, network engineers, and often database administrators physically to London, California, Melbourne, Tasmania, Malta, Gibraltar, Frankfurt or Boston.
They would need to assess and select a data center, then procure racks of hardware to set up, build, mount, cable, and install. Then troubleshoot. It was slow – it took time to do all of this travel and skilled labor. It was a high barrier to entry. You needed the hardware to support your business over the coming one to three years, all acquired upfront. It was also a high-risk investment – an incorrect hardware selection would be something that you may have spent tens or hundreds of millions of dollars on that you now needed to live with one way or another. And it was costly and labor-intensive to maintain. Those machines needed patching, repairs, disk replacements, and cards swapped out on an ongoing basis. In 2010, Betfair’s engineering team numbered nearly 1,000 crew and more than half of them were in infrastructure: almost all the nine-figure CapEx and over half the nine-figure OpEx was spent on infrastructure – and it was efficient for the technology available at the time.
Cloud computing was a great enabler. Amazon’s genius was to abstract almost all of this. You paid for what you used. If you made a choice you wanted to change, you could throw it away and make a different choice through the console.
It meant startups didn’t need to spend most of their capital in hardware and infrastructure. Growing companies could migrate to a far more cost-effective paradigm. Amazon took on the maintenance of databases through RDS, networks and firewalls through VPC, caching through Cloudfront, and more. Replication around the world became a feature, rather than a project.
Competitors started to arrive. Microsoft Azure, Google Cloud Platform, and IBM Bluemix – and companies seeking resilience, redundancy and optionality started working with technologies like Docker and then Kubernetes to abstract infrastructure further. This has been a significant area of strategic investment for many companies over the last five years. A cloud-based infrastructure that is truly abstract from the host that it sits upon is a competitive advantage.
Multi-cloud and infrastructure-in-a-box
Finder is a global comparison platform, with 2.4 million unique visitors per month in Australia. This year, Finder has served around four times as much traffic as its nearest competitor and hosts around 40 percent of total Australian visits, which is more than the next five competitors combined.
Even with today’s technologies such as scripting and built-in automation, it can still take days or weeks to set up a new system or service. Using an abstracted virtualized platform approach with AWS has allowed Finder to create dozens of new services in entirely new clusters, from scratch, around the planet, automatically in days – not even a week. This allows the engineering squads to spend the following five months focusing on building a competitive product for Finder – which is a good thing for me. I also worry less about the load generated from advertising campaigns as Finder can just scale up.
Virtualization – all the way down
But technology advances again and multi-cloud today is a lot more built-in to the cloud provider’s products and features than it was five years ago. The abstraction of the platform and the cloud provider is a part of this. This means that in the same way as the effort that needed to be spent in the creation of a physical platform became less necessary 10–15 years ago, the creation of an abstracted platform will become less necessary in the near future.
The regress of virtualization is the megatrend here as more and more layers of abstraction are enabled between the application and the physical hardware actually running it.
This does evoke a call to action. If, like a number of companies, you have a number of significant project investments being made in the abstraction of cloud computing technologies (which I think is smart), look again at what the cloud providers are now providing in terms of products and features that are portable.
If databases, message buses, worker solutions, and containers are native and standard on more and more cloud providers, then there may be less of a need to build abstractions to these as the risk of a direct implementation may not ultimately tie you to a service provider and its revenue optimization strategies. You will be able to deploy faster by using abstraction provided to you by the cloud provider, and you won’t incur the risk of being locked in.