The Future is in the Cloud; an Interview with Michael Purvis
This week we interviewed partner Michael Purvis, chief Data Engineer for Northwestern Analytics about the future of cloud computing.
Tony: Hi Michael! Thank you for taking the time to meet with me. Lets jump in! What is your background in cloud technology?
Michael: My background in cloud technology includes building custom ELT (Extract, Load, Transform) pipelines in Amazon Web Services (AWS) and building an Enterprise-grade Data Platform in Azure. In both efforts I was part Architect and part Engineer. I always try to build solutions on top of serverless architecture before standing up dedicated virtual hardware.
I spent almost two years building, deploying, and maintaining custom ELT pipelines in AWS using services including IAM, EC2, VPN, load balancers, S3, PostgreSQL DB, Redshift, SNS, and Lambda. Over the past three months I’ve been helping my team build a new Enterprise-grade Data Platform in Azure using Azure Container Registry, Azure Kubernetes Service, Key Vault, Blob Storage, Azure DB for Postgres, and Azure Cache for Redis.
Tony: How did you get into this field?
Michael: After many years as a web applications developer my interest in cloud technology had grown strong enough that I decided to pivot and apply my python skills to Data Engineering. I began that new career path at a major ed-tech company on a team charged with building data pipelines in AWS. After getting up to speed, the team restructured and I found myself responsible for not just the engineering work, but also DevOps (development operations) and architecture. Building cloud-first solutions is fascinating and ever-evolving, so I currently do not have plans to pivot again.
Tony: As you know we use Azure DevOps for our internal development work. Can you talk about the strengths and weaknesses of the top three cloud providers – AWS, Azure, and GCP?
Michael: Many articles and blog posts exist that pit these three platforms against one another and evaluate metrics such as “who was first to market”, or “which major companies use which platform”, or even “compute across the three platforms breaks down like this”. My response to this prompt will focus on my experience with each vendor and will offer a more subjective viewpoint.
I picked up AWS first because that’s what my team elected to use. I found the learning curve relatively steep, but I was also learning cloud technology in general – I didn’t have prior experience with another vendor. The documentation can be difficult to follow and the support team was sometimes lackluster, but overall they have a strong platform with more services to choose from than anyone else.
I may be biased since I started in AWS first and then picked up Azure second but, for my money, the Microsoft folks provide a more user-friendly portal, they maintain a more thorough and easier-to-read set of documentation, and I have yet to be disappointed – or even just unimpressed – by any of the support engineers who’ve worked cases with me. To boot, Microsoft continues to integrate Azure services with an array of Office 365 apps. Add in Teams, and now you have a combination of services that make it difficult for any organization to pass on Azure without serious consideration.
Tony: What do you think about Google’s horse in the race?
I do not have first-hand experience with Google Cloud Platform, but I know they offer services comparable to AWS and Azure. As applications become more heavily containerized, I would look to Google – the original creators of Kubernetes – to become the go-to for organizations providing PaaS or IaaS offerings.
All three of these vendors have launched initiatives to support users in need of hybrid (on-prem and cloud) and multi-cloud (multiple vendors) options. As AWS and GCP are relative newcomers in this space, my go-to here would be Microsoft.
Tony: For companies still utilizing on premise data centers, what is your recommendation to them?
Michael: Any company that has not begun at least partial transition to cloud-based services should undertake initiatives to do so immediately.
Tony: Especially with so many employees working from home I am sure this is even more critical. Where do you see the future of cloud?
Michael: Cloud technology continues to expand and evolve and I fully expect that every organization will have at least a hybrid approach before too long. That said, information security in the cloud is a totally different beast compared to physical security. When designing for the cloud, we need to move security right to the top of the priority list and build it in from the start. As more users move resources to the cloud, more resources are surely becoming available as targets, broadening the threat landscape.
Tony: What do you think is the next step beyond cloud?
Michael: The next step beyond cloud? Companies will offer generic cloud services. These companies will themselves be clients of all the major cloud providers and the value-add will be alleviating the stress of hemming and hawing over the different competitors by abstracting services away from the provider level. New cloud vendors will stand up a business model whereby the customer doesn’t need to know which major player the company is using to provide which services. Example sales pitches might be “We can save you money by seamlessly moving your services between vendors according to an ROI strategy you dictate” or “We can store your data in redundant systems across availability zones within a single vendor, sure, but we can also sync your data with other vendors as well for the next level of availability and reliability”. That said, this is a serious example of “easier said than done”.