my cloud passion

I am going to elaborate why I like the cloud so much. This is a fully personal story and I am totally aware that different people might have different thoughts, experiences or circumstances that might have let to different attitutes towards the topic.

To understand my cloud passion, I have to explain my datacenter experience. I have been responsible for our datacenter infrastructure consisting of (first) bare metal servers, switches and firewalls and (later) VMware based virtualization hardware for 14 years. We started in one building in the first years and later spread our infrastructure over 2 buildings.

The core problem from my point of view was, that our infrastructure was not redundant. Power and network outages always had severe impact on our business especially if you take into concern that back some years from this writing spinning hard disks were state of the art and they did not like power outages at all. So in the event of a power problem (and there had been a few in 14 years) it was quite common that some filesystems got corrupted and some servers even completely died. Almost every incident in the datacenter had to be dealt with manually by my team.

Being a system administrator is like playing the bass in a band. If you do it well, nobody notices you but if there is a problem, it is obvious for everyone that something is going wrong. I am not going to complain, it is part of the job after all. But from my perspective, a fully redundant design would have made my life much easier. But having a redundant design means paying (roughly) the double amount of money to run a (mirrored) datacenter and that wasn’t affordable for a long time for my mid sized company. And in the end even at times when (maybe) it would have been affordable my team did a great job getting things back to work in terms of hours (not days) and so this was the way to go.

I never liked that fact that hardware could break for no obvious reasons and that our means to plan for disaster was limited with no datacenter redundancy. Of course we spread traffic and workload over diverse servers and racks and planned for disaster to some extent but still sometimes you had to accept single points of failure at least if you take earthquake, floods our power failures etc. into consideration.

Another big aspect of datacenters is capacity. In my company we face quite a fluctuation of user requests dependend on the weather (just like Amazon does for black fridays or Christmas compared to normal business days). Capacity planning is a challenge when you have a few days per year with 10 times the traffic of “normal” days. You need flexibility and scaling mechanisms. If you pay the infrastructure to serve peak days this is not ideal as this infrastructure will be idle for 99% of the year with you paying the bill.

To ease the problem I introduced a CDN (Akamai) to my company at some point, offloading 90% of our requests and that helped a lot for many years.

AWS was introduced in 2006 and I am not sure when I first noticed AWS. But I clearly remember that in one of those nights when dealing with outages my thought was that it would be so much easier to manage your dataserver via a web browser (not having to wait on the phone to get a technician into the campus and describing them the way to your rack and then what to do) or a CDK or CLI and how perfect it would be if things were redundant, resilient and scalable.

Speaking of scalability I remember so many times when we started webservers in advance in anticipation of severe weather conditions (that always meant more traffic on our servers as people consume weather in these situations way more likely than on “boring” sunny days) so autoscalability also as on my wish list.

All these things are part of the concept of cloud providers and when my company started to make first steps into the cloud, I got aware of my dream coming true:

Redundancy, (auto)scalability (elasticity) and resiliancy by concept from a big company that faced the same issues as we did (variable requests triggered by events like black fridays or christmas) but had a bigger wallet so to speak 😁.

Regions (meaning clusters of availibility zones at least hundreds of miles apart spread all over the world), availability zones (datacenters spread hundreds of miles apart in one region), auto scaling groups, fully managed services like load balancers were just what I was dreaming of for years.

With these possibilities in mind I started learning AWS after work in the evenings when I was alone in my office and certified as a Solutions Architect in 2018.

Of course cloud is no magic and it means servers, switches and infrastructure in datacenters. But the thing is, that these are redundantly run by Amazon and, yes, outages still occur but if you spread your architecture over availibility zones or regions the impact is minimal and automatically mitigated for you. That means less hassle and pain and a better sleep. I strongly believe that in a world of fast developing new technologies and specilizations it is advisable to have your datacenter infrastructure being run by specialists of cloud providers so that you can concentrate on architecture and your devops projects.

Autoscalibilty means that you have more servers for peak loads and that these servers automatically scale in once the load decreases with you paying just what you use. This makes so much sense for companies with changing traffic patterns like us.

20 years back it was quite common to have a small datacenter or servers with a “big” uplink to the internet (where big has shifted to mbit/s in 2000 to gbit/s and more these days) but nowadays for start up companies the cloud is the way to go. If you have an idea or just want to test something you need no capacity planning, “try and buy” or comparable steps that waste time, money and nerves. You just set up your cloud infrastructure test your ideas, “pay as you go” and when done delete everything and stop paying. It could not be easier from my perspective.

You don’t have to read detailied data cheets of hardware components anymore and learn their CLI. You just make use of tools and script your infrastructure with tools like CloudFormation or CDK all GIT controlled and redeployable. So even if things get broken or accidentically destroyed, redeploying them is nothing more than rolling out your stack again.

Another thing about cloud in general or AWS in particular is that they offer hundreds of services deeply integrated. So you have a tool box and having a task to solve means to become creative and choosing the right tools and combining them for your needs taking into regard aspects like efficiency, cost, security, scalability etc. Working as a solutions architect does not mean being repetitive but being creative and aware of services, their do’s and don’ts and their synergies. I like that a lot.

That’s why I love the cloud. That’s why I want to spend time on the cloud and let other people specialize in doing datacenter work (which is very much appreciated as I know both parts now, datacenter and the cloud).