10,000 Servers Managed Per System Administrator. Is That Possible?

Posted by Thomas Ryd
November 18, 2013

William Gibson once said: “The future is here, but it is not evenly distributed”. Within the space of IT-operations and automation we see clear evidence of this. Last week at LISA, System Engineer Mike Svoboda from LinkedIn gave us a glimpse of insight into the future of infrastructure operations.

The lucky audience got to learn how LinkedIn automates IT-operations at one of the largest websites in the world. What LinkedIn has achieved with less than a handful of engineers the last couple of years is truly bleeding edge.

With a clear mandate from management to automate all aspects of their IT-operations 4 years ago, LinkedIn, with Mike Svoboda in the front seat, embarked on a journey that currently is among the most impressive we have seen here at CFEngine. Today every infrastructure change at LinkedIn is version controllled and automated.

Amazing results

At any point in time, LinkedIn can look back in time to see who did what on any server, within seconds. Per Mike Svoboda LinkedIn has real time visibility into all of their systems. New changes are (in a phased manner) deployed into production on a daily basis. “We know the impact of a change before we push it.” Hardware failure detections and replacements are fully automated, scaling of the systems is horizontal and software based (no physical load balancers or unnecessary additional tiering). LinkedIn has even taken the art of automation to the level where they can spin up and down logical data-centers anywhere in the world within minutes!

Thanks to CFEngine and the resultant automated infrastructure, LinkedIn infrastructure staff can focus their efforts on higher order initiatives. Once you have achieved the level of automation we have seen at LinkedIn, adding more servers is a piece of cake. This is why the same handful of engineers that started out on this journey manages their large scale deployments today. What is even cooler, is that there are no plans for adding more engineers as LinkedIn continues to successfully expand its business. It is all automated! The future is not evenly distributed.

Significant business value

LinkedIn extracts significant business value from its highly automated IT-operations. With daily changes into production, LinkedIn is able to test out new products quickly and adjust to market feedback faster than its competition. If it takes an average company about 4 months to deploy, test and rol-out a new service, while LinkedIn in the same period of time has been able to adjust its new service to market feedback several times, there is no magic to figuring out who is the most competitive.

With automation comes also significantly higher quality of service and higher customer satisfaction due to pleasant service experience in terms of latency and “it just works as expected”. Improved quality of service also means that a larger portion of changes relate to new features, and not fixes.

The sum of this can be translated into business agility. The lean IT-operations we see at LinkedIn saves the company millions of dollars in labor costs, compared to less automated operations, and it allows the company to move much faster than its competitors.

The future will hit you sooner thank you think

In today’s world where more and more products become software defined, the negative impact of not running a highly automated IT-operations increases. Soon, highly automated IT-operations will become a prerequisite for staying in the game. If your competitor moves faster, is leaner and provides higher quality of service then you will soon be out of business. More and more companies are realizing this, and we see a fast growing interest in programmable infrastructures across all industries.

Highly automated IT-operations is not only for fast-moving west-coast companies. This next generation IT-operations spans all industries. A few weeks ago, CFEngine was inducted into the JPMorgan Chase hall of innovation (link:https://cfengine.com/blog/cfengine-inducted-into-the-jpmorgan-chase-hall-of-innovation-1). Take this is as a sign of what is soon going to affect businesses across all industries, including your own. If you are a system administrator, embrace the future. Your future is bright (link: https://cfengine.com/blog/from-system-administrator-to-system-engineer).

Congratulations to LinkedIn. What you have achieved is world-class!

Learn more about CFEngine by visiting our website

Evaluate CFEngine for free (upto 25 licenses, unlimited usage)

Get in touch with us
to discuss how we can help!
Contact us
Sign up for
our newsletter
By signing up, you agree to your email address being stored and used to receive newsletters about CFEngine. We use tracking in our newsletter emails to improve our marketing content.