In an office, high above New York City, we are looking at the screen of a computer, discussing the how the recent blog entry on CFEngine, SysAdmin 3.0 and the Third Wave of IT Engineering applies to the challenges of institutional agility on Wall Street and beyond.
`Speed is the product,’ says R.
R is a CFEngine customer from a large, and heavily regulated organization, that has been choked with bureaucracy and process management. He has just explained how deploying a server has gone from taking 3-6 months, to taking just a few minutes, thanks to his new CFEngine-based process. The present moves faster than the past, and the future moves even faster than that. By casting off by-gone vestiges of industrialized mass production in favour of Third Wave individual customization, he has turned a bureaucratic mess into an inexpensive triumph.
A prehensile tale of evolution
Say the word `agility’ and my mind jumps to images of monkeys swinging from tree to tree, or along vines in the jungle. I associate agility with sufficient speed and pliability to master one’s environment, Is that what we mean in IT infrastructure?
In IT, the challenge of the environment is the jungle of changing demand, business imperatives, and the striving for competitive fitness in a cut-throat marketplace; it is network requests to servers, spanning a landscape of different operating systems and their curiosities; it is a backlog of security patches and bug-fixes that need to be navigated, without losing balance. Superficially, this is quite different from the primate jungle, but mastering this other world is still about having sufficient speed and dexterity.
Agility is a function of our surroundings as well as our own capabilities. We cannot achieve agility without the tacit `cooperation' of the environment. For a long time, system administration assumed that systems were not dynamical entities – they were set up once and for all, and nothing really changed, except for the personal data you were working on. Today, we cannot turn away from the dynamical aspect of systems – change is the norm, and we use IT in ways we never did in the past. All of this demands much more from infrastructure engineers.
Van Doesburg said,`every machine is the spiritualization of an organism’; today it is absolutely true. For Web companies, like social media sites that are the embodiment of entire systems, designed to bring new features at a rapid pace, the organism doesn’t stand still for long. The need to be ahead in the game drives a scheme of `continuous deployment’, or what is being called `DevOps’ today. In biology, this running to stand still in the market has come to be called `the Red Queen’ hypothesis, from Alice through the Looking Glass:
“Well, in our country,” said Alice, still panting a little, “you’d generally get to somewhere else if you run very fast for a long time, as we’ve been doing.”
“A slow sort of country!” said the Queen. “Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!”
Knowledge and Dexterity
For our Wall Street customer, the bleak Second Wave regulatory management techniques were trampling the natural flora of requirements that struggled to grow out of different subcultures in their organization. The ability to embrace that diversity not only removed the human conflict in the system (by far the most expensive and damaging aspect of IT management), it empowered the organization by exploiting rather than wiping out the specialist knowledge of each part.
To be nimble enough for a Third Wave information-rich marketplace, we see that three things are needed:
- Anticipation.
- Action.
- Adaptation.
These are all enabled by knowledge. In other words, we need to see what is coming, be able to respond quickly or even proactively to it, and finally we have to tailor that response to the challenge of the day.
This is not just a matter of buying faster machines, or hiring more people. Today’s problem is one of information management. Every challenge requires of us:
| Challenge | Tool |
|---|---|
| To comprehend the challenge | Knowledge and insight |
| To solve the challenge | An expressive language |
| To respond to the challenge | A rapid, scalable deployment engine |
| To confirm or verify the response | Up to date and relevant reporting |
As we move into this age of information, the machinery we employ to this end has to be knowledge driven, lightweight and maintainable – it has to transcend the industrial and support the `organic’ – because if you are oversimplifying the reality of the jungle by insisting on the a regularity of a parking lot, you will have a lot of expensive construction work to wipe it out and you will bear the long-term cost of the waste.
At CFEngine we designed technology to support diversity for this reason.
From packages to promises - a language for talking about systems
Granularity is the key to agility.
In the early 1990s, a major stride forward was made when package-based software installation was created. Early systems like HPUX no longer had to install `everything or nothing’ from a master tape. Packages replaced the `golden image’ method of steam-rollering systems into `die Norm’, allowing customization at the level of selecting applications.
Today, by comparison, packages look like quite a blunt instrument: we not only need to modify the set of packaged components, but configure and tune the internal details of how these software components operate. In short, we must create the kind of seamless organism that the original provider could not possibly have imagined. This is what creates fitness-for-purpose in today’s IT marketplace.
Such creativity requires very specific adaptations that tie together an ecosystem of contributing elements: web service, directory service, storage management, security, etc. Pre-packaging fails the agility test as the details of local needs cannot be known in advance by the operating system vendor, and pre-packaged.
A simple analogy makes it clear: a tin of soup, or a microwave meal might be a superficially fast or easy way to make dinner in a modern age, but this ease does not bring agility. The day you get a visitor with special dietary requirements (vegetarian or allergic etc) then the prepackaging is a major obstacle to adaptation: the recipe cannot be changed and repackaged without going back to the factory that made it.
Thus packaging as a strategy, or even as a menu of packages, is merely an over-simplification, and we end up having to work around the technology with an intervention. For instance, to order the perfect steak, you need a descriptive languageto explain the correct level of cooking, the side orders, and the choice of wine, etc. You neither want to settle for a prepackaged dinner, nor do you want to go into the kitchen and play with the utensils to make the food yourself.
# The CFEngine food replicator food: table_1.seat_3:: "lamb steak" cooked => "medium rare", seasoning => { "rosemary", "salt" }; table_1.seat_2:: "beef steak" cooked => "bloody", seasoning => { "salt" };CFEngine’s modelling language offers this kind of surgical, precision-control over the smallest ingredients in a configuration, and even allows you to package your own containers or work with other suppliers’ packages. This ensures that adaptability is not sacrificed for superficial ease. Rather than working with tins of soup, CFEngine deals with the raw ingredients to whip up a meal easily – something like Star Trek’s food replicators. Of course, this means you need to know a few basics about cooking to set up the initial choices, but once you do, you have a lasting investment that is agile and cheap for the long term.
Cloud-hopping: from demolition to renovation
IT-agility has found a natural partner in virtualized machine services, where capacity itself can be adjusted to changing demand on a continuous basis. For many companies, especially in the United States, public or private clouds are a rational approach to managing such resources. The technology, however, is still in its infancy for consumers.
For what seems like a refreshing antidote to the `tear down and rebuild’ methods of iron age industrialization, the Cloud does little to rescue consumers from such techniques – it only makes them look more affordable, by allowing it to happen with greater ease.
Cloud does what paging did for disks, and what packet transmission did for networks: it breaks down infrastructure into small units that can be handled cheaply in a throw-away manner. If you lose a piece, it doesn’t cost the world to replace it. If there is an error, throw the piece away and make a new one. But while this `demolish and rebuild' approach sounds easy at the level of disk sectors and network packets it is a different matter altogether at the level of a working server or a building full of people. This is not agile management – it is just cheaper brute force, and it results in loss of service.
To manage the cloud service, companies like Amazon have embraced modern information rich approaches to their own infrastructure. Ironically, what is delivered to customers is pretty much an old industrial-age tin of soup they have to make do with, or modify on their own – but this is about to change.
Configuration management in the CFEngine way can take a `tin of soup’, tap it with the magic wand of policy, and make it transform itself into the desired meal without destroying the tin, and even while the meal is being eaten! Moreover, CFEngine is a distributed agent approach to continuous change and maintenance that can arrange for different meals for different consumers without much overhead – an obvious answer to the problem of one-size fits all. It brings customization from within, altering the raw materials, or mixing pre-packaged ingredients at any level of detail. The key point is: it is a knowledge-based approach to customization, and that is the only way to scale up agility without expensive brute force.
Agility is knowledge: knowledge is the product
With massive scale, operational complexity and a rapid pace being the order of the day, a knowledge-based approach enables anticipation of change, and makes it cheap to model the present and future to adapt a system in real time. Knowledge is what will take configuration technologies forward, in or out of the cloud. This is why CFEngine 3 was redesigned as a knowledge engine, as much as a configuration or change engine. Configuration itself is knowledge.
For system infrastructure engineers, the job is increasingly going to be about content. Infrastructure engineers will be writers and teachers, learning to express intended behaviour simply and clearly with language, keeping knowledge explicit so it can be shared amongst the developers – who make the wheels of commerce turn – and the infrastructure engineers – who underpin society by making the wheels.
The future of agility lies in separating the what (knowledge) from how (action), i.e. separating slow change (infrastructure) from rapid change (application). This needs sharp, lightweight tools with fast reaction times. CFEngine 3 (Community and Nova) and their continuously improving range of capabilities is our answer to this challenge, and unlike package based approaches, the scope for improving our technology is still an open vista.
