When we reflected in our team about what worked and what didn’t work so well during 2012, one thing stood out: the changes from release to release are too big, and that makes it bloody hard to test that everything works once all the different moving pieces are put together. Core functionality, policy language, upgrade policies, mongodb and apache configurations for enterprise, package names and content - any part moving without the other parts being adjusted will quickly lead to a frustrating experience for everybody.
Small, incremental changes that converge towards a desired state
Sounds familiar? Well, there’s a reason why cf-agent is designed to run every 5 minutes on your system. Systems drift, and without verifying the status and making adjustments frequently, that drift multiplies, amplifies and quickly infects the whole system until it reaches a point where change is nearly impossible and where fixing the unwanted side-effect of any change becomes as expensive as implementing the changes in the first place. The result is that releasing new stuff becomes the most dreaded and risky activity in an organization, and that innovation comes to a stand-still. We see that in IT and in software engineering alike - and it is an organizational as much as it is a technical problem: releasing software is the most important activity of any software company - every hour that went into writing a magnificent new feature is of no value until users can get their hands on it - and if that activity is painful, then it will only be fixed if everybody feels that pain constantly.
So, given that in CFEngine we truly believe in the value of agile system management and continuous self-healing of systems (and we, the engineers, are part of that system), it wasn’t particularly difficult to agree that we need to release our software system more frequently and with a smaller set of changes. Those changes will be fewer and more testable on a system level - we already have continuous integration, testing and packaging to a staging area. However, by executing the whole release pipeline frequently we can make sure that separate but inter-dependent parts of the system don’t drift apart from each other without anyone noticing. A big part of this is not just about shipping code, but about capturing and updating knowledge into a form that makes it accessible to all stakeholders, in- and outside the company.
Monthly release of CFEngine
“Pah”, you might say, “What’s the big deal; I’m running continuous delivery and release new software into production every 15 seconds!”. Well, we don’t believe that we would do our users much of a favor if we were to roll out a new version of CFEngine every day or every week - CFEngine plays a fundamental role in many IT systems, and no matter how much trust you have in our ability to do a good job in building rock-solid CFEngine releases, we don’t think it’s very attractive for anyone to look at CFEngine as a rapidly changing and moving piece of technology.
Starting with January 30th, we will release a tested snapshot of CFEngine Enterprise and Community every last Wednesday of the month. These monthly releases will be designed to be technology previews, not stable and fully supported releases for a production environment. Every month will be organized around
- a week of bug fixing
- two weeks of development; bug fixes will be released as maintenance release (ie 3.0.x)
- a week of testing, documentation updates and release execution
These snapshots will be made available to all of you (packaged for a subset of our supported platforms) so that you can play around with the latest and tested improvements of both Community and Enterprise in a test environment, without having to pull the latest code from the github repository and building it yourself. We are not quite sure yet how we will label those releases - we will probably call them “technology preview” or “alpha” releases.
Product-quality releases that are covered by our SLA and designed for release into production will then be launched in larger intervals. We think that we will have two such supported releases every year, which follows the rhythm that we had in 2012. These will be schedule-driven releases; work that is not completed when the release is built will be left out of the release - either not integrated in the first place, or disabled/hidden.
Let us know in the comments below what you think about this! In future blogs tagged with Development we plan to share some more information about what our own continuous integration and deployment pipeline looks like, and what role CFEngine plays in current and future versions of that infrastructure.
Volker Hilsheimer
I’m a software
engineer turned team lead and project manager. I believe that there are
universal patterns that we can observe in all systems, be it software,
hardware or human, and that the principles of a shared purpose, autonomy
of the individual and the desire to achieve mastery give us the energy
to make progress.
Twitter: @vhilsheimer
