Show notes: The agent is in - Episode 58 - From monolith to modules with cfbs

The standard process for managing that monolithic set, it is a fair amount of git diffing. It’s not hard once you get used to it, but it’s still a lot to do and read.

In this episode we take a monolithic CFEngine policy set, the kind most of us have been managing for years in production, and turn it into cfbs-managed project using cfbs convert. We start with cfbs analyze to see what we’re working with, walk through the interactive conversion, and finish with running cfbs update to jump from masterfiles 3.24.0 to 3.27 in seconds.

Show Notes

The problem with monolithic policy sets

If you’ve been running CFEngine for a while, your masterfiles directory probably started as a clean copy of the Masterfiles Policy Framework, then someone added custom policy files. Someone else modified def.cf to change the agent run schedule.

Now try upgrading. You’re diffing files, hoping you don’t clobber a local tweak or miss an upstream fix. The upgrade documentation is thorough but it’s a lot to work through.

Look before you leap

cfbs analyze gives you a picture of what you’re working with before you commit to anything. Running it against our policy set, it told us:

Based on masterfiles 3.24.0
controls/def.cf has been modified (the schedule change)
Stale files from old CFEngine versions show leftover version-split library directories from the 3.6/3.7 era
Several custom files that aren’t part of upstream at all

This is the same analysis that cfbs convert uses under the hood to figure out what to keep, what to prune, and what to turn into patches.

cfbs convert

One thing to know about cfbs convert: it’s picky about the starting layout. It wants a directory containing a single subdirectory named masterfiles-<something>. That directory becomes the root of your new cfbs project. So we copied our policy set into a fresh directory as masterfiles-acme and ran cfbs convert from the parent.

It found promises.cf and detected masterfiles 3.24.0
It initialized a git repo and created a cfbs.json with 3.24.0 as the base
It picked up our custom policy files
It pruned all the unmodified vendored files, stripping the monolith down to just our stuff
For controls/def.cf, it showed the diff: someone had changed the agent run schedule to a 10-minute interval. We chose “keep as patch file”, which generated a .patch file and wired it up as a custom-masterfiles-patches module

What’s left is a clean cfbs.json that pulls in masterfiles 3.24.0, layers our custom policy on top, and applies the schedule patch. Everything else we never touched and it’s now gone from our repo because cfbs manages those now.

cfbs update

We ran cfbs build and got a working policy set with our patches applied. Good, but best part was:

cfbs update

Masterfiles jumped from 3.24.0 to 3.27. It noticed the build steps had changed between versions, asked if we wanted the new ones, and after a quick cfbs build we were running 3.27 with our schedule patch still cleanly applied.

That’s the whole point. An upgrade that used to mean a git diff marathon, done in one command.

Trailing commas in CFEngine lists

An audience question about a trailing comma in the schedule list is a useful syntax reminder. In CFEngine policy, trailing commas in lists are perfectly fine:

vars:
    "schedule" slist => {
      "Min00_05",
      "Min15_20",
      "Min30_35",
      "Min45_50",   # <-- valid CFEngine syntax
    };

Just don’t do this in JSON, your def.json augments files won’t tolerate it.

Video

The video recording is available on YouTube:

At the end of every webinar, we stop the recording for a nice and relaxed, off-the-record chat with attendees. Join the next webinar to not miss this discussion.