CFEngine Build System version 3

Our beloved cfbs CLI tool for working with CFEngine Build is rapidly evolving. At the time of writing, we are currently at version 3.2.1. Thus I would like to take this opportunity to talk a bit about the latest and greatest features; including support for users to manipulate input parameters in modules, as well as a couple of new build steps.

If you haven’t yet got a hold of the latest version of cfbs, you can update it with pip using the following command:

$ pip3 install --upgrade cfbs
Collecting cfbs
  Using cached cfbs-3.2.1-py3-none-any.whl (46 kB)
Installing collected packages: cfbs
Successfully installed cfbs-3.2.1

We’ll start off with a simple CFEngine Build project containing only the default master policy files. Note that I have disabled git, in order to make the examples a bit less noisy.

$ cfbs status
Name:        My awesome CFEngine Build project
Description: A project to demo new features in cfbs 3
File:        cfbs.json

Modules:
001 masterfiles @ a87b7fea6f7a88808b327730a4ba784a3dc664eb (Not downloaded)

New build steps

Build steps are usually added by module developers or advanced cfbs users in order to manipulate temporary files while building the policy set. An overview of all available build steps - and what they do - can be found in the cfbs repository here.

The build steps policy_files, bundles and input were added in cfbs version 3. The two former provide a more powerful way in which module developers can choose which policy files and bundles shall be evaluated by the agent during an agent run. This is in contrast to the autorun functionality that was previously used. While the latter build step is used to generate and merge augments from user input.

Adding local policy files

The main motivation for adding the policy_files and bundles build steps, was to improve the ease of use when it comes to adding local policy files for new CFEngine users. Previously bundles in local policy files had to be tagged with "autorun" using the meta promise type in order for the bundle to be evaluated. See example below:

# policy.cf

bundle agent my_bundle {
  meta:
      "tags"
        slist => { "autorun" };

  reports:
      "Hello from $(this.bundle)";
}

$ cfbs add policy.cf 
Added module: autorun (Dependency of ./policy.cf)
Added module: ./policy.cf

Adding a local policy file should be a primitive operation that can be used as a beginner example for new CFEngine users. Thus having to explain the autorun feature is undesirable. With the powers of our new build steps, cfbs version 3 is now able to ask you which bundle to run instead of users having to implicitly state this into their bundles.

# policy.cf

bundle agent my_bundle {
  reports:
      "Hello from $(this.bundle)";
}

$ cfbs add policy.cf 
Which bundle should be evaluated (added to bundle sequence)?
 1. ./policy.cf:my_bundle (default)
 2. (None)
 [1/2]: 1
Added module: ./policy.cf

Let’s see what this looks like if we have multiple bundles spread over multiple policy files. Given the two policy files below, we can add bogus/ as a local module directory. The CLI will browse the module directory looking for candidate bundles for you to choose from. I’ll choose the foobar bundle.

# bogus/bogus.cf

bundle agent foo {
  reports:
      "Hello from $(this.bundle)";
}

bundle agent bar {
  reports:
      "Hello from $(this.bundle)";
}

bundle agent baz {
  reports:
      "Hello from $(this.bundle)";
}

# bogus/doofus/doofus.cf

bundle agent foobar {
  reports:
      "Hello from $(this.bundle)";
}

bundle agent foobaz {
  reports:
      "Hello from $(this.bundle)";
}

$ cfbs add bogus/
Which bundle should be evaluated (added to bundle sequence)?
 1. ./bogus/bogus.cf:foo (default)
 2. ./bogus/bogus.cf:bar
 3. ./bogus/bogus.cf:baz
 4. ./bogus/doofus/doofus.cf:foobar
 5. ./bogus/doofus/doofus.cf:foobaz
 6. (None)
 [1/2/3/4/5/6]: 4
Added module: ./bogus/

If we build the project we can see the build steps in action:

$ cfbs build

Modules:
001 masterfiles @ a87b7fea6f7a88808b327730a4ba784a3dc664eb (Downloaded)
002 ./policy.cf @ local                                    (Copied)
003 ./bogus/    @ local                                    (Copied)

Steps:
001 masterfiles : run './prepare.sh -y'
001 masterfiles : copy './' 'masterfiles/'
002 ./policy.cf : copy './policy.cf' 'masterfiles/services/cfbs/policy.cf'
002 ./policy.cf : policy_files 'services/cfbs/policy.cf'
002 ./policy.cf : bundles 'my_bundle'
003 ./bogus/    : directory './' 'masterfiles/services/cfbs/bogus/'
003 ./bogus/    : policy_files 'services/cfbs/bogus/bogus.cf' 'services/cfbs/bogus/doofus/doofus.cf'
003 ./bogus/    : bundles 'foobar'

Generating tarball...

Build complete, ready to deploy 🐿
 -> Directory: out/masterfiles
 -> Tarball:   out/masterfiles.tgz

To install on this machine: sudo cfbs install
To deploy on remote hub(s): cf-remote deploy

By installing the policy set, followed by running the agent, we can see that the correct bundles are in fact run.

$ sudo cfbs install 
Installed to /var/cfengine/masterfiles
$ sudo cf-agent --no-lock --file update.cf
$ sudo cf-agent --no-lock
R: Hello from my_bundle
R: Hello from foobar

Pretty neat, right?

Adding user input

The greatest new feature by far is nothing less than the added support for user input in modules. This new feature allows for module developers to specify prompts that allow users to customize module parameters. User inputs can be added using the cfbs input command, or during cfbs add. Input is stored as JSON in the project repository, and can be converted into augments and merged into the policy framework using the input build step.

An example of a module that currently uses this feature, is the delete-files promise module, written by none other than the notorious CFEngine ninja Nick Anderson. Let’s check it out!

$ cfbs add delete-files
Added module: delete-files
The added module 'delete-files' accepts user input. Do you want to add it now? [yes/y/NO/n] yes
Collecting input for module 'delete-files'
Path to file: /tmp/virus
Why should this file be deleted? [Unknown] We don't want it to do malicious stuff on our host.
Specify another file you want deleted on your hosts? [yes/y/NO/n] no

We build and install using cfbs, followed by running the agent with the update.cf policy, just like we did in the previous example. Then we create a “good” file and a “bad” one in the /tmp/ directory. Running the agent now makes sure the bad file is deleted.

$ echo "Malicious code" > /tmp/virus
$ echo "Important stuff" > /tmp/important
$ sudo /var/cfengine/bin/cf-agent --no-lock --inform
    info: Deleted file '/tmp/virus'

A sneak peek

We are currently working hard on integrating the new input feature into the CFEngine Enterprise Web UI, Mission Portal. Without revealing too much of what is coming next, here is a little sneak peek on what we can expect.

Share your thoughts

If you have any ideas of new features or tips on how we can improve CFEngine Build? Please share your thoughts over at GitHub Discussions, we are eager to hear your feedback.