Introducing CFEngine custom promise types

In CFEngine 3.17, custom promise types were introduced. This allows you to extend policy language, managing resources which don’t have built in promise types. The implementation of custom promise types is open source, and available in both CFEngine Enterprise and CFEngine Community. To implement a new custom promise type, you need a promise module. (The promise type is what you use in policy language (the concept), while the module is the underlying implementation - can be a python script, compiled executable or similar).

Promise modules can be written in any programming language

The only requirement is that it communicates with the CFEngine agent on standard output and standard input. CFEngine (cf-agent) spawns a subprocess with the module, and sends it requests to validate and evaluate promises.

In this blog post, we will show how to install and use an example module. This example module is written in python, and uses our cfengine python library to simplify the implementation a lot. The module itself is just 39 lines of python code. Please note that it is just a minimal example, it does not implement everything you would expect from a complete, production ready, promise type. If you are interested in more technical details, you can read the specification.

The limitations of built in promise types

Anything can be managed using a combination of commands, files, users, and packages promises, but this can be tedious and error prone. As an example, if you want to use git to clone a repo, and then keep it up to date, you might end up doing something like this:

A packages promise to install git
A users promise to create the user which will clone git repositories
Some files promises to create folders, git config files and enforce permissions
Some commands to do what you want; git clone, git pull, git config, git remote, git checkout

The commands promises can be troublesome; the order matters, they have dependencies and they are not always idempotent. To do this well, you should use some combination of promise handles, if attributes, and classes to control which commands to run, and when. With a custom promise type, you can abstract things away from the policy writer. When writing policy, they can focus on what git repo to clone where, and write clean, readable policy, while leaving the technical details up to the module.

Installing a Promise Module

Using Masterfiles Policy Framework, adding modules is quite straight forward, you just copy the files into the right folder. Let’s use the example git promise type:

$ git clone --recursive https://github.com/cfengine/core
$ git clone --recursive https://github.com/cfengine/modules
$ mkdir /var/cfengine/masterfiles/modules/promises
$ cp core/misc/custom_promise_types/cfengine.py /var/cfengine/masterfiles/modules/promises/
$ cp modules/promise-types/git/git.py /var/cfengine/masterfiles/modules/promises/

(It is recommended to keep your masterfiles in a git repo, and you should commit your modules to it).

A normal policy run copies the modules to the right locations, and adjusts permissions:

$ cf-agent -Kf update.cf && cf-agent -K
$ ls -l /var/cfengine/inputs/modules/promises/
total 12
-rw------- 1 root root 5416 Dec  3 14:50 cfengine.py
-rw------- 1 root root 1352 Dec  3 14:50 git_using_lib.py
$ ls -l /var/cfengine/modules/promises/
total 12
-rwxr-xr-x 1 root root 5416 Dec  3 14:50 cfengine.py
-rwxr-xr-x 1 root root 1352 Dec  3 14:50 git_using_lib.py

This is identical to how package modules are handled. The update policy copies /var/cfengine/masterfiles from the policy server to /var/cfengine/inputs on all hosts. Modules are then placed in /var/cfengine/modules/promises with correct permissions, which is the preferred location to run them from.

Defining and using custom promise types in policy

To define a new promise type in policy, we use the promise block:

promise agent git
{
  path => "/var/cfengine/modules/promises/git_using_lib.py";
  interpreter => "/usr/bin/python3";
}

For now, you’ll have to do this in any policy file where you use the promise type. In the future we’ll let you define all your custom promise types in one place. After that, you can use the git promise type just like any other promise type:

bundle agent main
{
  git:
    "/home/ubuntu/kubernetes"
      repository => "https://github.com/kubernetes/kubernetes";
}

Here is the complete example policy (/var/cfengine/masterfiles/services/autorun/git_clone_kubernetes.cf):

promise agent git
{
  path => "/var/cfengine/modules/promises/git_using_lib.py";
  interpreter => "/usr/bin/python3";
}

bundle agent git_clone_kubernetes_autorun
{
  meta:
      "tags" slist => { "autorun" };

  git:
    "/home/ubuntu/kubernetes"
      repository => "https://github.com/kubernetes/kubernetes";
}

The example uses autorun functionality, to be automatically added to inputs and bundlesequence. You can use /var/cfengine/masterfiles/def.json to enable autorun:

{
  "classes": {
    "services_autorun": ["any"]
  }
}

The result

To summarize, at this point you should have added 4 files:

/var/cfengine/masterfiles/modules/promises/cfengine.py - The CFEngine promise module library
/var/cfengine/masterfiles/modules/promises/git_using_lib.py - The git example module
/var/cfengine/masterfiles/services/autorun/git_clone_kubernetes.cf
- Our policy which uses the git promise type to clone the kubernetes repository
/var/cfengine/masterfiles/def.json - Augments file to enable autorun

Remember to commit these files to your masterfiles git repo.

We can do a full agent run again, and observe the result:

$ cf-agent -Kf update.cf && cf-agent -KI
    info: Cloning 'https://github.com/kubernetes/kubernetes' -> '/home/ubuntu/kubernetes'...
    info: Successfully cloned 'https://github.com/kubernetes/kubernetes' -> '/home/ubuntu/kubernetes'
[...]
$ ls /home/ubuntu/kubernetes/
BUILD.bazel               SECURITY_CONTACTS   hack
CHANGELOG                 SUPPORT.md          logo
CHANGELOG.md              WORKSPACE           pkg
CONTRIBUTING.md           api                 plugin
LICENSE                   build               staging
LICENSES                  cluster             test
Makefile                  cmd                 third_party
Makefile.generated_files  code-of-conduct.md  translations
OWNERS                    docs                vendor
OWNERS_ALIASES            go.mod
README.md                 go.sum

Cloning kubernetes or any big git repo for the first time can take a few minutes, be patient. If you run the policy again, the module will see that the repo is already cloned, and will not do anything. A real git promise module should, in that case, go inside the folder and run more git commands, for example git pull.

Additional resources

If you’d like to learn more, here are a few useful resources: