Cloning git repos and creating systemd services with CFEngine

Using modules, you can add custom promise types to CFEngine, to manage new resources. In this blog post, I’d like to introduce some of the first official modules, namely git and systemd promise types. They were both written by Fabio Tranchitella, who normally works on our other product, Mender.io. He decided to learn some CFEngine and within a couple of weeks he’s contributed 3 modules, showing just how easy it is to implement new promise types. Thanks, Fabio!

Getting started

If you are using CFEngine 3.18 LTS, and a policy set based on the default one we distribute, it’s very easy to get started with custom promise types.

First, add the modules and any supporting libraries to /var/cfengine/masterfiles/modules/promises/. For example, you can add these 3 python files:

cfengine.py (Library)
git.py (module for git promise type)
systemd.py (module for systemd promise type)

(If you have your policy in git, or another version control system, make sure you commit these modules to version control).

Additionally, you need to enable the promise types using a promise type definition, typically placed /var/cfengine/masterfiles/services/init.cf:

promise agent git
{
  path => "$(sys.workdir)/modules/promises/git.py";
  interpreter => "/usr/bin/python3";
}
promise agent systemd
{
  path => "$(sys.workdir)/modules/promises/systemd.py";
  interpreter => "/usr/bin/python3";
}

There are 3 pieces of information to be aware of here. promise agent git defines a new promise type, only for the cf-agent component. This means you can use it in policy files by writing git: inside agent bundles. path is the path to the module itself (a script / executable). interpreter is the path to a program which runs the module. In this case, two python processes will be started by the agent, like this:

/usr/bin/python3 /var/cfengine/modules/promises/git.py
/usr/bin/python3 /var/cfengine/modules/promises/systemd.py

Cloning git repos

Once you have added and enabled the modules, you can use them from anywhere within your policy set. As an example, we can clone two git repos:

bundle agent website_git_repos
{
  git:
    "/opt/hugoBasicExample"
      repository => "https://github.com/gohugoio/hugoBasicExample",
      version => "master";
    "/opt/hugoBasicExample/themes/hugo-PaperMod"
      repository => "https://github.com/adityatelange/hugo-PaperMod",
      version => "master";
}

More documentation and examples for the git promise type is available here.

Creating a webserver systemd service

After cloning, we have a hugo site and theme, and would like to start a server. systemd is great for starting a process, and ensuring it keeps running, so let’s create a systemd service:

bundle agent website_systemd_service
{
  systemd:
    "website"
      name => "website",
      state => "started",
      unit_description => "My example hugo website service",
      service_exec_start => {"/bin/sh -c 'cd /opt/hugoBasicExample && hugo server -t hugo-PaperMod'"};
}

When writing systemd: as our promise type, we are using the systemd module, which handles creating a service, as well as enabling it. More documentation and examples for the systemd promise type is available here.

The complete example

The 2 bundles above are really all it takes to ensure a webserver is running, with the help of systemd, and content from git repos.

Here is a complete example which can be added as /var/cfengine/masterfiles/services/autorun/website.cf:

bundle agent website_hugo_install
{
  packages:
      "hugo"
        policy => "present";
}

bundle agent website_git_repos
{
  git:
      "/opt/hugoBasicExample"
        repository => "https://github.com/gohugoio/hugoBasicExample",
        version => "master";
      "/opt/hugoBasicExample/themes/hugo-PaperMod"
        repository => "https://github.com/adityatelange/hugo-PaperMod",
        version => "master";
}

bundle agent website_systemd_service
{
  systemd:
    "website"
      name => "website",
      state => "started",
      unit_description => "My example hugo website service",
      service_exec_start => {"/bin/sh -c 'cd /opt/hugoBasicExample && hugo server -t hugo-PaperMod'"};
}

bundle agent website
{
  meta:
    "tags"
      slist => { "autorun" };

  methods:
    "website_hugo_install";
    "website_git_repos";
    "website_systemd_service";
}

The first bundle, website_hugo_install uses the default package module to install hugo, if not installed already. Then, bundles for cloning git repos and creating systemd services are implemented with the modules, as explained above. Finally, the last bundle, website ties it all together, it is the entry point for the policy, and defines the order of the other bundles.

This bundle is tagged so it is automatically run, if you’ve enabled autorun functionality. Otherwise, you will have to manually add the policy file to inputs and the bundle to the bundle sequence.

You can enable autorun using /var/cfengine/masterfiles/def.json:

{
  "classes": {
    "services_autorun": [ "any::" ]
  }
}

If you run the policy, you should see something like this:

$ /var/cfengine/bin/cf-agent -KI
    info: Successfully installed package 'hugo'
    info: Cloning 'https://github.com/gohugoio/hugoBasicExample:master' to '/opt/hugoBasicExample'
    info: Cloning 'https://github.com/adityatelange/hugo-PaperMod:master' to '/opt/hugoBasicExample/themes/hugo-PaperMod'
    info: Enabled the service website
[...]

You can see it working using ps:

$ ps aux | grep hugo
root       64075  0.0  0.0   2608   608 ?        Ss   18:32   0:00 /bin/sh -c cd /opt/hugoBasicExample && hugo server -t hugo-PaperMod
root       64076  1.6  0.3 966980 64212 ?        Sl   18:32   0:00 hugo server -t hugo-PaperMod
root       64502  0.0  0.0   8160   740 pts/0    S+   18:32   0:00 grep --color=auto hugo

Or curl:

$ curl localhost:1313
<!DOCTYPE html>
<html lang="en" dir="auto">

<head><script src="/livereload.js?port=1313&mindelay=10&v=2" data-no-instant defer></script><meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="robots" content="noindex, nofollow">
<title>Hugo Themes</title>
[...]

Complete documentation for custom promise types is available here.

Tooling to manage policy and modules

We are working on tooling which makes it much easier to manage your policy, modules, and their dependencies. See the video below for a sneak peek: