Custom promise outcomes in Mission Portal

CFEngine 3.17.0 introduced custom promise types, which enable CFEngine users to extend core functionality and policy language in a simple way. As an example of the power and simplicity of this new feature, I will show a promise type that helps to observe a website’s status. The module which implements this promise type was written in a couple of hours.

Creating a promise type for whether a site is up

We will use Python and the CFEngine library to implement a promise module. Our previous blog post, “How to implement CFEngine custom promise types in Python”, explains this in detail.

First, we import dependencies, re for regex matching, ssl to configure ssl verification, urllib for making HTTP requests, and finally PromiseModule, ValidationError, and Result from the cfengine module which implements the necessary protocol, results, logging and error handling.

import re
import ssl
import urllib.request
import urllib.error
from cfengine import PromiseModule, ValidationError, Result

We then create a subclass of PromiseModule and use the constructor to define the promise module name and version:

class SiteUpPromiseTypeModule(PromiseModule):
    def __init__(self):
        super().__init__("site_up_promise_module", "0.0.3")

After that, we can implement validation under the validate_promise method. In my case, I check if the URL is valid, for that I’m performing pattern matching and if the URL isn’t valid then I raise a ValidationError.

def is_url_valid(self, url):
    regex = re.compile(
        r"https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,6}\b([-a-zA-Z0-9()!@:%_\+.~#?&\/\/=]*)",
        re.IGNORECASE,
    )
    return re.match(regex, url) is not None

def validate_promise(self, promiser, attributes, metadata):
    if not self.is_url_valid(promiser):
        raise ValidationError(f"URL '{promiser}' is invalid")

Finally, we need to implement evaluate_promise a method that performs the main business logic. Here I check if skip_ssl_verification exists in attributes and if so create unverified SSL context to avoid SSL verification. This can be useful if you are setting up a web server with a self signed certificate, for example for testing. Then I do HTTP requests and if there are no errors I return the promise result as kept:

def evaluate_promise(self, promiser, attributes, metadata):
    url = promiser
    ssl_ctx = ssl.create_default_context()
    if (
        "skip_ssl_verification" in attributes
        and attributes["skip_ssl_verification"] == "true"
    ):
        ssl_ctx = ssl._create_unverified_context()

    error = None
    try:
        code = urllib.request.urlopen(url, context=ssl_ctx).getcode()
        self.log_verbose(f"Site '{url}' is UP!")
        return Result.KEPT
    except urllib.error.HTTPError as e:
        # HTTPError exception returns response code and useful when handling exotic HTTP errors
        error = f"Site '{url}' is DOWN! Response code: '{e.code}'"
    except urllib.error.URLError as e:
        # URLError is the base exception class that returns generic info
        error = f"Site '{url}' is DOWN! Reason: '{e.reason}'"
    except Exception as e:
        error = str(e)

    assert error is not None
    self.log_error(error)
    return Result.NOT_KEPT

Download the full module here.

Using the new custom promise type

Now that we’re done with the custom promise type module we need to write the CFEngine policy to use it.

First, we define the custom promise type specifying the path to the module that implements the custom promise type as well as the interpreter that should be used.

promise agent site_up
{
  path => "/var/cfengine/modules/promises/site_up.py";
  interpreter => "/usr/bin/python3.8";
}

After that, we can define the bundle as we usually do in CFEngine. The newly added promise type site_up is available and we set a URL as the promiser.

bundle agent site_up_autorun
{
  meta:
    "tags" slist => { "autorun" };

  site_up:
    "http://172.28.128.10";
    "https://cfengine.com/";
    "https://cfengine2.com/";
    "https://unavailable.com"
      skip_ssl_verification => "true";
}

Download the policy here.

The result

Policy analyzer

Let’s check the result in Mission Portal’s Policy analyzer to see what’s happening in our CFEngine policy:

On the left side, policy files are listed in the file manager. On the right side, we see the content of the selected policy file, in my case site_up.cf. Not kept promises (inaccessible sites in other words) are highlighted in red. Below you can see all information presented in the table with log messages that fully describe the root cause.

Email notification

Additionally, we can create an alert to get an email notification once a site is down:

Compliance report

Finally, we can use the conditions from the alerts as part of a compliance report (introduced in CFEngine 3.17.0):

Watch the Compliance Reports video for more details about this new type of report:

Summary

To summarize, we’ve gone through a few steps:

Writing a promise module (about 40 lines of Python code)
Defining the promise type in policy (pointing to the module)
Adding policy which uses the promise type, site_up, to monitor some websites
Seeing whether the sites are up in mission portal, by looking at the promise outcomes in Policy Analyzer
Creating alerts for when sites go down
Creating a compliance report, a checklist of all the websites we need to stay up

This is just a small example, but it shows how easy it is to build a new promise type in CFEngine and see the results of those promise evaluations in Mission Portal. Here, we made a simple promise type that primarily provides reporting data, it doesn’t change anything on the system, but custom promise types are not limited to reporting, they can be used to manage new types of resources as well. To achieve this, we’d just expand the evaluate_promise method, with more code for what we want to check and change on the system. For an idea of how to do this, see the example in our previous blog post.