How to implement CFEngine custom promise types in bash
Posted by: Aleksei Shpakovskii
This blog post will focus on the bash programming part of implementing a promise type. To understand what custom promise types are, and how to use them, you should read the introduction first.
To implement a custom promise type in CFEngine, you need a promise module. The module is an executable, and can be written in any language. It’s possible to write one from scratch, but to make it as easy as possible, we decided to provide libraries for common programming languages. In our previous blog post, we showed how to implement modules in Python. That’s great, it’s a powerful, expressive and readable programming language, however there is one drawback; installing python. Many systems don’t have python already, or have a version which is too old. So you will need to add policy to install / update Python, to make sure modules work correctly everywhere.
CFEngine runs on a wide variety of platforms; Linux, BSD, Windows, Mac, HP-UX, IBM AIX, Solaris, etc. Most of these platforms have some version of bash already installed (except for Windows). Thus, by writing promise modules in bash we can make them very portable, and easy to use everywhere without additional dependencies.
Using the CFEngine promise module bash library
The library implements the protocol and the main loop of the program, receiving and answering requests from cf-agent
. The following examples will be based on it, but authors of shell-based custom promise modules are welcome to look through the source code and use it as they find fitting. To use the library, the module script should set environment variables and define functions, then source the library script and call module_main
function with two arguments: The module name and version.
The most trivial example, which does nothing, looks like this:
do_evaluate() {
response_result="kept"
}
. "$(dirname "$0")/cfengine.sh"
module_main "trivial_example_module" "1.0"
The function do_evaluate
is called during policy evaluation. It must set response_result
variable, otherwise policy evaluation will fail. After sourcing the library, module_main
is called with module name and version as two arguments.
Note: Generally, you cannot add echo
or printf
statements like you would in a normal shell script, to print error messages or log what the module is doing. Your module code should be silent (not print to stdout). This is because the module is communicating with cf-agent on standard output, so its output needs to follow the protocol.
Logging
To send log messages to the agent (which will in the end be printed to the user), follow one of these examples:
log error "Could not create file 'blah'"
log info "Created file 'blah'"
log verbose "File 'blah' already exists, skipping"
log debug "Module 'create_file_module' received request to validate 'blah'"
Logging should only be added inside do_evaluate
and do_validate
functions. What to log and what the different levels mean is explained in the specification. If you need additional logging, which will not be printed to CFEngine users, write to a file instead:
Variables
Currently, there are three optional variables, which instruct the library to check validity of promise attributes.
required_attributes
– space-separated list of attributes which must be presented in each promise. If provided – each promise is checked for presence of these attributes, and fails validation if at least one of them is not present. If not provided, no such check is performed.
optional_attributes
– space-separated list of attributes which are allowed, but not required, in each promise. Example: optional_attributes="this that"
.
all_attributes_are_valid
– yes
or no
(default: no
). By default, any attribute which is not in list of required or optional attributes is considered invalid and renders promise invalid. If set to yes
, then no extra check for optional attributes is performed – all attributes are considered valid.
Functions
Currently, there is only one required function – which performs evaluation (the step which makes changes to the system), and three optional ones: to validate and to do setup / teardown if needed.
do_evaluate
– required, the function which does promise evaluation. As inputs it can use request_promiser variable and a set of request_attribute_*
variables – one for each specified attribute. As outputs, it must set response_result
variable to one of kept
, not_kept
, or repaired
strings.
do_validate
– optional, called to validate promise. Uses same inputs as do_evaluate
, as output it can change response_result
to valid
or invalid
. Note that the library itself checks presence of required_attributes
and absence of invalid attributes (if requested), and sets response_result
accordingly before calling this function. This function can change value of this variable. Also, you can check, for example, that request_promiser
is a valid absolute pathname (that it starts with /
).
do_initialize
– called once at the very beginning of module lifecycle, before any other function. Can be used for initialization / set up, if needed.
do_terminate
– called once at the very end of module lifecycle, after any other function. Can be used for cleaning up, if needed.
Copy promises
Let’s have a look at a less trivial example – copying files:
required_attributes="from"
optional_attributes=""
all_attributes_are_valid="no"
do_evaluate() {
if diff -q "$request_attribute_from" "$request_promiser" 2>/dev/null; then
response_result="kept"
return 0
fi
if cp -a "$request_attribute_from" "$request_promiser"; then
response_result="repaired"
else
response_result="not_kept"
fi
}
. "$(dirname "$0")/cfengine.sh"
module_main "cp" "1.0"
This declares cp
custom promise type, which has one required attribute: from
, and no optional attributes. In do_evaluate
, value of this attribute is available in request_attribute_from
variable.
Also note that instead of overwriting the target file every time, this promise module first compares source and destination, and does the copying only if they differ. Promise result reflects that: the promise will be “kept” if source and destination were the same before policy evaluation, “repaired” if they were different, but cp
command succeeded, and “not_kept” if cp
command failed.
In the policy it can be used like this:
promise agent cp
{
interpreter => "/bin/bash";
path => "$(sys.inputdir)/cp.sh";
}
bundle agent main
{
cp:
"/tmp/dst"
from => "/tmp/src";
}
When evaluating above policy, do_evaluate
will be called with variables like this:
Backup promises
As a final example, I’ve added my pet project, namely backup3, as a CFEngine custom promise module:
required_attributes="from"
optional_attributes="root"
all_attributes_are_valid="no"
do_evaluate() {
# Prepare environment for wrapped script
BACKUP_ROOT="$request_attribute_root"
BACKUP_BIN="$(dirname "$0")/backup3"
BACKUP_KEEP_TMP="1"
run_this() {
run_rsync always "$request_promiser" "$request_attribute_from"
}
# call the script
. "$BACKUP_BIN/backup.sh"
# Analyse its results
if [ "$?" != 0 -o ! -f "$BACKUP_TMP".files ]; then
response_result="not_kept"
rm -f "$BACKUP_TMP".sql "$BACKUP_TMP".files
return 0
fi
# Note: $BACKUP_TMP".files file contains lists of created and deleted files,
# one file per line, separated by line saying "separator", and first line
# being literally "first line". So if script succeeded, but there were no
# changed files, then this file has only 2 lines.
local filelist_lines="$(cat "$BACKUP_TMP".files | wc -l)"
if [ "$filelist_lines" -gt 2 ]; then
response_result="repaired"
else
response_result="kept"
fi
# Clean up
rm -f "$BACKUP_TMP".sql "$BACKUP_TMP".files
}
. "$(dirname "$0")/cfengine.sh"
module_main "backup" "1.0"
From first three lines we see that promises for this module must always have a “from” attribute, and, optionally, a “root” attribute (if not provided, it’s expected that the backup3/backup.sh
script called in do_evaluate
will figure it out).
In the do_evaluate
function we see a different order of operations compared to the “copy promise”: instead of checking if something needs to be done first, promise module always first calls the backup3/backup.sh
script, and only then checks if it did anything. That’s due to how the backup3 is written: it always first runs rsync
, and then checks if it did anything. The promise module script analyzes one of its temporary files and concludes if there were any changes or not, based on its contents.
Summary
That’s it. Whether you’re using bash, python, or another programming language, extending CFEngine policy language has never been easier. If you encounter any issues or have any questions when working with promise modules, we encourage you to post them here. We are excited to see what modules the community creates, and to share more than just example modules soon.
If you want to learn more, here are some useful resources: