The Complete Examples and Tutorials

Table of Content

  • Example Snippets: This section is divided into topical areas and includes many examples of policy and promises. Each of the snippets can be easily copied or downloaded to a policy server and used as is.

Note: CFEngine also includes a small set of examples by default, which can be found in /var/cfengine/share/doc/examples.

See Also:

Tutorial for Running Examples

In this tutorial, you will perform the following:

  • Create a simple "Hello World!" example policy file
  • Make the example a standalone policy
  • Make the example an executable script
  • Add the example to the main policy file (promises.cf)

Note if your CFEngine administrator has enabled continuous deployment of the policy from a Version Control System, your changes may be overwritten!

"Hello World" Policy Example

Policies contain bundles, which are collections of promises. A promise is a declaration of intent. Bundles allow related promises to be grouped together, as illustrated in the steps that follow.

Following these steps, you will login to your policy server via the SSH protocol, use the vi command line editor to create a policy file named hello_world.cf, and create a bundle that calls a promise to display some text.

  1. Log into a running server machine using ssh (PuTTY may be used if using Windows).
  2. Type sudo su for super user (enter your password if prompted).
  3. To get to the masterfiles directory, type cd /var/cfengine/masterfiles.
  4. Create the file with the command: vi hello_world.cf
  5. In the vi editor, enter i for "Insert" and enter the following content (ie. copy and paste from a text editor):

    bundle agent hello_world
    {
      reports:
    
        any::
    
          "Hello World!";
    
    }
    
  6. Exit the "Insert" mode by pressing the "esc" button. This will return to the command prompt.

  7. Save the changes to the file by typing :w then "Enter".

  8. Exit vi by typing :q then "Enter".

In the policy file above, we have defined an agent bundle named hello_world. Agent bundles are only evaluated by cf-agent, the agent component of CFEngine.

This bundle promises to report on any class of hosts.

Activate a Bundle Manually

Activate the bundle manually by executing the following command at prompt:

% /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf --bundlesequence hello_world

This command instructs CFEngine to ignore locks, load the hello_world.cf policy, and activate the hello_world bundle. See the output below:

# /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf --bundlesequence hello_world
2013-08-20T14:03:43-0500   notice: R: Hello World!

As you get familiar with CFEngine, you'll probably start shortening this command to this equivalent:

/var/cfengine/bin/cf-agent -Kf ./hello_world.cf -b hello_world

Note the full path to the binary in the above command. CFEngine stores its binaries in /var/cfengine/bin on Linux and Unix systems. Your path might vary depending on your platform and the packages your are using. CFEngine uses /var because it is one of the Unix file systems that resides locally. Thus, CFEngine can function even if everything else fails (your other file systems, your network, and even system binaries) and possibly repair problems.

Make the Example Stand Alone

Instead of specifying the bundle sequence on the command line (as it was above), a body common control section can be added to the policy file. The body common control refers to those promises that are hard-coded into all CFEngine components and therefore affect the behavior of all components. Note that only one body common control is allowed per agent activation.

Go back into vi by typing "vi" at the prompt. Then type i to insert body common control to hello_world.cf. Place it above bundle agent hello_world, as shown in the following example:

body common control
{
  bundlesequence => { "hello_world" };
}

bundle agent hello_world
{
  reports:

    any::

      "Hello World!";

}

Now press "esc" to exit the "Insert" mode, then type :w to save the file changes and "Enter". Exit vi by typing :q then "Enter." This will return to the prompt.

Execute the following command:
/var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf

The output is shown below:

# /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf
2013-08-20T14:25:36-0500   notice: R: Hello World!

Note: It may be necessary to add a reference to the standard library within the body common control section, and remove the bundlesequence line. Example:

body common control {

    inputs => {
       "libraries/cfengine_stdlib.cf",
    };
}
Make the Example an Executable Script

Add the #! marker ("shebang") to hello_world.cf in order to invoke CFEngine policy as an executable script: Again type "vi" then "Enter" then i to insert the following:

#!/var/cfengine/bin/cf-agent --no-lock

Add it before body common control, as shown below:

#!/var/cfengine/bin/cf-agent --no-lock
body common control
{
  bundlesequence => { "hello_world" };
}

bundle agent hello_world
{
  reports:

    any::

      "Hello World!";

}

Now exit "Insert" mode by pressing "esc". Save file changes by typing :w then "Enter" then exit vi by typing :q then "Enter". This will return to the prompt.

Make the policy file executable, and then run it, by typing the following two commands:

chmod +x ./hello_world.cf 

Followed by:

./hello_world.cf

See the output below:

# chmod +x ./hello_world.cf
# ./hello_world.cf
2013-08-20T14:39:34-0500   notice: R: Hello World!
Integrating the Example into your Main Policy

Make the example policy part of your main policy by doing the following on your policy server:

  1. Ensure the example is located in /var/cfengine/masterfiles.

  2. If the example contains a body common control section, delete it. That section will look something like this:

          body common control
          {
            bundlesequence  => { "hello_world" };
          }
    

You cannot have duplicate control bodies (i.e. two agent control bodies, one in the main file and one in the example) as CFEngine won't know which it should use and they may conflict.

To resolve this, copy the contents of the control body section from the example into the identically named control body section in the main policy file /var/cfengine/masterfiles/promises.cfand then remove the control body from the example.

  1. Insert the example's bundle name in the bundlesequence section of the main policy file /var/cfengine/masterfiles/promises.cf:

          bundlesequence => {
              ...
              "hello_world",
              ...
          };
    
  2. Insert the policy file name in the inputs section of the main policy file /var/cfengine/masterfiles/promises.cf:

    inputs => {
        ...
        "hello_world.cf",
        ...
    };
    
  3. You must also remove any inputs section from the example that includes the external library:

    inputs => {
        "libraries/cfengine_stdlib.cf"
    };
    

    This is necessary, since cfengine_stdlib.cf is already included in the inputs section of the master policy.

  4. The example policy will now be executed every five minutes along with the rest of your main policy.

Notes: You may have to fill the example with data before it will work. For example, the LDAP query in active_directory.cf needs a domain name. In the variable declaration, replace "cftesting" with your domain name:

  vars:
    # NOTE: Edit this to your domain, e.g. "corp"
    "domain_name" string => "cftesting";

Example Snippets


General Examples

Basic Example

To get started with CFEngine, you can imagine the following template for entering examples. This part of the code is common to all the examples.

body common control
{
      bundlesequence => { "main" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent main
{
      # example
}
The general pattern

The general pattern of the syntax is like this (colors in html version: red, CFEngine word; blue, user-defined word):

bundle component name(parameters)
{
what_type:
 where_when::

  # Traditional comment


  "promiser" -> { "promisee1", "promisee2" },
        comment => "The intention ...",
         handle => "unique_id_label",
    attribute_1 => body_or_value1,
    attribute_2 => body_or_value2;
}
Hello world
body common control
{
      bundlesequence => { "hello" };
}

bundle agent hello
{
  reports:
    linux::
      "Hello world!";
}
Array example
body common control
{
      bundlesequence => { "array" };
}

bundle common g
{
  vars:
      "array[1]" string => "one";
      "array[2]" string => "two";
}

bundle agent array
{
  vars:
      "localarray[1]" string => "one";
      "localarray[2]" string => "two";
  reports:
    linux::
      "Global $(g.array[1]) and $(localarray[2])";
}

Common Promise Patterns

This section includes includes common promise patterns. Refer to them as you write policy for your system.


Change detection

This policy will look for changes recursively in a directory.

body common control

{
      bundlesequence  => { "example"  };
}


bundle agent example

{
  files:

      "/home/mark/tmp/web"  # Directory to monitor for changes.

      changes      => detect_all_change,
      depth_search => recurse("inf");
}


body changes detect_all_change

{
      report_changes => "all";  
      update_hashes  => "true";
}


body depth_search recurse(d)

{
      depth        => "$(d)";
}

This policy can be found in /var/cfengine/share/doc/examples/change_detect.cf and downloaded directly from github.

Here is an example run.

First, let's create some files for CFEngine to monitor:

# mkdir /etc/example
# date > /etc/example/example.conf

CFEngine detects new files and adds them to the file integrity database:

# cf-agent -f unit_change_detect.cf
2013-06-06T20:53:26-0700    error: /example/files/'/etc/example':
File '/etc/example/example.conf' was not in 'md5' database - new file found
# cf-agent -f unit_change_detect.cf -K

If there are no changes, CFEngine runs silently:

# cf-agent -f unit_change_detect.cf
#

Now let's update the mtime, and then the mtime and content. CFEngine will notice the changes and record the new profile:

# touch /etc/example/example.conf # update mtime
# cf-agent -f unit_change_detect.cf -K
2013-06-06T20:53:52-0700    error: Last modified time for
'/etc/example/example.conf' changed 'Thu Jun  6 20:53:18 2013'
-> 'Thu Jun  6 20:53:49 2013'
# date >> /etc/example/example.conf # update mtime and content
# cf-agent -f unit_change_detect.cf -K
2013-06-06T20:54:01-0700    error: Hash 'md5' for '/etc/example/example.conf' changed!
2013-06-06T20:54:01-0700    error: /example/files/'/etc/example': Updating hash for
'/etc/example/example.conf' to 'MD5=8576cb25c9f78bc9ab6afd2c32203ca1'
2013-06-06T20:54:01-0700    error: Last modified time for '/etc/example/example.conf'
changed 'Thu Jun  6 20:53:49 2013' -> 'Thu Jun  6 20:53:59 2013'
#

Copy single files

This is a standalone policy example that will copy single files, locally (local_cp) and from a remote site (secure_cp). The CFEngine Standard Library (cfengine_stdlib.cf) should be included in the /var/cfengine/inputs/libraries/ directory and inputs list as below.

body common control
{
      bundlesequence  => { "mycopy" };
      inputs => { "libraries/cfengine_stdlib.cf" };
}

bundle agent mycopy
{
  files:
      "/tmp/test_plain"

Path and name of the file we wish to copy to

      comment => "/tmp/test_plain promises to be an up-to-date copy of /bin/echo to demonstrate copying a local file",
      copy_from => local_cp("$(sys.workdir)/bin/file");

Copy locally from path/filename

      "/tmp/test_remote_plain"
      comment => "/tmp/test_plain_remote promises to be a copy of cfengine://serverhost.example.org/repo/config-files/motd",
      copy_from => secure_cp("/repo/config-files/motd", "serverhost.example.org");
}

Copy remotely from path/filename and specified host ```

This policy can be found in /var/cfengine/share/doc/examples/copy_copbl.cf and downloaded directly from github.


Aborting execution

Sometimes it is useful to abort a bundle execution if certain conditions are not met, for example when validating input to a bundle. The following policy uses a list of regular expressions for classes, or class expressions that cf-agent will watch out for. If any of these classes becomes defined, it will cause the current bundle to be aborted.

body common control

{
      bundlesequence => { "example" };
}

body agent control

{
      abortbundleclasses => { "invalid" };
}


bundle agent example

{
  vars:

      #"userlist" slist => { "mark", "john" };           # contains all valid entries
      "userlist" slist => { "mark", "john", "thomas" };  # contains one invalid entry

  classes:

      "invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(userlist)"); # The class 'invalid' is set if the user name does not
      # contain exactly four un-capitalized letters (bundle
      # execution will be aborted if set)

  reports:

    !invalid::

      "User name $(userlist) is valid at 4 letters";
}

This policy can be found in /var/cfengine/share/doc/examples/abort.cf and downloaded directly from github.

This is how the policy runs when the userlist is valid:

# cf-agent -f unit_abort.cf
R: User name mark is valid at 4 letters
R: User name john is valid at 4 letters
#

This is how the policy runs when the userlist contains an invalid entry:

# cf-agent -f unit_abort.cf
Bundle example aborted on defined class "invalid"
#

To run this example file as part of your main policy you need to make an additional change:

There cannot be two body agent control in the main policy. Delete the body agent control section from /var/cfengine/masterfiles/unit_abort.cf. Copy and paste abortbundleclasses => { "invalid" }; into /var/cfengine/masterfiles/controls/cf_agent.cf. If you add it to the end of the file it should look something like this:

...
    #  dryrun => "true";

    abortbundleclasses => { "invalid" };
}

Check filesystem space

Check how much space (in KB) is available on a directory's current partition.

body common control
{
      bundlesequence => { "example" };
}

bundle agent example
{
  classes:
      "has_space" expression => isgreaterthan($(free), 0);

  vars:
      "free" int => diskfree("/tmp");

  reports:
    has_space::
      "The filesystem has free space";
    !has_space::
      "The filesystem has NO free space";
}
R: The filesystem has free space

This policy can be found in /var/cfengine/share/doc/examples/diskfree.cf and downloaded directly from github.

Example output:

# cf-agent -f unit_diskfree.cf
R: Freedisk 48694692
# df -k /tmp
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sda1      149911836 93602068  48694692  66% /
#

Distribute ssh keys

This example shows a simple ssh key distribution implementation.

The policy was designed to work with the services_autorun feature in the Masterfiles Policy Framework. The services_autorun feature can be enabled from the augments_file. If you do not have a def.json in the root of your masterfiles directory simply create it with the following content.

{
  "classes": {
               "services_autorun": [ "any" ]
             }
}

In the following example we will manage the authorized_keys file for bob, frank, and kelly.

For each listed user the ssh_key_distribution bundle is activated if the user exists on the system. Once activated the ssh_key_distribution bundle ensures that proper permissions are set on the users .ssh directory (home is assumed to be in /home/username) and ensures that the users .ssh/authorized_keys is a copy of the users authorized_keys file as found on the server as defined in the ssh_key_info bundle.

Let's assume we collected all users' public keys into a single directory on the server and that users exist on the clients (and have corresponding home directory).

Note: special variable $(sys.policy_hub) contains the hostname of the policy server.

To deploy this policy simply place it in the services/autorun directory of your masterfiles.

bundle common ssh_key_info
{
  meta:
    "description"
      string => "This bundle defines common ssh key information, like which
                 directory and server keys should be sourced from.";

  vars:
    "key_server" string => "$(sys.policy_hub)";

    # We set the path to the repo in a common bundle so that we can reference
    # the same path when defining access rules and when copying files.
    # This directory is expected to contain one file for each users authorized
    # keys, named for the username. For example: /srv/ssh_authorized_keys/kelly
    "repo_path" string => "/srv/ssh_authorized_keys";
}

bundle agent autorun_ssh_key_distribution
{
  meta:
    # Here we simply tag the bundle for use with the `services_autorun`
    # feature.
    "tags" slist => { "autorun" };

  vars:
    "users" slist => { "bob", "frank", "kelly" };

  methods:
    "Distribute SSH Keys"
      usebundle => ssh_key_distribution( $(users) ),
      if => userexists( $(users) ),
      comment => "It's important that we make sure each of these users
                  ssh_authorized_keys file has the correct content and
                  permissions so that they can successfully log in, if
                  the user exists on the executing agents host.";
}

bundle agent ssh_key_distribution(users)
{
  meta:
    "description"
      string => "Ensure that specified users are able to log in using their ssh
                 keys";
  vars:
    # We get the users UID so that we can set permission appropriately
    "uid[$(users)]" int =>  getuid( $(users) );

  files:
    "/home/$(users)/.ssh/."
      create => "true",
      perms => mo( 700, "$(uid[$(users)])"),
      comment => "It is important to set the proper restrictive permissions and
                  ownership so that the ssh authorized_keys feature works
                  correctly.";

    "/home/$(users)/.ssh/authorized_keys"
      perms => mo( 600, "$(uid[$(users)])" ),
      copy_from => remote_dcp( "$(ssh_key_info.repo_path)/$(users)",
                               $(ssh_key_info.key_server) ),
      comment => "We centrally manage and users authorized keys. We source each
                  users complete authorized_keys file from the central server.";
}


bundle server ssh_key_access_rules
{
  meta:
    "description"
      string => "This bundle handles sharing the directory where ssh keys
                 are distributed from.";

  access:
    # Only hosts with class `policy_server` should share the path to ssh
    # authorized_keys
    policy_server::
      "$(ssh_key_info.repo_path)"
        admit => { @(def.acl) },
        comment => "We share the ssh authorized keys with all authorized
                    hosts.";
}

This policy can be found in /var/cfengine/share/doc/examples/simple_ssh_key_distribution.cf and downloaded directly from github.

Example Run:

First make sure the users exist on your system.

root@host001:~# useradd bob
root@host001:~# useradd frank
root@host001:~# useradd kelly

Then update the policy and run it:

root@host001:~# cf-agent -Kf update.cf; cf-agent -KI
    info: Installing cfe_internal_non_existing_package...
    info: Created directory '/home/bob/.ssh/.'
    info: Owner of '/home/bob/.ssh' was 0, setting to 1002
    info: Object '/home/bob/.ssh' had permission 0755, changed it to 0700
    info: Copying from '192.168.33.2:/srv/ssh_authorized_keys/bob'
    info: Owner of '/home/bob/.ssh/authorized_keys' was 0, setting to 1002
    info: Created directory '/home/frank/.ssh/.'
    info: Owner of '/home/frank/.ssh' was 0, setting to 1003
    info: Object '/home/frank/.ssh' had permission 0755, changed it to 0700
    info: Copying from '192.168.33.2:/srv/ssh_authorized_keys/frank'
    info: Owner of '/home/frank/.ssh/authorized_keys' was 0, setting to 1003
    info: Created directory '/home/kelly/.ssh/.'
    info: Owner of '/home/kelly/.ssh' was 0, setting to 1004
    info: Object '/home/kelly/.ssh' had permission 0755, changed it to 0700
    info: Copying from '192.168.33.2:/srv/ssh_authorized_keys/kelly'
    info: Owner of '/home/kelly/.ssh/authorized_keys' was 0, setting to 1004

Ensure a process is not running

This is a standalone policy that will kill the sleep process. You can adapt it to make sure that any undesired process is not running.

body common control
{
bundlesequence => { "process_kill" };
}

bundle agent process_kill
{
processes:

  "sleep"

    signals => { "term", "kill" }; #Signals are presented as an ordered list to the process.
                                   #On Windows, only the kill signal is supported, which terminates the process.

}

This policy can be found in /var/cfengine/share/doc/examples/unit_process_kill.cf.

Example run:

# /bin/sleep 1000 &
[1] 5370
# cf-agent -f unit_process_kill.cf
[1]+  Terminated              /bin/sleep 1000
#

Now let's do it again with inform mode turned on, and CFEngine will show the process table entry that matched the pattern we specified ("sleep"):

# /bin/sleep 1000 &
[1] 5377
# cf-agent -f unit_process_kill.cf -IK
2013-06-08T16:30:06-0700     info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T16:30:06-0700     info: Running full policy integrity checks
2013-06-08T16:30:06-0700     info: /process_kill/processes/'sleep': Signalled 'term' (15) to process 5377 (root      5377  3854  5377  0.0  0.0  11352   0   612    1 16:30 00:00:00 /bin/sleep 1000)
[1]+  Terminated              /bin/sleep 1000
#

If we add the -v switch to turn on verbose mode, we see the /bin/ps command CFEngine used to dump the process table:

# cf-agent -f unit_process_kill.cf -Kv
...
2013-06-08T16:38:20-0700  verbose: Observe process table with /bin/ps -eo user,pid,ppid,pgid,pcpu,pmem,vsz,ni,rss,nlwp,stime,time,args
2013-06-08T16:38:20-0700  verbose: Matched 'root      5474  3854  5474  0.0  0.0  11352   0   612    1 16:38 00:00:00 /bin/sleep 1000'
...

Updating from a central policy server

This is a conceptual example without any test policy associated with it.

The default policy shipped with CFEngine contains a centralized updating of policy that covers more subtleties than this example, and handles fault tolerance. Here is the main idea behind it. For simplicity, we assume that all hosts are on network 10.20.30.* and that the central policy server is 10.20.30.123.

bundle agent update
{
vars:

 "master_location" string => "/var/cfengine/masterfiles";

 "policy_server"   string => "10.20.30.123";
                   comment => "IP address to locate your policy host.";

files:

  "$(sys.workdir)/inputs"

    perms => system("600"),
    copy_from => remote_cp("$(master_location)",$(policy_server)),
    depth_search => recurse("inf"); # This ensures recursive copying of all subdirectories

  "$(sys.workdir)/bin"

    perms => system("700"),
    copy_from => remote_cp("/usr/local/sbin","localhost"),
    depth_search => recurse("inf"); # This ensures recursive copying of all subdirectories
}

In addition the server needs to grant access to the clients, this is done in the body server control:

body server control

{
allowconnects         => { "127.0.0.1" , "10.20.30.0/24" };
allowallconnects      => { "127.0.0.1" , "10.20.30.0/24" };
trustkeysfrom         => { "127.0.0.1" , "10.20.30.0/24" };
}

Since we assume that all hosts are on network 10.20.30.* they will be granted access. In the default policy this is set to $(sys.policy_hub)/16, i.e. all hosts in the same class B network as the hub will gain access. You will need to modify the access control list in body server control if you have clients outside of the policy server's class B network.

Granting access to files and folders needs to be done in bundle server access_rules():

bundle server access_rules()
{
access:

 10_20_30_123::

  "/var/cfengine/masterfiles"

    admit   => { "127.0.0.1", "10.20.30.0/24" };
}

Customize Message of the Day

The Message of the Day is displayed when you log in or connect to a server. It typically shows information about the operating system, license information, last login, etc.

It is often useful to customize the Message of the Day to inform your users about some specifics of the system they are connecting to. In this example we render a /etc/motd using a mustache template and add useful information as:

  • The role of the server ( staging / production )
  • The hostname of the server
  • The CFEngine version we are running on the host
  • The CFEngine role of the server ( client / hub )
  • The administrative contacts details conditionally to the environment
  • The primary Ipv4 IP address
  • The number of packages updates available for this host

The bundle is defined like this:

body file control {
  # To make this policy "standalone" we load parts of the standard library that
  # we use
  inputs => { "$(sys.libdir)/files.cf", "$(sys.libdir)/packages.cf" };
}

bundle agent motd {

  vars:
    "motd_path" string => "/etc/motd";
    "template_path" string => "$(this.promise_filename).mustache";

    ## This is to extract the cf-engine role ( hub or client )
    "cf_role"
      string => ifelse( "policy_server", "Policy Server", "Policy Client");

    ## This is to extract the available package updates status
    "updates_available"
      data => packageupdatesmatching(".*", ".*", ".*", ".*");
    "amount_updates" int => length("updates_available");

    ## This is to define a production / stage example role for the server
    "environment_list" slist => { "/etc/prod", "/etc/staging" };
    "environment"
      string => ifelse( filesexist( @(environment_list) ), "unknown",
                        fileexists("/etc/prod"), "production",
                        fileexists("/etc/staging"), "staging",
                        "unknown" );

    ## Based on the environment we define different contatcs.
    "admin_contact"
      slist => { "admin@acme.com", "oncall@acme.com" },
      if => strcmp( $(environment), "production" );

    "admin_contact"
      slist => { "dev@acme.com" },
      if => strcmp( $(environment), "staging" );

    "admin_contact"
      slist => { "root@localhost" },
      if => strcmp( $(environment), "unknown" );

  ## We define a class to conditionally render the role status in the final
  ## motd.
  classes:
    "env_$(environment)" expression => "any";

  ## In order to get package inventory, we need to have a legacy package
  ## promise.
  packages:
    "vim" -> { "https://dev.cfengine.com/issues/7864" }
      package_policy => "add";

  ## Here is where we render the mustache template
  files:
    "$(motd_path)"
      create => "true",
      perms => mog( 444, "root", "root"),
      template_method => "mustache",
      edit_template => "$(template_path)";

  ## These are good to understand what is going on.
  reports:
    DEBUG|DEBUG_motd::
      "DEBUG $(this.bundle): $(sys.cf_version) is the detected version";
      "DEBUG $(this.bundle): $(sys.fqhost) is the detected hostname";
      "DEBUG $(this.bundle): $(sys.ipv4) is the ipv4 address for $(sys.fqhost)";
      "DEBUG $(this.bundle): $(motd.cf_role) is the detected role";
      "DEBUG $(this.bundle): $(amount_updates) packages can be updated";
      "DEBUG $(this.bundle): This host is managed by $(admin_contact)";
}

This policy can be found in /var/cfengine/share/doc/examples/mustache_template_motd.cf and downloaded directly from github.

Here is the mustache:

{{#classes.env_unknown}} WARNING Environment Unknown{{/classes.env_unknown}}
      ¤¤¤
      ¤¤¤
      ¤¤¤     Welcome into {{{vars.sys.fqhost}}}

    ¤ ¤¤¤ ¤      This system is controlled by
    ¤ ¤¤¤ ¤      CFEngine {{{vars.sys.cf_version}}}
    ¤ ¤¤¤ ¤     And is a {{{vars.motd.cf_role}}}
    ¤     ¤
      ¤¤¤
      ¤ ¤  
      ¤ ¤  Host IP {{{vars.sys.ipv4}}}
      ¤ ¤  {{{vars.motd.amount_updates}}} package updates available.
                 {{#vars.motd.admin_contact}}Support Contact:
                   - {{{.}}}
{{/vars.motd.admin_contact}}

This policy can be found in /var/cfengine/share/doc/examples/mustache_template_motd.cf.mustache and downloaded directly from github.

Example run:

root@debian8:~/core/examples# cf-agent --no-lock --bundlesequence motd --define DEBUG_motd --file ./mustache_template_motd.cf
    info: Using command line specified bundlesequence
R: 3.7.2 is the detected version
R: debian8 is the detected hostname
R: 10.100.251.53 is the ipv4 address for debian8
R: Policy Client is the detected role for debian8
R: 20 packages can be updated
R: This host is managed by root@localhost
root@debian8:~/core/examples# cat /etc/motd
 WARNING Environment Unknown
      ¤¤¤
      ¤¤¤
      ¤¤¤     Welcome into debian8

    ¤ ¤¤¤ ¤      This system is controlled by
    ¤ ¤¤¤ ¤      CFEngine 3.8.0
    ¤ ¤¤¤ ¤     And is a Policy Client
    ¤     ¤
      ¤¤¤
      ¤ ¤ 
      ¤ ¤  Host IP 10.100.251.53
      ¤ ¤  20 package updates available.
                 Support Contact:
                   - root@localhost

Restart a Process

This is a standalone policy that will restart three CFEngine processes if they are not running.

body common control
{
bundlesequence => { "process_restart" };
}


bundle agent process_restart
{
vars:

  "component" slist => {                  # List of processes to monitor
                         "cf-monitord",
                         "cf-serverd",
                         "cf-execd"
                       };
processes:

  "$(component)"
      restart_class => canonify("start_$(component)"); # Set the class "start_<component>" if it is not running

commands:

   "/var/cfengine/bin/$(component)"
       ifvarclass => canonify("start_$(component)"); # Evaluate the class "start_<component>", CFEngine will run
                                                   # the command if "start_<component> is set.

}

Notes: The canonify function translates illegal characters to underscores, e.g. start_cf-monitord becomes start_cf_monitord. Only alphanumerics and underscores are allowed in CFEngine identifiers (names of variables, classes, bundles, etc.)

This policy can be found in /var/cfengine/share/doc/examples/unit_process_restart.cf.

Example run:

# ps -ef |grep cf-
root      4305     1  0 15:14 ?        00:00:02 /var/cfengine/bin/cf-execd
root      4311     1  0 15:14 ?        00:00:05 /var/cfengine/bin/cf-serverd
root      4397     1  0 15:15 ?        00:00:06 /var/cfengine/bin/cf-monitord
# kill 4311
# ps -ef |grep cf-
root      4305     1  0 15:14 ?        00:00:02 /var/cfengine/bin/cf-execd
root      4397     1  0 15:15 ?        00:00:06 /var/cfengine/bin/cf-monitord
# cf-agent -f unit_process_restart.cf
# ps -ef |grep cf-
root      4305     1  0 15:14 ?        00:00:02 /var/cfengine/bin/cf-execd
root      4397     1  0 15:15 ?        00:00:06 /var/cfengine/bin/cf-monitord
root      8008     1  0 18:18 ?        00:00:00 /var/cfengine/bin/cf-serverd
#

And again, in Inform mode:

# kill 8008
# cf-agent -f unit_process_restart.cf -I
2013-06-08T18:19:51-0700     info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T18:19:51-0700     info: Running full policy integrity checks
2013-06-08T18:19:51-0700     info: /process_restart/processes/'$(component)': Making a one-time restart promise for 'cf-serverd'
2013-06-08T18:19:51-0700     info: Executing 'no timeout' ... '/var/cfengine/bin/cf-serverd'
2013-06-08T18:19:52-0700     info: Completed execution of '/var/cfengine/bin/cf-serverd'
#

Set up sudo

Setting up sudo is straightforward, we recommend managing it by copying trusted files from a repository. The following bundle will copy a master sudoers file to /etc/sudoers (/tmp/sudoers in this example - change it to /etc/sudoers to use in production).

body common control
{
bundlesequence => { "sudoers" };
inputs => { "libraries/cfengine_stdlib.cf" };
}


bundle agent sudoers
{

# Define the master location of the sudoers file
vars:

  "master_location" string => "/var/cfengine/masterfiles";


# Copy the master sudoers file to /etc/sudoers
files:

  "/tmp/sudoers"  # change to /etc/sudoers to use in production

     comment => "Make sure the sudo configuration is secure and up to date",
       perms => mog("440","root","root"),
   copy_from => secure_cp("$(master_location)/sudoers","$(sys.policy_hub)");

}

We recommend editing the master sudoers file using visudo or a similar tool. It is possible to use CFEngine's file editing capabilities to edit sudoers directly, but this does not guarantee syntax correctness and you might end up locked out.

Example run:

# cf-agent -f temp.cf -KI
2013-06-08T19:13:21-0700     info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T19:13:22-0700     info: Running full policy integrity checks
2013-06-08T19:13:23-0700     info: Copying from '192.168.183.208:/var/cfengine/masterfiles/sudoers'
2013-06-08T19:13:23-0700     info: /sudoers/files/'/tmp/sudoers': Object '/tmp/sudoers' had permission 0600, changed it to 0440
#

For reference we include an example of a simple sudoers file:

# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#

Defaults        env_reset

# User privilege specification
root    ALL=(ALL) ALL

# Allow members of group sudo to execute any command after they have
# provided their password
%sudo ALL=(ALL) ALL

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
john  ALL=(ALL)       ALL

Set up name resolution with DNS

There are many ways to configure name resolution. A simple and straightforward approach is to implement this as a simple editing promise for the /etc/resolv.conf file.

body common control
{
bundlesequence => { "edit_name_resolution" };
}

bundle agent edit_name_resolution
{

files:

  "/tmp/resolv.conf"   # This is for testing, change to "$(sys.resolv)" to put in production

     comment       => "Add lines to the resolver configuration",
     create        => "true",     # Make sure the file exists, create it if not
     edit_line     => resolver,   # Call the resolver bundle defined below to do the editing
     edit_defaults => empty;      # Baseline memory model of file to empty before processing
                                  # bundle edit_line resolver
}

bundle edit_line resolver
{

insert_lines:

 any::   # Class/context where you use the below nameservers. Change to appropriate class
     # for your system (if not any::, for example server_group::, ubuntu::, etc.)

  # insert the search domain or name servers we want
  "search mydomain.tld" location => start;  # Replace mydomain.tld with your domain name
                              # The search line will always be at the start of the file
  "nameserver 128.39.89.8";
  "nameserver 128.39.74.66";
}

body edit_defaults empty
{
empty_file_before_editing => "true";
}

body location start
{
before_after => "before";
}

Example run:

# cf-agent -f unit_edit_name_resolution.cf  # set up resolv.conf
# cat /tmp/resolv.conf # show resolv.conf
search mydomain.tld
nameserver 128.39.89.8
nameserver 128.39.74.66
# echo 'nameserver 0.0.0.0' >> /tmp/resolv.conf  # mess up resolv.conf
# cf-agent -f ./unit_edit_name_resolution.cf -KI  # heal resolv.conf
2013-06-08T18:38:12-0700     info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T18:38:12-0700     info: Running full policy integrity checks
2013-06-08T18:38:12-0700     info: /edit_name_resolution/files/'/tmp/resolv.conf': Edit file '/tmp/resolv.conf'
# cat /tmp/resolv.conf # show healed resolv.conf
search mydomain.tld
nameserver 128.39.89.8
nameserver 128.39.74.66
#

Find the MAC address

Finding the ethernet address can vary between operating systems.

We will use CFEngine's built in function execresult to execute commands adapted for different operating systems, assign the output to variables, and filter for the MAC address. We then report on the result.

body common control
{
bundlesequence => { "example" };
}


bundle agent example
{
vars:

  linux::
    "interface" string => execresult("/sbin/ifconfig eth0", "noshell");

  solaris::
    "interface" string => execresult("/usr/sbin/ifconfig bge0", "noshell");

  freebsd::
    "interface" string => execresult("/sbin/ifconfig le0", "noshell");

  darwin::
    "interface" string => execresult("/sbin/ifconfig en0", "noshell");

# Use the CFEngine function 'regextract' to match the MAC address,
# assign it to an array called mac and set a class to indicate positive match
classes:

  linux::

  "ok" expression => regextract(
                                ".*HWaddr ([^\s]+).*(\n.*)*",  # pattern to match
                                "$(interface)",  # string to scan for pattern
                                "mac"  # put the text that matches the pattern into this array
                                );

  solaris|freebsd::

   "ok" expression => regextract(
                                ".*ether ([^\s]+).*(\n.*)*",
                                "$(interface)",
                                "mac"
                                );


  darwin::

   "ok" expression => regextract(
                                "(?s).*ether ([^\s]+).*(\n.*)*",
                                "$(interface)",
                                "mac"
                                );

# Report on the result
reports:

  ok::

    "MAC address is $(mac[1])";  # return first element in array "mac"

}

This policy can be found in /var/cfengine/masterfiles/example_find_mac_addr.cf

Example run:

# cf-agent -f example_find_mac_addr.cf
2013-06-08T16:59:19-0700   notice: R: MAC address is a4:ba:db:d7:59:32
#

While the above illustrates the flexiblity of CFEngine in running external commands and parsing their output, as of CFEngine 3.3.0, Nova 2.2.0 (2011), you can get the MAC address natively:

body common control
{
bundlesequence => { "example" };
}


bundle agent example
{
vars:

  linux::   "interface" string => "eth0";

  solaris:: "interface" string => "bge0";

  freebsd:: "interface" string => "le0";

  darwin::  "interface" string => "en0";


reports:

    "MAC address of $(interface) is: $(sys.hardware_mac[$(interface)])";
}

Install packages

Install desired packages.

body common control
{
bundlesequence => { "install_packages" };
inputs => { "libraries/cfengine_stdlib.cf" };
}

bundle agent install_packages
{

vars:
    "desired_packages"
        slist => {        # list of packages we want
                  "ntp",
                  "lynx"
                 };

packages:

    "$(desired_packages)"  # operate on listed packages

         package_policy => "add",     # What to do with packages: install them.
         package_method => generic;   # Infer package manager (e.g. apt, yum) from the OS.
}

Caution: package management is a dirty business. If things don't go smoothly using the generic method, you may have to use a method specific to your package manager and get to your elbows in the details. But try generic first. You may get lucky.

Mind package names can differ OS to OS. For example, Apache httpd is "httpd" on Red Hat, and "apache2" on Debian.

Version comparison can be tricky when involving multipart version identifiers with numbers and letters.

CFEngine downloads the necessary packages from the default repositories if they are not present on the local machine, then installs them if they are not already installed.

Example run:

# dpkg -r lynx ntp # remove packages so CFEngine has something to repair
(Reading database ... 234887 files and directories currently installed.)
Removing lynx ...
Removing ntp ...
 * Stopping NTP server ntpd                                                                                                                     [ OK ]
Processing triggers for ureadahead ...
Processing triggers for man-db ...
# cf-agent -f install_packages.cf # install packages
# dpkg -l lynx ntp # show installed packages
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                            Version              Architecture         Description
+++-===============================-====================-====================-====================================================================
ii  lynx                            2.8.8dev.12-2ubuntu0 all                  Text-mode WWW Browser (transitional package)
ii  ntp                             1:4.2.6.p3+dfsg-1ubu amd64                Network Time Protocol daemon and utility programs
#

There are examples in /var/cfengine/share/doc/examples/ of installing packages using specific package managers: - Red Hat (unit_package_yum.cf) - Debian (unit_package_apt.cf) - MSI for Windows (unit_package_msi_file.cf) - Solaris (unit_package_solaris.cf) - SuSE Linux (unit_package_zypper.cf)


Ensure a service is enabled and running

This example shows how to ensure services are started or stopped appropriately.

body file control
{
  inputs => { "$(sys.libdir)/services.cf", "$(sys.libdir)/commands.cf" };
}

bundle agent main
{
  vars:

    linux::
      "enable[ssh]"
        string => ifelse( "debian|ubuntu", "ssh", "sshd"),
        comment => "The name of the ssh service varies on different platforms.
                    Here we set the name of the service based on existing
                    classes and defaulting to `sshd`";

      "disable[apache]"
        string => ifelse( "debian|ubuntu", "apache2", "httpd" ),
        comment => "The name of the apache web service varies on different
                    platforms. Here we set the name of the service based on
                    existing classes and defaulting to `httpd`";

      "enable[cron]"
        string  => ifelse( "debian|ubuntu", "cron", "crond" ),
        comment => "The name of the cron service varies on different
                    platforms. Here we set the name of the service based on
                    existing classes and defaulting to `crond`";

      "disable[cups]"
        string => "cups",
        comment => "Printing services are not needed on most hosts.";

      "enabled" slist => getvalues( enable );
      "disabled" slist => getvalues( disable );

  services:

    linux::

      "$(enabled)" -> { "SysOps" }
        service_policy => "start",
        comment => "These services should be running because x, y or z.";

      "$(disabled)" -> { "SysOps" }
        service_policy => "stop",
        comment => "These services should not be running because x, y or z.";

    systemd::

      "sysstat"
        service_policy => "stop",
        comment => "Standard service handling for sysstat only works with
                    systemd. Other inits need cron entries managed.";
}

This policy can be found in /var/cfengine/share/doc/examples/services.cf and downloaded directly from github.

Note: Not all services behave in the standard way. Some services may require custom handling. For example it is not uncommon for some services to not provide correct return codes for status checks.

See Also:

Example usage on systemd

We can see that before the policy run sysstat is inactive, apache2 is active, cups is active, ssh is active and cron is inactive.

root@ubuntu:# systemctl is-active sysstat apache2 cups ssh cron
inactive
active
active
active
inactive

Now we run the policy to converge the system to the desired state.

root@ubuntu:# cf-agent --no-lock --inform --file ./services.cf
    info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q stop apache2'
    info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q stop apache2'
    info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q stop cups'
    info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q stop cups'
    info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q start cron'
    info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q start cron'

After the policy run we can see that systat, apache2, and cups are inactive. ssh and cron are active as specified in the policy.

root@ubuntu:/home/nickanderson/CFEngine/core/examples# systemctl is-active sysstat apache2 cups ssh cron
inactive
inactive
inactive
active
active
Example usage with System V

We can see that before the policy run sysstat is not reporting status correctly , httpd is running, cups is running, sshd is running and crond is not running.

[root@localhost examples]# service sysstat status; echo $?
3
[root@localhost examples]# service httpd status; echo $?
httpd (pid  3740) is running...
0
[root@localhost examples]# service cups status; echo $?
cupsd (pid  3762) is running...
0
[root@localhost examples]# service sshd status; echo $?
openssh-daemon (pid  3794) is running...
0
[root@localhost examples]# service crond status; echo $?
crond is stopped
3

Now we run the policy to converge the system to the desired state.

[root@localhost examples]# cf-agent -KIf ./services.cf
    info: Executing 'no timeout' ... '/etc/init.d/crond start'
    info: Completed execution of '/etc/init.d/crond start'
    info: Executing 'no timeout' ... '/etc/init.d/httpd stop'
    info: Completed execution of '/etc/init.d/httpd stop'
    info: Executing 'no timeout' ... '/etc/init.d/cups stop'
    info: Completed execution of '/etc/init.d/cups stop'

After the policy run we can see that systat is still not reporting status correctly (some services do not respond to standard checks), apache2, and cups are inactive. ssh and cron are active as specified in the policy.

[root@localhost examples]# service sysstat status; echo $?
3
[root@localhost examples]# service httpd status; echo $?
httpd is stopped
3
[root@localhost examples]# service cups status; echo $?
cupsd is stopped
3
[root@localhost examples]# service sshd status; echo $?
openssh-daemon (pid  3794) is running...
0
[root@localhost examples]# service crond status; echo $?
crond (pid  3929) is running...
0

Create files and directories

The following is a standalone policy that will create the file /home/mark/tmp/test_plain and the directory /home/mark/tmp/test_dir/ and set permissions on both.

body common control

{
      bundlesequence  => { "example"  };
}


bundle agent example

{
  files:

      "/home/mark/tmp/test_plain" 

The promiser specifies the path and name of the file.

      perms => system,
      create => "true";

The perms attribute sets the file permissions as defined in the system body below. The create attribute makes sure that the files exists. If it doesn't, CFEngine will create it.

      "/home/mark/tmp/test_dir/." 

      perms => system,
      create => "true";

The trailing /. in the filename tells CFEngine that the promiser is a directory.

}


body perms system

{
      mode  => "0640";
}

This body sets permissions to "0640"

This policy can be found in /var/cfengine/share/doc/examples/create_filedir.cf and downloaded directly from github.

Example output:

# cf-agent -f unit_create_filedir.cf -I
2013-06-08T14:56:26-0700     info: /example/files/'/home/mark/tmp/test_plain': Created file '/home/mark/tmp/test_plain', mode 0640
2013-06-08T14:56:26-0700     info: /example/files/'/home/mark/tmp/test_dir/.': Created directory '/home/mark/tmp/test_dir/.'
2013-06-08T14:56:26-0700     info: /example/files/'/home/mark/tmp/test_dir/.': Object '/home/mark/tmp/test_dir' had permission 0755, changed it to 0750
#

Set up time management through NTP

The following sets up a local NTP server that synchronizes with pool.ntp.org and clients that synchronize with your local NTP server. See bottom of this example if you don't want to build a server, but use a "brute force" method (repeated ntpdate syncs).

This example demonstrates you can have a lot of low-level detailed control if you want it.

    bundle agent system_time_ntp
    {
     vars:

      linux::

       "cache_dir"   string => "$(sys.workdir)/cache";  # Cache directory for NTP config files

       "ntp_conf"    string => "/etc/ntp.conf";  # Target file for NTP configuration

       "ntp_server"  string => "172.16.12.161";  #
       "ntp_network" string => "172.16.12.0";    # IP address and netmask of your local NTP server
       "ntp_mask"    string => "255.255.255.0";  #

       "ntp_pkgs"    slist => { "ntp" };         # NTP packages to be installed to ensure service


    # Define a class for the NTP server
     classes:

      any::

       "ntp_hosts"         or => { classmatch(canonify("ipv4_$(ntp_server)")) };


    # Ensure that the NTP packages are installed
     packages:

      ubuntu::

       "$(ntp_pkgs)"

             comment => "setup NTP",
          package_policy => "add",
          package_method => generic;


    # Ensure existence of file and directory for NTP drift learning statistics
     files:

      linux::

       "/var/lib/ntp/ntp.drift"

          comment => "Enable ntp service",
           create => "true";

       "/var/log/ntpstats/."

          comment => "Create a statistic directory",
        perms => mog("644","ntp","ntp"),
           create => "true";

      ntp_hosts::


    # Build the cache configuration file for the NTP server
       "/var/cfengine/cache/ntp.conf"

            comment => "Build $(this.promiser) cache file for NTP server",
             create => "true",
          edit_defaults => empty,
          edit_line => restore_ntp_master("$(ntp_network)","$(ntp_mask)");

      centos.ntp_hosts::


    # Copy the cached configuration file to its target destination
       "$(ntp_conf)"

        comment => "Ensure $(this.promiser) in a perfect condition",
          copy_from => local_cp("$(cache_dir)/ntp.conf"),
        classes => if_repaired("refresh_ntpd_centos");

      ubuntu.ntp_hosts::

       "$(ntp_conf)"

        comment => "Ensure $(this.promiser) in a perfect condition",
          copy_from => local_cp("$(cache_dir)/ntp.conf"),
        classes => if_repaired("refresh_ntpd_ubuntu");

      !ntp_hosts::


    # Build the cache configuration file for the NTP client
       "$(cache_dir)/ntp.conf"

            comment => "Build $(this.promiser) cache file for NTP client",
             create => "true",
          edit_defaults => empty,
          edit_line => restore_ntp_client("$(ntp_server)");

      centos.!ntp_hosts::


    # Copy the cached configuration file to its target destination
       "$(ntp_conf)"

        comment => "Ensure $(this.promiser) in a perfect condition",
          copy_from => local_cp("$(cache_dir)/ntp.conf"),
            classes => if_repaired("refresh_ntpd_centos");

      ubuntu.!ntp_hosts::

       "$(ntp_conf)"

        comment => "Ensure $(this.promiser) in a perfect condition",
          copy_from => local_cp("$(cache_dir)/ntp.conf"),
        classes => if_repaired("refresh_ntpd_ubuntu");


    # Set classes (conditions) for to restart the NTP daemon if there have been any changes to configuration
     processes:

      centos::

       "ntpd.*"

          restart_class => "refresh_ntpd_centos";

      ubuntu::

       "ntpd.*"

          restart_class => "refresh_ntpd_ubuntu";


    # Restart the NTP daemon if the configuration has changed
     commands:

      refresh_ntpd_centos::

       "/etc/init.d/ntpd restart";

      refresh_ntpd_ubuntu::

       "/etc/init.d/ntp restart";

    }

    #######################################################

    bundle edit_line restore_ntp_master(network,mask)
    {
    vars:
     "list" string =>
    "######################################
    # ntp.conf-master

    driftfile /var/lib/ntp/ntp.drift
    statsdir /var/log/ntpstats/

    statistics loopstats peerstats clockstats
    filegen loopstats file loopstats type day enable
    filegen peerstats file peerstats type day enable
    filegen clockstats file clockstats type day enable

    # Use public servers from the pool.ntp.org project.
    # Please consider joining the pool (http://www.pool.ntp.org/join.html).
    # Consider changing the below servers to a location near you for better time
    # e.g. server 0.europe.pool.ntp.org, or server 0.no.pool.ntp.org etc.
    server 0.centos.pool.ntp.org
    server 1.centos.pool.ntp.org
    server 2.centos.pool.ntp.org

    # Permit time synchronization with our time source, but do not
    # permit the source to query or modify the service on this system.
    restrict -4 default kod nomodify notrap nopeer noquery
    restrict -6 default kod nomodify notrap nopeer noquery

    # Permit all access over the loopback interface.  This could
    # be tightened as well, but to do so would effect some of
    # the administrative functions.
    restrict 127.0.0.1
    restrict ::1

    # Hosts on local network are less restricted.
    restrict $(network) mask $(mask) nomodify notrap";

    insert_lines:
     "$(list)";
    }

    #######################################################

    bundle edit_line restore_ntp_client(serverip)
    {
    vars:
     "list" string =>
    "######################################
    # This file is protected by cfengine #
    ######################################
    # ntp.conf-client

    driftfile /var/lib/ntp/ntp.drift
    statsdir /var/log/ntpstats/

    statistics loopstats peerstats clockstats
    filegen loopstats file loopstats type day enable
    filegen peerstats file peerstats type day enable
    filegen clockstats file clockstats type day enable

    # Permit time synchronization with our time source, but do not
    # permit the source to query or modify the service on this system.
    restrict -4 default kod nomodify notrap nopeer noquery
    restrict -6 default kod nomodify notrap nopeer noquery

    # Permit all access over the loopback interface.  This could
    # be tightened as well, but to do so would effect some of
    # the administrative functions.
    restrict 127.0.0.1
    restrict ::1
    server $(serverip)
    restrict $(serverip) nomodify";

    insert_lines:
     "$(list)";
    }

This policy can be found in /var/cfengine/share/doc/examples/example_ntp.cf

If you don't want to build a server, you might do like this:

    bundle agent time_management
    {
     vars:

      any::

       "ntp_server" string => "no.pool.ntp.org";

     commands:

      any::

          "/usr/sbin/ntpdate $(ntp_server)"

         contain => silent;

    }

This is a hard reset of the time, it corrects it immediately. This may cause problems if there are large deviations in time and you are using time sensitive software on your system. An NTP daemon setup as shown above, on the other hand, slowly adapts the time to avoid causing disruption. In addition, the NTP daemon can be configured to learn your system's time drift and automatically adjust for it without having to be in touch with the server at all times.


Mount NFS filesystem

Mounting an NFS filesystem is straightforward using CFEngine's storage promises. The following bundle specifies the name of a remote file system server, the path of the remote file system and the mount point directory on the local machine:

body common control
{
bundlesequence => { "mounts" };
}


bundle agent mounts
{
storage:

  "/mnt" mount  => nfs("fileserver","/home");  # "/mnt" is the local moint point
                                               # "fileserver" is the remote fileserver
                                               # "/home" is the path to the remote file system
}


body mount nfs(server,source)
{
mount_type => "nfs";           # Protocol type of remote file system
mount_source => "$(source)";   # Path of remote file system
mount_server => "$(server)";   # Name or IP of remote file system server
mount_options => { "rw" };     # List of option strings to add to the file system table ("fstab")
edit_fstab => "true";          # True/false add or remove entries to the file system table ("fstab")
}

This policy can be found in /var/cfengine/share/doc/examples/example_mount_nfs.cf

Here is an example run. At start, the filesystem is not in /etc/fstab and is not mounted:

# grep mnt /etc/fstab # filesystem is not in /etc/fstab
# df |grep mnt # filesystem is not mounted

Now we run CFEngine to mount the filesystem and add it to /etc/fstab:

# cf-agent -f example_mount_nfs.cf
2013-06-08T17:48:42-0700    error: Attempting abort because mount went into a retry loop.
# grep mnt /etc/fstab
fileserver:/home     /mnt    nfs     rw
# df |grep mnt
fileserver:/home 149912064 94414848  47882240  67% /mnt
#

Note: CFEngine errors out after it mounts the filesystem and updates /etc/fstab. There is a ticket https://cfengine.com/dev/issues/2937 open on this issue.


CFEngine Administration Examples

Ordering promises

This counts to five by default. If we change ‘/bin/echo one’ to ‘/bin/echox one’, then the command will fail, causing us to skip five and go to six instead.

This shows how dependencies can be chained in spite of the order of promises in the bundle.

Normally the order of promises in a bundle is followed, within each promise type, and the types are ordered according to normal ordering.

body common control
{
      bundlesequence => { "order" };
}

bundle agent order
{
  vars:
      "list" slist => { "three", "four" };

  commands:
    ok_later::
      "/bin/echo five";

    otherthing::
      "/bin/echo six";

    any::

      "/bin/echo one"     classes => d("ok_later","otherthing");
      "/bin/echo two";
      "/bin/echo $(list)";

    preserved_class::
      "/bin/echo seven";
}

body classes d(if,else)
{
      promise_repaired => { "$(if)" };
      repair_failed => { "$(else)" };
      persist_time => "0";
}
Aborting execution
body common control
{
      bundlesequence  => { "testbundle"  };
      version => "1.2.3";
}

body agent control
{
      abortbundleclasses => { "invalid.Hr16" };
}

bundle agent testbundle
{
  vars:
      "userlist" slist => { "xyz", "mark", "jeang", "jonhenrik", "thomas", "eben" };

  methods:
      "any" usebundle => subtest("$(userlist)");
}

bundle agent subtest(user)
{
  classes:
      "invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(user)");

  reports:
    !invalid::
      "User name $(user) is valid at 4 letters";
    invalid::
      "User name $(user) is invalid";
}

Measuring Examples

Measurements
body common control
{
      bundlesequence => { "report" };
}


body monitor control
{
      forgetrate => "0.7";
      histograms => "true";
}


bundle agent report
{
  reports:
      "
   Free memory read at $(mon.av_free_memory_watch)
   cf_monitord read $(mon.value_monitor_self_watch)   
   ";
}


bundle monitor watch
{
  measurements:

      # Test 1 - extract string matching

      "/home/mark/tmp/testmeasure"

      handle => "blonk_watch",
      stream_type => "file",
      data_type => "string",
      history_type => "weekly",
      units => "blonks",
      match_value => find_blonks,
      action => sample_min("10");

      # Test 2 - follow a special process over time
      # using cfengine's process cache to avoid resampling

      "/var/cfengine/state/cf_rootprocs"

      handle => "monitor_self_watch",
      stream_type => "file",
      data_type => "int",
      history_type => "static",
      units => "kB",
      match_value => proc_value(".*cf-monitord.*",
                                "root\s+[0-9.]+\s+[0-9.]+\s+[0-9.]+\s+[0-9.]+\s+([0-9]+).*");

      # Test 3, discover disk device information

      "/bin/df"

      handle => "free_disk_watch",
      stream_type => "pipe",
      data_type => "slist",
      history_type => "static",
      units => "device",
      match_value => file_system;
      # Update this as often as possible

      # Test 4

      "/tmp/file"

      handle => "line_counter",
      stream_type => "file",
      data_type => "counter",
      match_value => scanlines("MYLINE.*"),
      history_type => "log";

}


body match_value scanlines(x)
{
      select_line_matching => "^$(x)$";
}


body action sample_min(x)
{
      ifelapsed => "$(x)";
      expireafter => "$(x)";
}


body match_value find_blonks
{
      select_line_number => "2";
      extraction_regex => "Blonk blonk ([blonk]+).*";
}


body match_value free_memory # not willy!
{
      select_line_matching => "MemFree:.*";
      extraction_regex => "MemFree:\s+([0-9]+).*";
}


body match_value proc_value(x,y)
{
      select_line_matching => "$(x)";
      extraction_regex => "$(y)";
}


body match_value file_system
{
      select_line_matching => "/.*";
      extraction_regex => "(.*)";
}

Software Administration Examples

Software and patch installation

Example for Debian:

body common control
{
      bundlesequence => { "packages" };
}
body agent control
{
      environment => { "DEBIAN_FRONTEND=noninteractive" };
}

bundle agent packages
{
  vars:
      # Test the simplest case -- leave everything to the yum smart manager

      "match_package" slist => {
                                 "apache2"
                                 #                          "apache2-mod_php5",
                                 #                          "apache2-prefork",
                                 #                          "php5"
      };

  packages:
      "$(match_package)"
      package_policy => "add",
      package_method => apt;
}

body package_method apt
{
    any::
      # ii  acpi      0.09-3ubuntu1

      package_changes => "bulk";
      package_list_command => "/usr/bin/dpkg -l";
      package_list_name_regex    => "ii\s+([^\s]+).*";
      package_list_version_regex => "ii\s+[^\s]+\s+([^\s]+).*";
      # package_list_arch_regex    => "none";

      package_installed_regex => ".*"; # all reported are installed
      #package_name_convention => "$(name)_$(version)_$(arch)";
      package_name_convention => "$(name)";
      # Use these only if not using a separate version/arch string
      # package_version_regex => "";
      # package_name_regex => "";
      # package_arch_regex => "";

      package_add_command => "/usr/bin/apt-get --yes install";
      package_delete_command => "/usr/bin/apt-get --yes remove";
      package_update_command =>  "/usr/bin/apt-get --yes dist-upgrade";
      #package_verify_command => "/bin/rpm -V";
}

Examples MSI for Windows, by name:

body common control
{
      bundlesequence => { "packages" };
}

bundle agent packages
{
  vars:
      "match_package" slist => {
                                 "7zip"
      };
  packages:
      "$(match_package)"
      package_policy => "update",
      package_select => ">=",
      package_architectures => { "x86_64" },
      package_version => "3.00",
      package_method => msi_vmatch;
}

body package_method msi_vmatch
{
      package_changes => "individual";
      package_file_repositories => { "$(sys.workdir)\software_updates\windows", "s:\su" };
      package_installed_regex => ".*";

      package_name_convention => "$(name)-$(version)-$(arch).msi";
      package_add_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
      package_update_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
      package_delete_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /x";
}

Windows MSI by version:

body common control
{
      bundlesequence => { "packages" };
}

bundle agent packages
{
  vars:
      "match_package" slist => {
                                 "7zip"
      };
  packages:
      "$(match_package)"
      package_policy => "update",
      package_select => ">=",
      package_architectures => { "x86_64" },
      package_version => "3.00",
      package_method => msi_vmatch;
}

body package_method msi_vmatch
{
      package_changes => "individual";
      package_file_repositories => { "$(sys.workdir)\software_updates\windows", "s:\su" };
      package_installed_regex => ".*";

      package_name_convention => "$(name)-$(version)-$(arch).msi";
      package_add_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
      package_update_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
      package_delete_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /x";
}

Examples for solaris:

bundle agent example_using_ips_package_method
{

  packages:

    solaris::

      "shell/zsh"
        package_policy => "add",
        package_method => ips;
}

bundle agent example_using_solaris_package_method
{
  files:

    solaris::

      "/tmp/$(admin_file)"
        create => "true",
        edit_defaults => empty_file, # defined in stdlib
        edit_line => create_solaris_admin_file; # defined in stdlib

  packages:

    solaris::

      "SMCzlib"
        package_policy => "add",
        package_method => solaris( "SMCzlib",
                                   "zlib-1.2.3-sol10-sparc-local",
                                   "$(admin_file)");
}

bundle agent example_using_solaris_install_package_method
{
  packages:

    solaris::

      "SMCzlib"
        package_method => solaris_install("/tmp/SMCzlib.adminfile")
}

bundle agent example_using_pkgsrc_module
{

  packages:

    solaris::

      "vim"
        policy => "present",
        package_module => pkgsrc;
}

Examples for yum based systems:

body common control
{
      bundlesequence => { "packages" };
      inputs => { "cfengine_stdlib.cf" }
}

bundle agent packages
{
  vars:
      # Test the simplest case -- leave everything to the yum smart manager

      "match_package" slist => {
                                 "apache2",
                                 "apache2-mod_php5",
                                 "apache2-prefork",
                                 "php5"
      };

  packages:
      "$(match_package)"
      package_policy => "add",
      package_method => yum;
}

SuSE Linux's package manager zypper is the most powerful alternative:

body common control
{
      bundlesequence => { "packages" };
      inputs => { "cfengine_stdlib.cf" }
}

bundle agent packages
{
  vars:
      # Test the simplest case -- leave everything to the zypper smart manager

      "match_package" slist => {
                                 "apache2",
                                 "apache2-mod_php5",
                                 "apache2-prefork",
                                 "php5"
      };

  packages:
      "$(match_package)"
      package_policy => "add",
      package_method => zypper;
}
Postfix mail configuration
body common control
{
      inputs => { "$(sys.libdir)/stdlib.cf" };
      bundlesequence  => { postfix };
}

bundle agent postfix
{
  vars:
      "prefix"     string => "/etc";
      "smtpserver" string => "localhost";
      "mailrelay"  string => "mailx.example.org";

  files:
      "$(prefix)/main.cf"
      edit_line => prefix_postfix;

      "$(prefix)/sasl-passwd"
      create    => "true",
      perms     => mo("0600","root"),
      edit_line => append_if_no_line("$(smtpserver) _$(sys.fqhost):chmsxrcynz4etfrejizhs22");
}


bundle edit_line prefix_postfix
{
      #
      # Value have the form NAME = "quoted space separated list"
      #
  vars:
      "ps[relayhost]"                  string => "[$(postfix.mailrelay)]:587";
      "ps[mydomain]"                   string => "iu.hio.no";
      "ps[smtp_sasl_auth_enable]"      string => "yes";
      "ps[smtp_sasl_password_maps]"    string => "hash:/etc/postfix/sasl-passwd";
      "ps[smtp_sasl_security_options]" string => "";
      "ps[smtp_use_tls]"               string => "yes";
      "ps[default_privs]"              string => "mailman";
      "ps[inet_protocols]"             string => "all";
      "ps[inet_interfaces]"            string => "127.0.0.1";
      "parameter_name" slist => getindices("ps");

  delete_lines:
      "$(parameter_name).*";

  insert_lines:
      "$(parameter_name) = $(ps[$(parameter_name)])";
}

bundle edit_line AppendIfNSL(parameter)
{
  insert_lines:
      "$(parameter)"; # This is default
}
Set up a web server

Adapt this template to your operating system by adding multiple classes. Each web server runs something like the present module, which is entered into the bundlesequence like this:

bundle agent web_server(state)
{
  vars:
      "document_root" string => "/";
      ####################################################
      # Site specific configuration - put it in this file
      ####################################################

      "site_http_conf" string => "/home/mark/CFEngine-inputs/httpd.conf";
      ####################################################
      # Software base
      ####################################################

      "match_package" slist => {
                                 "apache2",
                                 "apache2-mod_php5",
                                 "apache2-prefork",
                                 "php5"
      };
      #########################################################

  processes:
    web_ok.on::
      "apache2"
      restart_class => "start_apache";

    off::
      "apache2"
      process_stop => "/etc/init.d/apache2 stop";

      #########################################################

  commands:
    start_apache::
      "/etc/init.d/apache2 start"; # or startssl
      #########################################################

  packages:
      "$(match_package)"
      package_policy => "add",
      package_method => zypper,
      classes => if_ok("software_ok");
      #########################################################

  files:
    software_ok::
      "/etc/sysconfig/apache2"
      edit_line => fixapache,
      classes => if_ok("web_ok");
      #########################################################

  reports:
    !software_ok.on::
      "The web server software could not be installed";

      #########################################################

  classes:
      "on"  expression => strcmp("$(state)","on");
      "off" expression => strcmp("$(state)","off");
}


bundle edit_line fixapache
{
  vars:
      "add_modules"     slist => {
                                   "ssl",
                                   "php5"
      };

      "del_modules"     slist => {
                                   "php3",
                                   "php4",
                                   "jk"
      };

  insert_lines:
      "APACHE_CONF_INCLUDE_FILES=\"$(web_server.site_http_conf)\"";

  field_edits:
      #####################################################################
      # APACHE_MODULES="actions alias ssl php5 dav_svn authz_default jk" etc..
      #####################################################################

      "APACHE_MODULES=.*"
      # Insert module "columns" between the quoted RHS
      # using space separators
      edit_field => quotedvar("$(add_modules)","append");

      "APACHE_MODULES=.*"
      # Delete module "columns" between the quoted RHS
      # using space separators
      edit_field => quotedvar("$(del_modules)","delete");

      # if this line already exists, edit it

}
Add software packages to the system
body common control
{
      inputs => { "$(sys.libdir)/packages.cf" }
      bundlesequence => { "packages" };
}

bundle agent packages
{
  vars:
      "match_package" slist => {
                                 "apache2",
                                 "apache2-mod_php5",
                                 "apache2-prefork",
                                 "php5"
      };

  packages:
    solaris::
      "$(match_package)"
      package_policy => "add",
      package_method => solaris;

    redhat|SuSE::
      "$(match_package)"
      package_policy => "add",
      package_method => yum_rpm;

  methods:
      # equivalent in 3.6, no OS choices
      "" usebundle => ensure_present($(match_package));
}

Note you can also arrange to hide all the differences between package managers on an OS basis, but since some OSs have multiple managers, this might not be 100 percent correct.

Application baseline
bundle agent app_baseline
{
  methods:
    windows::
      "any" usebundle => detect_adobereader;

}

bundle agent detect_adobereader
{
  vars:
    windows::
      "value1" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "ENU_GUID");
      "value2" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "VersionMax");
      "value3" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "VersionMin");

  classes:
    windows::
      "is_correct" and => {
                            strcmp($(value1), "{AC76BA86-7AD7-1033-7B44-A93000000001}"),
                            strcmp($(value2), "90003"),
                            islessthan($(value3), "10001" )
      };

  reports:
    windows.!is_correct::
      'Adobe Reader is not correctly deployed - got "$(value1)", "$(value2)", "$(value3)"';
}
Service management (windows)
body common control
{
      bundlesequence  => { "winservice" };
}

bundle agent winservice
{
  vars:
      "bad_services" slist => { "Alerter",  "ClipSrv" };

  services:
    windows::
      "$(bad_services)"
      service_policy => "disable",
      comment => "Disable services that create security issues";
}
Software distribution
bundle agent check_software
{
  vars:
      # software to install if not installed
      "include_software" slist => {
                                    "7-zip-4.50-$(sys.arch).msi"
      };
      # this software gets updated if it is installed
      "autoupdate_software" slist => {
                                       "7-zip"
      };
      # software to uninstall if it is installed
      "exclude_software" slist => {
                                    "7-zip-4.65-$(sys.arch).msi"
      };

  methods:
      #  "any" usebundle => add_software( "@(check_software.include_software)", "$(sys.policy_hub)" );
      #  "any" usebundle => update_software( "@(check_software.autoupdate_software)", "$(sys.policy_hub)" );
      #  "any" usebundle => remove_software( "@(check_software.exclude_software)", "$(sys.policy_hub)" );
}

bundle agent add_software(pkg_name)
{
  vars:
    # dir to install from locally - can also check multiple directories
    "local_software_dir" string => "C:\Program Files\Cfengine\software\add";

  files:
    "$(local_software_dir)"
    copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/add", "$(srv)"),
    depth_search => recurse("1"),
        classes => if_repaired("got_newpkg"),
    comment => "Copy software from remote repository";

  packages:
    # When to check if the package is installed ?
    got_newpkg|any::
    "$(pkg_name)"
    package_policy           => "add",
    package_method           => msi_implicit( "$(local_software_dir)" ),
    classes                  => if_else("add_success", "add_fail" ),
    comment                  => "Install new software, if not already present";

    reports::
    add_fail::
    "Failed to install one or more packages";
}
      #########################################################################

bundle agent update_software(sw_names)
{
  vars:
    # dir to install from locally - can also check multiple directories
    "local_software_dir" string => "C:\Program Files\Cfengine\software\update";

  files:
    "$(local_software_dir)"
    copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/update", "$(srv)"),
    depth_search => recurse("1"),
        classes => if_repaired("got_newpkg"),
    comment => "Copy software updates from remote repository";


  packages:
    # When to check if the package is updated ?
    got_newpkg|any::
    "$(sw_names)"
    package_policy           => "update",
    package_select           => ">=",                 # picks the newest update available
    package_architectures    => { "$(sys.arch)" },    # install 32 or 64 bit package ?
    package_version          => "1.0",                # at least version 1.0
    package_method           => msi_explicit( "$(local_software_dir)" ),
    classes                  => if_else("update_success", "update_fail");

  reports:
    update_fail::
    "Failed to update one or more packages";
}
      #########################################################################

bundle agent remove_software(pkg_name)
{
  vars:
    # dir to install from locally - can also check multiple directories
    "local_software_dir" string => "C:\Program Files\Cfengine\software\remove";

  files:
    "$(local_software_dir)"
    copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/remove", "$(srv)"),
    depth_search => recurse("1"),
        classes => if_repaired("got_newpkg"),
        comment => "Copy removable software from remote repository";

  packages:
    got_newpkg::
    "$(pkg_name)"
    package_policy           => "delete",
    package_method           => msi_implicit( "$(local_software_dir)" ),
    classes                  => if_else("remove_success", "remove_fail" ),
    comment                  => "Remove software, if present";

    reports::
    remove_fail::
    "Failed to remove one or more packages";
}
Web server modules

The problem of editing the correct modules into the list of standard modules for the Apache web server. This example is based on the standard configuration deployment of SuSE Linux. Simply provide the list of modules you want and another list that you don't want.

body common control
{
      inputs => { "$(sys.libdir)/stdlib.cf" };
      bundlesequence  => {
                           apache
      };
}

bundle agent apache
{
  files:
    SuSE::
      "/etc/sysconfig/apache2"
      edit_line => fixapache;
}

bundle edit_line fixapache
{
  vars:
      "add_modules"     slist => {
                                   "dav",
                                   "dav_fs",
                                   "ssl",
                                   "php5",
                                   "dav_svn",
                                   "xyz",
                                   "superduper"
      };
      "del_modules"     slist => {
                                   "php3",
                                   "jk",
                                   "userdir",
                                   "imagemap",
                                   "alias"
      };
  insert_lines:
      "APACHE_CONF_INCLUDE_FILES=\"/site/masterfiles/local-http.conf\"";

  field_edits:
      #####################################################################
      # APACHE_MODULES="authz_host actions alias ..."
      #####################################################################

      # Values have the form NAME = "quoted space separated list"

      "APACHE_MODULES=.*"
      # Insert module "columns" between the quoted RHS
      # using space separators

      edit_field => quoted_var($(add_modules), "append");
      "APACHE_MODULES=.*"

      # Delete module "columns" between the quoted RHS
      # using space separators

      edit_field => quoted_var($(del_modules), "delete");
      # if this line already exists, edit it

}

Commands, Scripts, and Execution Examples

Command or script execution

Execute a command, for instance to start a MySQL service. Note that simple shell commands like rm or mkdir cannot be managed by CFEngine, so none of the protections that CFEngine offers can be applied to the process. Moreover, this starts a new process, adding to the burden on the system.

body common control
{
      bundlesequence  => { "my_commands" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent my_commands
{
  commands:
    Sunday.Hr04.Min05_10.myhost::
      "/usr/bin/update_db";

    any::
      "/etc/mysql/start"
      contain => setuid("mysql");
}
Change directory for command
body common control
{
      bundlesequence  => { "example" };
}

body contain cd(dir)
{
      chdir => "${dir}";
      useshell => "true";
}

bundle agent example
{
  commands:
      "/bin/pwd"
      contain => cd("/tmp");
}
Commands example
body common control
{
      bundlesequence  => { "my_commands" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent my_commands
{
  commands:
    Sunday.Hr04.Min05_10.myhost::
      "/usr/bin/update_db";

    any::
      "/etc/mysql/start"
      contain => setuid("mysql");
}
Execresult example
body common control
{
      bundlesequence  => { "example" };
}

bundle agent example
{
  vars:
      "my_result" string => execresult("/bin/ls /tmp","noshell");

  reports:
      "Variable is $(my_result)";
}
Methods
body common control
{
      bundlesequence  => { "testbundle"  };
      version => "1.2.3";
}

bundle agent testbundle
{
  vars:
      "userlist" slist => { "mark", "jeang", "jonhenrik", "thomas", "eben" };
  methods:
      "any" usebundle => subtest("$(userlist)");
}


bundle agent subtest(user)
{
  commands:
      "/bin/echo Fix $(user)";
  reports:
      "Finished doing stuff for $(user)";
}
Method validation
body common control
{
      bundlesequence  => { "testbundle"  };
      version => "1.2.3";
}

body agent control
{
      abortbundleclasses => { "invalid" };
}

bundle agent testbundle
{
  vars:
      "userlist" slist => { "xyz", "mark", "jeang", "jonhenrik", "thomas", "eben" };

  methods:
      "any" usebundle => subtest("$(userlist)");
}

bundle agent subtest(user)
{
  classes:
      "invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(user)");

  reports:
    !invalid::
      "User name $(user) is valid at 4 letters";
    invalid::
      "User name $(user) is invalid";
}
Trigger classes
body common control
{
    any::
      bundlesequence  => { "insert" };
}


bundle agent insert
{
  vars:
      "v" string => "
                One potato
                Two potato
                Three potahto
                Four
                ";

  files:
      "/tmp/test_insert"
      edit_line => Insert("$(insert.v)"),
      edit_defaults => empty,
      classes => trigger("edited");

  commands:
    edited::
      "/bin/echo make bananas";

  reports:
    edited::
      "The potatoes are bananas";
}


bundle edit_line Insert(name)
{
  insert_lines:
      "Begin$(const.n) $(name)$(const.n)End";
}

body edit_defaults empty
{
      empty_file_before_editing => "true";
}

body classes trigger(x)
{
      promise_repaired => { $(x) };
}

File and Directory Examples

Create files and directories

Create files and directories and set permissions.

body common control
{
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/test_plain"
      perms => system,
      create => "true";

      "/home/mark/tmp/test_dir/."
      perms => system,
      create => "true";
}

body perms system
{
      mode  => "0640";
}
Copy single files

Copy single files, locally (local_cp) or from a remote site (secure_cp). The Community Open Promise-Body Library (COPBL; cfengine_stdlib.cf) should be included in the /var/cfengine/inputs/ directory and input as below.

body common control
{
      bundlesequence  => { "mycopy" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent mycopy
{
  files:

      "/home/mark/tmp/test_plain"
      copy_from => local_cp("$(sys.workdir)/bin/file");

      "/home/mark/tmp/test_remote_plain"
      copy_from => secure_cp("$(sys.workdir)/bin/file","serverhost");
}
Copy directory trees

Copy directory trees, locally (local_cp) or from a remote site (secure_cp). (depth_search => recurse("")) defines the number of sublevels to include, ("inf") gets entire tree.

body common control
{
      bundlesequence  => { "my_recursive_copy" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent my_recursive_copy
{
  files:

      "/home/mark/tmp/test_dir"

      copy_from => local_cp("$(sys.workdir)/bin/."),
      depth_search => recurse("inf");

      "/home/mark/tmp/test_dir"

      copy_from => secure_cp("$(sys.workdir)/bin","serverhost"),
      depth_search => recurse("inf");

}
Disabling and rotating files

Use the following simple steps to disable and rotate files. See the Community Open Promise-Body Library if you wish more details on what disable and rotate does.

body common control
{
      bundlesequence  => { "my_disable" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent my_disable
{

  files:

      "/home/mark/tmp/test_create"
      rename => disable;

      "/home/mark/tmp/rotate_my_log"
      rename => rotate("4");

}
Add lines to a file

There are numerous approaches to adding lines to a file. Often the order of a configuration file is unimportant, we just need to ensure settings within it. A simple way of adding lines is show below.

body common control
{
    any::
      bundlesequence  => { "insert" };
}

bundle agent insert
{
  vars:
      "lines" string =>
      "
                One potato
                Two potato
                Three potatoe
                Four
                ";

  files:
      "/tmp/test_insert"
      create => "true",
      edit_line => append_if_no_line("$(insert.lines)");
}

Also you could write this using a list variable:

body common control
{
    any::
      bundlesequence  => { "insert" };
}

bundle agent insert
{
  vars:
      "lines" slist => { "One potato", "Two potato",
                         "Three potatoe", "Four" };

  files:
      "/tmp/test_insert"
      create => "true",
      edit_line => append_if_no_line("@(insert.lines)");
}
Check file or directory permissions
bundle agent check_perms
{
  vars:
      "ns_files" slist => {
                            "/local/iu/logs/admin",
                            "/local/iu/logs/security",
                            "/local/iu/logs/updates",
                            "/local/iu/logs/xfer"
      };

  files:
    NameServers::
      "/local/dns/pz"
      perms => mo("644","dns"),
      depth_search => recurse("1"),
      file_select => exclude("secret_file");

      "/local/iu/dns/pz/FixSerial"
      perms => m("755"),
      file_select => plain;

      "$(ns_files)"
      perms => mo("644","dns"),
      file_select => plain;

      "$(ftp)/pub"
      perms => mog("644","root","other");

      "$(ftp)/pub"
      perms => m("644"),
      depth_search => recurse("inf");

      "$(ftp)/etc"        perms => mog("111","root","other");
      "$(ftp)/usr/bin/ls" perms => mog("111","root","other");
      "$(ftp)/dev"        perms => mog("555","root","other");
      "$(ftp)/usr"        perms => mog("555","root","other");
}
Commenting lines in a file
body common control
{
      version => "1.2.3";
      inputs => { "$(sys.libdir)/stdlib.cf" };
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/cf3_test"
      create    => "true",
      edit_line => myedit("second");
}

bundle edit_line myedit(parameter)
{
  vars:
      "edit_variable" string => "private edit variable is $(parameter)";

  replace_patterns:
      # replace shell comments with C comments

      "#(.*)"
      replace_with => C_comment,
      select_region => MySection("New section");
}

body replace_with C_comment
{
      replace_value => "/* $(match.1) */"; # backreference 0
      occurrences => "all";  # first, last all
}

body select_region MySection(x)
{
      select_start => "\[$(x)\]";
      select_end => "\[.*\]";
}


body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/comment_test"
      create    => "true",
      edit_line => comment_lines_matching;
}

bundle edit_line comment_lines_matching
{
  vars:
      "regexes" slist => { "one.*", "two.*", "four.*" };
  replace_patterns:
      "^($(regexes))$"
      replace_with => comment("# ");
}

body replace_with comment(c)
{
      replace_value => "$(c) $(match.1)";
      occurrences => "all";
}


body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}



bundle agent testbundle
{
  files:
      "/home/mark/tmp/comment_test"
      create    => "true",
      edit_line => uncomment_lines_matching("\s*mark.*","#");
}

bundle edit_line uncomment_lines_matching(regex,comment)
{
  replace_patterns:
      "#($(regex))$" replace_with => uncomment;
}

body replace_with uncomment
{
      replace_value => "$(match.1)";
      occurrences => "all";
}
Copy files
  files:

"/var/cfengine/inputs"

handle => "update_policy",
perms => m("600"),
copy_from => u_scp("$(master_location)",@(policy_server)),
depth_search => recurse("inf"),
file_select => input_files,
action => immediate;

"/var/cfengine/bin"

perms => m("700"),
copy_from => u_scp("/usr/local/sbin","localhost"),
depth_search => recurse("inf"),
file_select => cf3_files,
action => immediate,
classes => on_change("reload");
Copy and flatten directory
body common control
{
      bundlesequence  => { "testbundle" };
      version => "1.2.3";
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/testflatcopy"
      comment  => "test copy promise",
      copy_from    => mycopy("/home/mark/LapTop/words","127.0.0.1"),
      perms        => system,
      depth_search => recurse("inf"),
      classes      => satisfied("copy_ok");

      "/home/mark/tmp/testcopy/single_file"
      comment  => "test copy promise",
      copy_from    => mycopy("/home/mark/LapTop/Cfengine3/trunk/README","127.0.0.1"),
      perms        => system;

  reports:
    copy_ok::
      "Files were copied..";
}

body perms system
{
      mode  => "0644";
}

body depth_search recurse(d)
{
      depth => "$(d)";
}

body copy_from mycopy(from,server)
{
      source      => "$(from)";
      servers     => { "$(server)" };
      compare     => "digest";
      verify      => "true";
      copy_backup => "true";                  #/false/timestamp
      purge       => "false";
      type_check  => "true";
      force_ipv4  => "true";
      trustkey => "true";
      collapse_destination_dir => "true";
}

body classes satisfied(x)
{
      promise_repaired => { "$(x)" };
      persist_time => "0";
}

body server control
{
      allowconnects         => { "127.0.0.1" , "::1" };
      allowallconnects      => { "127.0.0.1" , "::1" };
      trustkeysfrom         => { "127.0.0.1" , "::1" };
}

bundle server access_rules()
{
  access:
      "/home/mark/LapTop"
      admit   => { "127.0.0.1" };
}
Copy then edit a file convergently

To convergently chain a copy followed by edit, you need a staging file. First you copy to the staging file. Then you edit the final file and insert the staging file into it as part of the editing. This is convergent with respect to both stages of the process.

bundle agent master
{
  files:
      "$(final_destination)"
      create => "true",
      edit_line => fix_file("$(staging_file)"),
      edit_defaults => empty,
      perms => mo("644","root"),
      action => ifelapsed("60");
}

bundle edit_line fix_file(f)
{
  insert_lines:

      "$(f)"
      # insert this into an empty file to reconstruct
      insert_type => "file";

  replace_patterns:
      "searchstring"
      replace_with => With("replacestring");
}
Deleting lines from a file
body common control
{
      bundlesequence => { "test" };
}

bundle agent test
{
  files:
      "/tmp/resolv.conf"  # test on "/tmp/resolv.conf" #
      create        => "true",
      edit_line     => resolver,
      edit_defaults => def;
}


bundle edit_line resolver
{
  vars:
      "search" slist => { "search iu.hio.no cfengine.com", "nameserver 128.39.89.10" };

  delete_lines:
      "search.*";

  insert_lines:
      "$(search)" location => end;
}

body edit_defaults def
{
      empty_file_before_editing => "false";
      edit_backup => "false";
      max_file_size => "100000";
}

body location start
{
      # If not line to match, applies to whole text body
      before_after => "before";
}

body location end
{
      # If not line to match, applies to whole text body
      before_after => "after";
}
Deleting lines exception
body common control
{
      bundlesequence  => { "testbundle" };
}

bundle agent testbundle
{
  files:
      "/tmp/passwd_excerpt"
      create    => "true",
      edit_line => MarkNRoot;
}

bundle edit_line MarkNRoot
{
  delete_lines:
      "mark.*|root.*" not_matching => "true";
}
Delete files recursively

The rm_rf and rm_rf_depth bundles in the standard library make it easy to prune directory trees.

Editing files

This is a huge topic. See also See Add lines to a file, See Editing tabular files, etc. Editing a file can be complex or simple, depending on needs.

Here is an example of how to comment out lines matching a number of patterns:

body common control
{
      version         =>   "1.2.3";
      bundlesequence  => { "testbundle"  };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent testbundle
{
  vars:
      "patterns" slist => { "finger.*", "echo.*", "exec.*", "rstat.*",
                            "uucp.*", "talk.*" };
  files:
      "/etc/inetd.conf"
      edit_line => comment_lines_matching("@(testbundle.patterns)","#");
}
Editing tabular files
body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  vars:
      "userset" slist => { "one-x", "two-x", "three-x" };

  files:
      # Make a copy of the password file

      "/home/mark/tmp/passwd"
      create    => "true",
      edit_line => SetUserParam("mark","6","/set/this/shell");

      "/home/mark/tmp/group"
      create    => "true",
      edit_line => AppendUserParam("root","4","@(userset)");

  commands:
      "/bin/echo" args => $(userset);
}

bundle edit_line SetUserParam(user,field,val)
{
  field_edits:
      "$(user):.*"
      # Set field of the file to parameter
      edit_field => col(":","$(field)","$(val)","set");
}

bundle edit_line AppendUserParam(user,field,allusers)
{
  vars:
      "val" slist => { @(allusers) };

  field_edits:
      "$(user):.*"
      # Set field of the file to parameter
      edit_field => col(":","$(field)","$(val)","alphanum");
}

body edit_field col(split,col,newval,method)
{
      field_separator => $(split);
      select_field    => $(col);
      value_separator  => ",";
      field_value     => $(newval);
      field_operation => $(method);
      extend_fields => "true";
}
Inserting lines in a file
body common control
{
    any::
      bundlesequence  => { "insert" };
}


bundle agent insert
{
  vars:
      "v" string => "  One potato";

  files:
      "/tmp/test_insert"
      create => "true",
      edit_line => Insert("$(insert.v)");
}

bundle edit_line Insert(name)
{
  insert_lines:
      "  $(name)"
      whitespace_policy => { "ignore_leading", "ignore_embedded" };
}

body edit_defaults empty
{
      empty_file_before_editing => "true";
}


body common control
{
    any::
      bundlesequence  => { "insert" };
}


bundle agent insert
{
  vars:
      "v" string => "
                One potato
                Two potato
                Three potatoe
                Four
                ";

  files:
      "/tmp/test_insert"
      create => "true",
      edit_line => Insert("$(insert.v)"),
      edit_defaults => empty;
}

bundle edit_line Insert(name)
{
  insert_lines:
      "Begin$(const.n)$(name)$(const.n)End";
}

body edit_defaults empty
{
      empty_file_before_editing => "false";
}


body common control
{
    any::
      bundlesequence  => { "insert" };
}


bundle agent insert
{
  vars:
      "v" slist => {
                     "One potato",
                     "Two potato",
                     "Three potatoe",
                     "Four"
      };

  files:
      "/tmp/test_insert"
      create => "true",
      edit_line => Insert("@(insert.v)");
      #  edit_defaults => empty;

}

bundle edit_line Insert(name)
{
  insert_lines:
      "$(name)";
}

body edit_defaults empty
{
      empty_file_before_editing => "true";
}
Back references in filenames
body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  files:
      # The back reference in a path only applies to the last link
      # of the pathname, so the (tmp) gets ignored

      "/tmp/(cf3)_(.*)"
      edit_line => myedit("second $(match.2)");

      # but ...

      #  "/tmp/cf3_test"
      #       create    => "true",
      #       edit_line => myedit("second $(match.1)");

}

bundle edit_line myedit(parameter)
{
  vars:
      "edit_variable" string => "private edit variable is $(parameter)";

  insert_lines:
      "$(edit_variable)";

}
Add variable definitions to a file
body common control
{
      bundlesequence => { "setvars" };
      inputs => { "cf_std_library.cf" };
}


bundle agent setvars
{
  vars:

      # want to set these values by the names of their array keys


      "rhs[lhs1]" string => " Mary had a little pig";
      "rhs[lhs2]" string => "Whose Fleece was white as snow";
      "rhs[lhs3]" string => "And everywhere that Mary went";

      # oops, now change pig -> lamb


  files:

      "/tmp/system"

      create => "true",
      edit_line => set_variable_values("setvars.rhs");

}

Results in:

  • lhs1= Mary had a little pig
  • lhs2=Whose Fleece was white as snow
  • lhs3=And everywhere that Mary went

An example of this would be to add variables to /etc/sysctl.conf on Linux:

body common control
{
      bundlesequence => { "setvars" };
      inputs => { "cf_std_library.cf" };
}


bundle agent setvars
{
  vars:

      # want to set these values by the names of their array keys


      "rhs[net/ipv4/tcp_syncookies]" string => "1";
      "rhs[net/ipv4/icmp_echo_ignore_broadcasts]" string => "1";
      "rhs[net/ipv4/ip_forward]" string => "1";

      # oops, now change pig -> lamb


  files:

      "/etc/sysctl"

      create => "true",
      edit_line => set_variable_values("setvars.rhs");

}
Linking files
body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  files:
      # Make a copy of the password file

      "/home/mark/tmp/passwd"
      link_from     => linkdetails("/etc/passwd"),
      move_obstructions => "true";

      "/home/mark/tmp/linktest"
      link_from     => linkchildren("/usr/local/sbin");

      #child links
}

body link_from linkdetails(tofile)
{
      source        => "$(tofile)";
      link_type     => "symlink";
      when_no_source  => "force";      # kill
}

body link_from linkchildren(tofile)
{
      source        => "$(tofile)";
      link_type     => "symlink";
      when_no_source  => "force";      # kill
      link_children => "true";
      when_linking_children => "if_no_such_file"; # "override_file";
}


body common control
{
    any::
      bundlesequence  => {
                           "testbundle"
      };
}


bundle agent testbundle
{
  files:
      "/home/mark/tmp/test_to" -> "someone"
      depth_search => recurse("inf"),
      perms => modestuff,
      action => tell_me;
}

body depth_search recurse(d)
{
      rmdeadlinks => "true";
      depth => "$(d)";
}

body perms modestuff
{
      mode => "o-w";
}

body action tell_me
{
      report_level => "inform";
}
Listing files-pattern in a directory
body common control
{
      bundlesequence  => { "example" };
}

bundle agent example
{
  vars:
      "ls" slist => lsdir("/etc","p.*","true");

  reports:
      "ls: $(ls)";
}
Locate and transform files
body common control
{
    any::
      bundlesequence  => {
                           "testbundle"
      };
      version => "1.2.3";
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/testcopy"
      file_select => pdf_files,
      transformer => "/usr/bin/gzip $(this.promiser)",
      depth_search => recurse("inf");
}

body file_select pdf_files
{
      leaf_name => { ".*.pdf" , ".*.fdf" };
      file_result => "leaf_name";
}

body depth_search recurse(d)
{
      depth => "$(d)";
}
BSD flags
body common control
{
      bundlesequence => { "test" };
}
bundle agent test
{
  files:
    freebsd::
      "/tmp/newfile"
      create => "true",
      perms => setbsd;
}

body perms setbsd
{
      bsdflags => { "+uappnd","+uchg", "+uunlnk", "-nodump" };
}
Search and replace text
body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}


bundle agent testbundle
{
  files:
      "/tmp/replacestring"
      create    => "true",
      edit_line => myedit("second");
}

bundle edit_line myedit(parameter)
{
  vars:
      "edit_variable" string => "private edit variable is $(parameter)";

  replace_patterns:
      # replace shell comments with C comments

      "puppet"
      replace_with => With("cfengine 3");
}


body replace_with With(x)
{
      replace_value => $(x);
      occurrences => "first";
}

body select_region MySection(x)
{
      select_start => "\[$(x)\]";
      select_end => "\[.*\]";
}
Selecting a region in a file
body common control
{
      version => "1.2.3";
      bundlesequence  => { "testbundle"  };
}


bundle agent testbundle
{
  files:
      "/tmp/testfile"

      create    => "true",
      edit_line => myedit("second");
}


bundle edit_line myedit(parameter)
{
  vars:

      "edit_variable" string => "private edit variable is $(parameter)";

  replace_patterns:

      # comment out lines after start
      "([^#].*)"

      replace_with => comment,
      select_region => ToEnd("Start.*");
}


body replace_with comment
{
      replace_value => "# $(match.1)"; # backreference 0
      occurrences => "all";  # first, last all
}



body select_region ToEnd(x)
{
      select_start => $(x);
}
Warn if matching line in file
body common control
{
      bundlesequence  => { "testbundle" };
}

bundle agent testbundle
{
  files:
      "/var/cfengine/inputs/.*"
      edit_line => DeleteLinesMatching(".*cfenvd.*"),
      action => WarnOnly;
}

bundle edit_line DeleteLinesMatching(regex)
{
  delete_lines:
      "$(regex)" action => WarnOnly;
}

body action WarnOnly
{
      action_policy => "warn";
}

Interacting with Directory Services

Active directory example
bundle agent active_directory
{
  vars:
      # NOTE: Edit this to your domain, e.g. "corp", may also need more DC's after it
      "domain_name" string => "cftesting";
      "user_name"    string => "Guest";


      # NOTE: We can also extract data from remote Domain Controllers

    dummy.DomainController::
      "domain_controller"  string => "localhost";

      "userlist"    slist => ldaplist(
                                       "ldap://$(domain_controller)",
                                       "CN=Users,DC=$(domain_name),DC=com",
                                       "(objectClass=user)",
                                       "sAMAccountName",
                                       "subtree",
                                       "none");
  classes:
    dummy.DomainController::
      "gotuser" expression => ldaparray(
                                         "userinfo",
                                         "ldap://$(domain_controller)",
                                         "CN=$(user_name),CN=Users,DC=$(domain_name),DC=com",
                                         "(name=*)",
                                         "subtree",
                                         "none");

  reports:
    dummy.DomainController::
      'Username is "$(userlist)"';
    dummy.gotuser::
      "Got user data; $(userinfo[name]) has logged on $(userinfo[logonCount]) times";
}
Active list users directory example
bundle agent ldap
{
  vars:
      "userlist" slist => ldaplist(
                                    "ldap://cf-win2003",
                                    "CN=Users,DC=domain,DC=cf-win2003",
                                    "(objectClass=user)",
                                    "sAMAccountName",
                                    "subtree",
                                    "none");
  reports:
      'Username: "$(userlist)"';
}
Active directory show users example
bundle agent ldap
{
  classes:
      "gotdata" expression => ldaparray(
                                         "myarray",
                                         "ldap://cf-win2003",
                                         "CN=Test Pilot,CN=Users,DC=domain,DC=cf-win2003",
                                         "(name=*)",
                                         "subtree",
                                         "none");
  reports:
    gotdata::
      "Got user data";
    !gotdata::
      "Did not get user data";
}
LDAP interactions
body common control
{
      bundlesequence => { "ldap" , "followup"};
}

bundle agent ldap
{
  vars:
      # Get the first matching value for "uid"

      "value" string => ldapvalue("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","none");

      # Get all matching values for "uid" - should be a single record match
      "list" slist =>  ldaplist("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","none");

  classes:
      "gotdata" expression => ldaparray("myarray","ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(uid=mark)","subtree","none");
      "found" expression => regldap("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","jon.*","none");

  reports:
    linux::
      "LDAP VALUE $(value) found";
      "LDAP LIST VALUE $(list)";
    gotdata::
      "Found specific entry data  ...$(ldap.myarray[uid]),$(ldap.myarray[gecos]), etc";
    found::
      "Matched regex";
}
bundle agent followup
{
  reports:
    linux::
      "Different bundle ...$(ldap.myarray[uid]),$(ldap.myarray[gecos]),...";
}

File Template Examples

Templating

With CFEngine you have a choice between editing `deltas' into files or distributing more-or-less finished templates. Which method you should choose depends should be made by whatever is easiest.

If you are managing only part of the file, and something else (e.g. a package manager) is managing most of it, then it makes sense to use CFEngine file editing.
If you are managing everything in the file, then it makes sense to make the edits by hand and install them using CFEngine. You can use variables within source text files and let CFEngine expand them locally in situ, so that you can make generic templates that apply netwide.

Example template:

MYVARIABLE = something or other
HOSTNAME = $(sys.host)           # CFEngine fills this in

To copy and expand this template, you can use a pattern like this:

bundle agent get_template(final_destination,mode)
{
  vars:

      # This needs to ne preconfigured to your site

      "masterfiles"   string => "/home/mark/tmp";
      "this_template" string => lastnode("$(final_destination)","/");

  files:

      "$(final_destination).staging"

      comment => "Get template and expand variables for this host",
      perms => mo("400","root"),
      copy_from => remote_cp("$(masterfiles)/templates/$(this_template)","$(policy_server)"),
      action => if_elapsed("60");


      "$(final_destination)"

      comment => "Expand the template",
      create => "true",
      edit_line => expand_template("$(final_destination).staging"),
      edit_defaults => empty,
      perms => mo("$(mode)","root"),
      action => if_elapsed("60");
}

The the following driving code (based on `copy then edit') can be placed in a library, after configuring to your environmental locations:

bundle agent get_template(final_destination,mode)
{
  vars:

      # This needs to ne preconfigured to your site

      "masterfiles"   string => "/home/mark/tmp";
      "this_template" string => lastnode("$(final_destination)","/");

  files:

      "$(final_destination).staging"

      comment => "Get template and expand variables for this host",
      perms => mo("400","root"),
      copy_from => remote_cp("$(masterfiles)/templates/$(this_template)","$(policy_server)"),
      action => if_elapsed("60");


      "$(final_destination)"

      comment => "Expand the template",
      create => "true",
      edit_line => expand_template("$(final_destination).staging"),
      edit_defaults => empty,
      perms => mo("$(mode)","root"),
      action => if_elapsed("60");
}

Database Examples

Database creation
body common control
{
      bundlesequence => { "dummy" };
}

body knowledge control

{
      #sql_database => "postgres";

      sql_owner => "postgres";
      sql_passwd => ""; # No passwd
      sql_type => "postgres";
}

bundle knowledge dummy
{
  topics:
}

body common control
{
      bundlesequence => { "databases" };
}

bundle agent databases
{
      #commands:
      #  "/usr/bin/createdb cf_topic_maps",
      #        contain => as_user("mysql");

  databases:
      "knowledge_bank/topics"

      database_operation => "create",
      database_type => "sql",
      database_columns => {
                            "topic_name,varchar,256",
                            "topic_comment,varchar,1024",
                            "topic_id,varchar,256",
                            "topic_type,varchar,256",
                            "topic_extra,varchar,26"
      },
      database_server => myserver;
}



body database_server myserver
{
    none::
      db_server_owner => "postgres";
      db_server_password => "";
      db_server_host => "localhost";
      db_server_type => "postgres";
      db_server_connection_db => "postgres";

    any::
      db_server_owner => "root";
      db_server_password => "";
      db_server_host => "localhost";
      db_server_type => "mysql";
      db_server_connection_db => "mysql";
}

body contain as_user(x)
{
      exec_owner => "$(x)";
}

Network Examples

Find MAC address

Finding the ethernet address can be hard, but on Linux it is straightforward.

bundle agent test
{
  vars:

    linux::
      "interface" string => execresult("/sbin/ifconfig eth0","noshell");

    solaris::
      "interface" string => execresult("/usr/sbin/ifconfig bge0","noshell");

    freebsd::
      "interface" string => execresult("/sbin/ifconfig le0","noshell");

    darwin::
      "interface" string => execresult("/sbin/ifconfig en0","noshell");

  classes:

    linux::

      "ok" expression => regextract(
                                     ".*HWaddr ([^\s]+).*(\n.*)*",
                                     "$(interface)",
                                     "mac"
      );

    solaris::

      "ok" expression => regextract(
                                     ".*ether ([^\s]+).*(\n.*)*",
                                     "$(interface)",
                                     "mac"
      );

    freebsd::

      "ok" expression => regextract(
                                     ".*ether ([^\s]+).*(\n.*)*",
                                     "$(interface)",
                                     "mac"
      );

    darwin::

      "ok" expression => regextract(
                                     "(?s).*ether ([^\s]+).*(\n.*)*",
                                     "$(interface)",
                                     "mac"
      );

  reports:

    ok::

      "MAC address is $(mac[1])";

}
Client-server example
body common control
{
      bundlesequence  => { "testbundle" };
      version => "1.2.3";
      #fips_mode => "true";
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/testcopy"
      comment  => "test copy promise",
      copy_from    => mycopy("/home/mark/LapTop/words","127.0.0.1"),
      perms        => system,
      depth_search => recurse("inf"),
      classes      => satisfied("copy_ok");

      "/home/mark/tmp/testcopy/single_file"
      comment  => "test copy promise",
      copy_from    => mycopy("/home/mark/LapTop/Cfengine3/trunk/README","127.0.0.1"),
      perms        => system;

  reports:
    copy_ok::
      "Files were copied..";
}

body perms system
{
      mode  => "0644";
}

body depth_search recurse(d)
{
      depth => "$(d)";
}

body copy_from mycopy(from,server)
{
      source      => "$(from)";
      servers     => { "$(server)" };
      compare     => "digest";
      encrypt     => "true";
      verify      => "true";
      copy_backup => "true";                  #/false/timestamp
      purge       => "false";
      type_check  => "true";
      force_ipv4  => "true";
      trustkey => "true";
}

body classes satisfied(x)
{
      promise_repaired => { "$(x)" };
      persist_time => "0";
}

body server control
{
      allowconnects         => { "127.0.0.1" , "::1" };
      allowallconnects      => { "127.0.0.1" , "::1" };
      trustkeysfrom         => { "127.0.0.1" , "::1" };
      # allowusers
}

bundle server access_rules()
{
  access:
      "/home/mark/LapTop"
      admit   => { "127.0.0.1" };
}
Read from a TCP socket
body common control
{
      bundlesequence  => { "example" };
}

bundle agent example
{
  vars:
      "my80" string => readtcp("research.iu.hio.no","80","GET /index.php HTTP/1.1$(const.r)$(const.n)Host: research.iu.hio.no$(const.r)$(const.n)$(const.r)$(const.n)",20);

  classes:
      "server_ok" expression => regcmp(".*200 OK.*\n.*","$(my80)");

  reports:
    server_ok::
      "Server is alive";
    !server_ok::
      "Server is not responding - got $(my80)";
}
Set up a PXE boot server

Use CFEngine to set up a PXE boot server.

body common control
{
      bundlesequence => { "pxe" };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}


bundle agent pxe
{
  vars:
      "software" slist => {
                            "atftp",
                            "dhcp-server",
                            "syslinux",
                            "apache2"
      };

      "dirs" slist => {
            "/tftpboot",
            "/tftpboot/CFEngine/rpm",
            "/tftpboot/CFEngine/inputs",
            "/tftpboot/pxelinux.cfg",
            "/tftpboot/kickstart",
            "/srv/www/repos"
      };

      "tmp_location"    string => "/tftpboot/CFEngine/inputs";

      # Distros that we can install

      "rh_distros" slist => { "4.7", "5.2" };
      "centos_distros" slist => { "5.2" };

      # File contents of atftp configuration

      "atftpd_conf" string =>
      "

ATFTPD_OPTIONS=\"--daemon \"
ATFTPD_USE_INETD=\"no\"
ATFTPD_DIRECTORY=\"/tftpboot\"
ATFTPD_BIND_ADDRESSES=\"\"
       ";
      # File contents of DHCP configuration

      "dhcpd" string =>
      "

DHCPD_INTERFACE=\"eth0\"
DHCPD_RUN_CHROOTED=\"yes\"
DHCPD_CONF_INCLUDE_FILES=\"\"
DHCPD_RUN_AS=\"dhcpd\"
DHCPD_OTHER_ARGS=\"\"
DHCPD_BINARY=\"\"
       ";
      "dhcpd_conf" string =>
      "

allow booting;
allow bootp;
ddns-update-style none; ddns-updates off;
 subnet 192.168.0.0 netmask 255.255.255.0 {
   range 192.168.0.20 192.168.0.254;
   default-lease-time 3600;
   max-lease-time 4800;
   option routers 192.168.0.1;
   option domain-name \"test.CFEngine.com\";
   option domain-name-servers 192.168.0.1;
   next-server 192.168.0.1;
   filename \"pxelinux.0\";
 }
 group {
   host node1 {
     # Dummy machine

     hardware ethernet 00:0F:1F:94:FE:07;
     fixed-address 192.168.0.11;
     option host-name \"node1\";
   }
   host node2 {
     # Dell Inspiron 1150

     hardware ethernet 00:0F:1F:0E:70:E7;
     fixed-address 192.168.0.12;
     option host-name \"node2\";
   }
 }
        ";
      # File contains of Apache2 HTTP configuration

      "httpd_conf" string =>
      "

<Directory /srv/www/repos>
Options Indexes
AllowOverride None
</Directory>
Alias /repos /srv/www/repos

<Directory /tftpboot/distro/RHEL/5.2>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/rhel/5.2 /tftpboot/distro/RHEL/5.2
<Directory /tftpboot/distro/RHEL/4.7>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/rhel/4.7 /tftpboot/distro/RHEL/4.7
<Directory /tftpboot/distro/CentOS/5.2>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/centos/5.2 /tftpboot/distro/CentOS/5.2
<Directory /tftpboot/kickstart>
Options Indexes
AllowOverride None
</Directory>
Alias /kickstart /tftpboot/kickstart
<Directory /tftpboot/CFEngine>
Options Indexes
AllowOverride None
</Directory>
Alias /CFEngine /tftpboot/CFEngine
        ";
      # File contains of Kickstart for RHEL5 configuration

      "kickstart_rhel5_conf" string =>
      "


auth  --useshadow  --enablemd5
bootloader --location=mbr
clearpart --all --initlabel
graphical
firewall --disabled
firstboot --disable
key 77244a6377a8044a
keyboard no
lang en_US
logging --level=info
url --url=http://192.168.0.1/distro/rhel/5.2
network --bootproto=dhcp --device=eth0 --onboot=on
reboot
rootpw --iscrypted $1$eOnXdDPF$279sQ//zry6rnQktkATeM0
selinux --disabled
timezone --isUtc Europe/Oslo
install
part swap --bytes-per-inode=4096 --fstype=\"swap\" --recommended
part / --bytes-per-inode=4096 --fstype=\"ext3\" --grow --size=1
%packages
@core
@base
db4-devel
openssl-devel
gcc
flex
bison
libacl-devel
libselinux-devel
pcre-devel

device-mapper-multipath
-sysreport
%post
cd /root
rpm -i http://192.168.0.1/CFEngine/rpm/CFEngine-3.0.1b1-1.el5.i386.rpm

cd /etc/yum.repos.d
wget http://192.168.0.1/repos/RHEL5.Base.repo
rpm --import /etc/pki/rpm-gpg/*
yum clean all
yum update
mkdir -p /root/CFEngine_init
cd /root/CFEngine_init
wget -nd -r http://192.168.0.1/CFEngine/inputs/
/usr/local/sbin/cf-agent -B
/usr/local/sbin/cf-agent
        ";
      # File contains of PXElinux boot menu

      "pxelinux_boot_menu" string =>
      "

boot options:
     rhel5   - install 32 bit i386 RHEL 5.2             (MANUAL)
     rhel5w  - install 32 bit i386 RHEL 5.2             (AUTO)
     rhel4   - install 32 bit i386 RHEL 4.7 AS          (MANUAL)
     centos5 - install 32 bit i386 CentOS 5.2 (Desktop) (MANUAL)
        ";
      # File contains of PXElinux default configuration

      "pxelinux_default" string =>
      "


default rhel5
timeout 300
prompt 1
display pxelinux.cfg/boot.msg
F1 pxelinux.cfg/boot.msg

label rhel5
   kernel vmlinuz-RHEL5U2
   append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/rhel/5.2

label rhel5w
   kernel vmlinuz-RHEL5U2
   append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 ks=http://192.168.0.1/kickstart/kickstart-RHEL5U2.cfg

label rhel4
   kernel vmlinuz-RHEL4U7
   append initrd=initrd-RHEL4U7 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/rhel/4.7

label centos5
   kernel vmlinuz-CentOS5.2
   append initrd=initrd-CentOS5.2 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/centos/5.2
        ";
      # File contains of specified PXElinux default to be a RHEL5 webserver

      "pxelinux_rhel5_webserver" string =>
      "


default rhel5w
label rhel5w
   kernel vmlinuz-RHEL5U2
   append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 ks=http://192.168.0.1/kickstart/kickstart-RHEL5U2.cfg
        ";
      # File contains of a local repository for RHEL5

      "rhel5_base_repo" string =>
      "


[Server]
name=Server
baseurl=http://192.168.0.1/repos/rhel5/Server/
enable=1
[VT]
name=VT
baseurl=http://192.168.0.1/repos/rhel5/VT/
enable=1
[Cluster]
name=Cluster
baseurl=http://192.168.0.1/repos/rhel5/Cluster/
enable=1
[ClusterStorage]
name=Cluster Storage
baseurl=http://192.168.0.1/repos/rhel5/ClusterStorage/
enable=1
        ";
      #####################################################


  files:
    packages_ok::
      # Create files/dirs and edit the new files

      "/tftpboot/distro/RHEL/$(rh_distros)/."
      create => "true";

      "/tftpboot/distro/CentOS/$(centos_distros)/."
      create => "true";

      "$(dirs)/."
      create => "true";

      "/tftpboot/pxelinux.cfg/boot.msg"
      create => "true",
      perms => mo("644","root"),
      edit_line => append_if_no_line("$(pxelinux_boot_menu)"),
      edit_defaults => empty;

      "/tftpboot/pxelinux.cfg/default"
      create => "true",
      perms => mo("644","root"),
      edit_line => append_if_no_line("$(pxelinux_default)"),
      edit_defaults => empty;

      "/tftpboot/pxelinux.cfg/default.RHEL5.webserver"
      create => "true",
      perms => mo("644","root"),
      edit_line => append_if_no_line("$(pxelinux_rhel5_webserver)"),
      edit_defaults => empty;

      "/tftpboot/kickstart/kickstart-RHEL5U2.cfg"
      create => "true",
      perms => mo("644","root"),
      edit_line => append_if_no_line("$(kickstart_rhel5_conf)"),
      edit_defaults => empty;

      "/srv/www/repos/RHEL5.Base.repo"
      create => "true",
      perms => mo("644","root"),
      edit_line => append_if_no_line("$(rhel5_base_repo)"),
      edit_defaults => empty;

      # Copy files

      "/tftpboot"
      copy_from => local_cp("/usr/share/syslinux"),
      depth_search => recurse("inf"),
      file_select => pxelinux_files,
      action => immediate;

      "$(tmp_location)"
      perms => m("644"),
      copy_from => local_cp("/var/cfengine/inputs"),
      depth_search => recurse("inf"),
      file_select => input_files,
      action => immediate;

      # Edit atftp, dhcp and apache2 configurations

      "/etc/sysconfig/atftpd"
      edit_line => append_if_no_line("$(atftpd_conf)"),
      edit_defaults => empty,
      classes => satisfied("atftpd_ready");

      "/etc/sysconfig/dhcpd"
      edit_line => append_if_no_line("$(dhcpd)"),
      edit_defaults => empty;

      "/etc/dhcpd.conf"
      edit_line => append_if_no_line("$(dhcpd_conf)"),
      edit_defaults => empty,
      classes => satisfied("dhcpd_ready");

      "/etc/apache2/httpd.conf"
      edit_line => append_if_no_line("$(httpd_conf)"),
      edit_defaults => std_defs,
      classes => satisfied("apache2_ok");
      # Make a static link

      "/tftpboot/pxelinux.cfg/C0A8000C"
      link_from => mylink("/tftpboot/pxelinux.cfg/default.RHEL5.webserver");
      # Hash comment some lines for apaches

    apache2_ok::
      "/etc/apache2/httpd.conf"
      edit_line => comment_lines_matching_apache2("#"),
      classes => satisfied("apache2_ready");
  commands:
      # Restart services

    atftpd_ready::
      "/etc/init.d/atftpd restart";
    dhcpd_ready::
      "/etc/init.d/dhcpd restart";
    apache2_ready::
      "/etc/init.d/apache2 restart";

      #####################################################

  packages:

    ipv4_192_168_0_1::
      # Only the PXE boot server

      "$(software)"
      package_policy => "add",
      package_method => zypper,
      classes => satisfied("packages_ok");
}



body file_select pxelinux_files
{
      leaf_name => { "pxelinux.0" };
      file_result => "leaf_name";
}

body copy_from mycopy_local(from,server)
{
      source      => "$(from)";
      compare     => "digest";
}

body link_from mylink(x)
{
      source => "$(x)";
      link_type => "symlink";
}

body classes satisfied(new_class)
{
      promise_kept => { "$(new_class)"};
      promise_repaired => { "$(new_class)"};
}

bundle edit_line comment_lines_matching_apache2(comment)
{
  vars:
      "regex" slist => { "\s.*Options\sNone", "\s.*AllowOverride\sNone", "\s.*Deny\sfrom\sall" };

  replace_patterns:
      "^($(regex))$"
      replace_with => comment("$(comment)");
}

body file_select input_files
{
      leaf_name => { ".*.cf",".*.dat",".*.txt" };
      file_result => "leaf_name";
}
Resolver management
bundle common g # globals
{
  vars:
      "searchlist"  slist => {
                               "search iu.hio.no",
                               "search cfengine.com"
      };
      "nameservers" slist => {
                               "128.39.89.10",
                               "128.39.74.16",
                               "192.168.1.103"
      };
  classes:
      "am_name_server" expression => reglist("@(nameservers)","$(sys.ipv4[eth1])");
}


body common control
{
    any::
      bundlesequence  => {
                           "g",
                           resolver(@(g.searchlist),@(g.nameservers))
      };
      domain => "iu.hio.no";
}


bundle agent resolver(s,n)
{
  files:
      # When passing parameters down, we have to refer to
      # a source context

      "$(sys.resolv)"  # test on "/tmp/resolv.conf" #
      create        => "true",
      edit_line     => doresolv("@(this.s)","@(this.n)"),
      edit_defaults => reconstruct;
      # or edit_defaults => modify
}


bundle edit_line doresolv(s,n)
{
  vars:
      "line" slist => { @(s), @(n) };
  insert_lines:
      "$(line)";
}

body edit_defaults reconstruct
{
      empty_file_before_editing => "true";
      edit_backup => "false";
      max_file_size => "100000";
}

body edit_defaults modify
{
      empty_file_before_editing => "false";
      edit_backup => "false";
      max_file_size => "100000";
}
Mount NFS filesystem
body common control
{
      bundlesequence => { "mounts" };
}

bundle agent mounts
{
  storage:
      "/mnt" mount  => nfs("slogans.iu.hio.no","/home");
}

body mount nfs(server,source)
{
      mount_type => "nfs";
      mount_source => "$(source)";
      mount_server => "$(server)";
      #mount_options => { "rw" };
      edit_fstab => "true";
      unmount => "true";
}
Unmount NFS filesystem
body common control
{
      bundlesequence => { "mounts" };
}

bundle agent mounts
{
  storage:
      # Assumes the filesystem has been exported

      "/mnt" mount  => nfs("server.example.org","/home");
}

body mount nfs(server,source)
{
      mount_type => "nfs";
      mount_source => "$(source)";
      mount_server => "$(server)";
      edit_fstab => "true";
      unmount => "true";
}

System Security Examples

Distribute root passwords
body common control
{
      version => "1.2.3";
      inputs => { "$(sys.libdir)/stdlib.cf" };
      bundlesequence  => { "SetRootPassword" };
}

bundle common g
{
  vars:
      "secret_keys_dir" string => "/tmp";
}

bundle agent SetRootPassword
{
  vars:
      # Or get variables directly from server with Enterprise
      "remote-passwd" string => remotescalar("rem_password","127.0.0.1","yes");

      # Test this on a copy
  files:
      "/var/cfengine/ppkeys/rootpw.txt"
      copy_from => secure_cp("$(sys.fqhost)-root.txt","master_host.example.org");
      # or $(pw_class)-root.txt

      "/tmp/shadow"
      edit_line => SetRootPw;
}

bundle edit_line SetRootPw
{
  vars:
      # Assume this file contains a single string of the form root:passwdhash:
      # with : delimiters to avoid end of line/file problems

      "pw" int => readstringarray("rpw","$(sys.workdir)/ppkeys/rootpw.txt",
                                  "#[^\n]*",":","1","200");

  field_edits:
      "root:.*"
      # Set field of the file to parameter
      edit_field => col(":","2","$(rpw[root][1])","set");
}

bundle server passwords
{
  vars:
      # Read a file of format
      #
      # classname: host1,host2,host4,IP-address,regex.*,etc
      #

      "pw_classes" int => readstringarray("acl","$(g.secret_keys_dir)/classes.txt",
                                          "#[^\n]*",":","100","4000");
      "each_pw_class" slist => getindices("acl");

  access:
      "/secret/keys/$(each_pw_class)-root.txt"
      admit => splitstring("$(acl[$(each_pw_class)][1])" , ":" , "100"),
      ifencrypted => "true";
}
Distribute ssh keys
bundle agent allow_ssh_rootlogin_from_authorized_keys(user,sourcehost)
{
  vars:
      "local_cache"       string => "/var/cfengine/ssh_cache";
      "authorized_source" string => "/master/CFEngine/ssh_keys";

  files:
      "$(local_cache)/$(user).pub"
      comment => "Copy public keys from a an authorized cache into a cache on localhost",
      perms => mo("600","root"),
      copy_from => remote_cp("$(authorized_source)/$(user).pub","$(sourcehost)"),
      action => if_elapsed("60");

      "/root/.ssh/authorized_keys"
      comment => "Edit the authorized keys into the user's personal keyring",
      edit_line => insert_file_if_no_line_matching("$(user)","$(local_cache)/$(user).pub"),
      action => if_elapsed("60");
}

bundle agent allow_ssh_login_from_authorized_keys(user,sourcehost)
{
  vars:
      "local_cache"       string => "/var/cfengine/ssh_cache";
      "authorized_source" string => "/master/CFEngine/ssh_keys";

  files:
      "$(local_cache)/$(user).pub"
      comment => "Copy public keys from a an authorized cache into a cache on localhost",
      perms => mo("600","root"),
      copy_from => remote_cp("$(authorized_source)/$(user).pub","$(sourcehost)"),
      action => if_elapsed("60");

      "/home/$(user)/.ssh/authorized_keys"
      comment => "Edit the authorized keys into the user's personal keyring",
      edit_line => insert_file_if_no_line_matching("$(user)","$(local_cache)/$(user).pub"),
      action => if_elapsed("60");
}

bundle edit_line insert_file_if_no_line_matching(user,file)
{
  classes:
      "have_user" expression => regline("$(user).*","$(this.promiser)");
  insert_lines:
    !have_user::
      "$(file)"
      insert_type => "file";
}

System Information Examples

Change detection
body common control
{
      bundlesequence  => { "testbundle"  };
      inputs => { "cfengine_stdlib.cf" };
}

bundle agent testbundle
{
  files:
      "/usr"
      changes      => detect_all_change,
      depth_search => recurse("inf"),
      action       => background;
}
Hashing for change detection (tripwire)

Change detection is a powerful and easy way to monitor your environment, increase awareness and harden your system against security breaches.

body common control
{
      bundlesequence  => { "testbundle"  };
      inputs => { "$(sys.libdir)/stdlib.cf" };
}

bundle agent testbundle
{
  files:
      "/home/mark/tmp/web" -> "me"
      changes      => detect_all_change,
      depth_search => recurse("inf");
}
Check filesystem space
body common control
{
      bundlesequence  => { "example" };
}

bundle agent example
{
  vars:
      "free" int => diskfree("/tmp");

  reports:
      "Freedisk $(free)";
}
Class match example
body common control
{
      bundlesequence  => { "example" };
}

bundle agent example
{
  classes:
      "do_it" and => { classmatch(".*_3"), "linux" };

  reports:
    do_it::
      "Host matches pattern";
}
Global classes
body common control
{
      bundlesequence => { "g","tryclasses_1", "tryclasses_2" };
}

bundle common g
{
  classes:
      "one" expression => "any";
      "client_network" expression => iprange("128.39.89.0/24");
}

bundle agent tryclasses_1
{
  classes:
      "two" expression => "any";
}

bundle agent tryclasses_2
{
  classes:
      "three" expression => "any";
  reports:
    one.three.!two::
      "Success";
}

body common control
{
      bundlesequence => { "g","tryclasses_1", "tryclasses_2" };
}

bundle common g
{
  classes:
      "one" expression => "any";
      "client_network" expression => iprange("128.39.89.0/24");
}

bundle agent tryclasses_1
{
  classes:
      "two" expression => "any";
}

bundle agent tryclasses_2
{
  classes:
      "three" expression => "any";
  reports:
    one.three.!two::
      "Success";
}
Logging
body common control
{
      bundlesequence => { "test" };
}

bundle agent test
{
  vars:

      "software" slist => { "/root/xyz", "/tmp/xyz" };

  files:

      "$(software)"

      create => "true",
      action => logme("$(software)");

}

body action logme(x)
{
      log_kept => "/tmp/private_keptlog.log";
      log_failed => "/tmp/private_faillog.log";
      log_repaired => "/tmp/private_replog.log";
      log_string => "$(sys.date) $(x) promise status";
}

body common control
{
      bundlesequence => { "one" };
}

bundle agent one
{
  files:

      "/tmp/xyz"

      create => "true",
      action => log;

}

body action log
{
      log_level => "inform";
}

System Administration Examples

Centralized Management

These examples show a simple setup for starting with a central approach to management of servers. Centralization of management is a simple approach suitable for small environments with few requirements. It is useful for clusters where systems are all alike.

All hosts the same
Variation in hosts
Updating from a central hub
All hosts the same

This shows the simplest approach in which all hosts are the same. It is too simple for most environments, but it serves as a starting point. Compare it to the next section that includes variation.

body common control
{
      bundlesequence  => { "central" };
}


bundle agent central
{
  vars:
      "policy_server" string => "myhost.domain.tld";
      "mypackages" slist => {
                              "nagios"
                              "gcc",
                              "apache2",
                              "php5"
      };

  files:
      # Password management can be very simple if all hosts are identical

      "/etc/passwd"
      comment   => "Distribute a password file",
      perms     => mog("644","root","root"),
      copy_from => secure_cp("/home/mark/LapTop/words/RoadAhead","$(policy_server)");

  packages:
      "$(mypackages)"
      package_policy => "add",
      package_method => generic;

      # Add more promises below ...

}


body server control
{
      allowconnects         => { "127.0.0.1" , "::1", "10.20.30.0/24" };
      allowallconnects      => { "127.0.0.1" , "::1", "10.20.30.0/24" };
      trustkeysfrom         => { "127.0.0.1" , "::1", "10.20.30.0/24" };
      # allowusers
}

bundle server access_rules()
{
  access:
      # myhost.domain.tld makes this file available to 10.20.30*

    myhost_domain_tld::
      "/etc/passwd"
      admit   => { "127.0.0.1", "10.20.30.0/24" };
}
Variation in hosts
body common control
{
      bundlesequence  => { "central" };
}


bundle agent central
{
  classes:
      "mygroup_1" or => { "myhost", "host1", "host2", "host3" };
      "mygroup_2" or => { "host4", "host5", "host6" };

  vars:
      "policy_server" string => "myhost.domain.tld";
    mygroup_1::
      "mypackages" slist => {
                              "nagios"
                              "gcc",
                              "apache2",
                              "php5"
      };
    mygroup_2::
      "mypackages" slist => {
                              "apache"
                              "mysql",
                              "php5"
      };

  files:
      # Password management can be very simple if all hosts are identical

      "/etc/passwd"
      comment   => "Distribute a password file",
      perms     => mog("644","root","root"),
      copy_from => secure_cp("/etc/passwd","$(policy_server)");

  packages:
      "$(mypackages)"
      package_policy => "add",
      package_method => generic;

      # Add more promises below ...

}


body server control
{
      allowconnects         => { "127.0.0.1" , "::1", "10.20.30.0/24" };
      allowallconnects      => { "127.0.0.1" , "::1", "10.20.30.0/24" };
      trustkeysfrom         => { "127.0.0.1" , "::1", "10.20.30.0/24" };
      # allowusers
}

bundle server access_rules()
{
  access:
      # myhost.domain.tld makes this file available to 10.20.30*

    myhost_domain_tld::
      "/etc/passwd"
      admit   => { "127.0.0.1", "10.20.30.0/24" };
}
Updating from a central hub

The configuration bundled with the CFEngine source code contains an example of centralized updating of policy that covers more subtleties than this example, and handles fault tolerance. Here is the main idea behind it. For simplicity, we assume that all hosts are on network 10.20.30.* and that the central policy server/hub is 10.20.30.123.

bundle agent update
{
  vars:
      "master_location" string => "/var/cfengine/masterfiles";

      "policy_server"   string => "10.20.30.123",
      comment => "IP address to locate your policy host.";

  files:
      "$(sys.workdir)/inputs"
      perms => system("600"),
      copy_from => remote_cp("$(master_location)",$(policy_server)),
      depth_search => recurse("inf");

      "$(sys.workdir)/bin"
      perms => system("700"),
      copy_from => remote_cp("/usr/local/sbin","localhost"),
      depth_search => recurse("inf");
}

body server control
{
      allowconnects         => { "127.0.0.1" , "10.20.30.0/24" };
      allowallconnects      => { "127.0.0.1" , "10.20.30.0/24" };
      trustkeysfrom         => { "127.0.0.1" , "10.20.30.0/24" };
}

bundle server access_rules()
{
  access:
    10_20_30_123::
      "/var/cfengine/masterfiles"
      admit   => { "127.0.0.1", "10.20.30.0/24" };
}
Laptop support configuration

Laptops do not need a lot of confguration support. IP addresses are set by DHCP and conditions are changeable. But you want to set your DNS search domains to familiar settings in spite of local DHCP configuration, and another useful trick is to keep a regular backup of disk changes on the local disk. This won't help against disk destruction, but it is a huge advantage when your user accidentally deletes files while travelling or offline.

body common control
{
      bundlesequence  => {
                           "update",
                           "garbage_collection",
                           "main",
                           "backup",
      };
      inputs          => {
                           "update.cf",
                           "site.cf",
                           "library.cf"
      };
}

body agent control
{
      # if default runtime is 5 mins we need this for long jobs
      ifelapsed => "15";
}

body monitor control
{
      forgetrate => "0.7";
}

body executor control
{
      splaytime => "1";
      mailto => "mark@iu.hio.no";
      smtpserver => "localhost";
      mailmaxlines => "30";
      # Instead of a separate update script, now do this

      exec_command => "$(sys.workdir)/bin/cf-agent -f failsafe.cf && $(sys.workdir)/bin/cf-agent";
}

bundle agent main
{
  vars:
      "component" slist => { "cf-monitord", "cf-serverd" };
      # - - - - - - - - - - - - - - - - - - - - - - - -

  files:
      "$(sys.resolv)"  # test on "/tmp/resolv.conf" #
      create        => "true",
      edit_line     => resolver,
      edit_defaults => def;

  processes:
      "$(component)" restart_class => canonify("start_$(component)");
      # - - - - - - - - - - - - - - - - - - - - - - - -

  commands:
      "$(sys.workdir)/bin/$(component)"
      ifvarclass => canonify("start_$(component)");
}

bundle agent backup
{
  files:
      "/home/backup"
      copy_from => cp("/home/mark"),
      depth_search => recurse("inf"),
      file_select => exclude_files,
      action => longjob;
}

bundle agent garbage_collection
{
  files:
      "$(sys.workdir)/outputs"
      delete => tidy,
      file_select => days_old("3"),
      depth_search => recurse("inf");

}
Process management
body common control
{
      bundlesequence => { "test" };
}

bundle agent test
{
  processes:
      "sleep"
      signals => { "term", "kill" };
}

body common control
{
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  processes:
      "sleep"
      process_count   => up("sleep");
  reports:
    sleep_out_of_control::
      "Out of control";
}

body process_count up(s)
{
      match_range => "5,10"; # or irange("1","10");
      out_of_range_define => { "$(s)_out_of_control" };
}

body common control
{
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  processes:
      ".*"
      process_select  => proc_finder("a.*"),
      process_count   => up("cfservd");
}

body process_count up(s)
{
      match_range => "1,10"; # or irange("1","10");
      out_of_range_define => { "$(s)_out_of_control" };
}

body process_select proc_finder(p)
{
      stime_range => irange(ago("0","0","0","2","0","0"),now);
      process_result => "stime";
}


body common control
{
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  processes:
      ".*"
      process_select  => proc_finder("a.*"),
      process_count   => up("cfservd");
}

body process_count up(s)
{
      match_range => "1,10"; # or irange("1","10");
      out_of_range_define => { "$(s)_out_of_control" };
}

body process_select proc_finder(p)
{
      process_owner  => { "avahi", "bin" };
      command        => "$(p)";
      pid            => "100,199";
      vsize          => "0,1000";
      process_result => "command.(process_owner|vsize)";
}

body common control
{
      bundlesequence => { "process_restart" };
}

bundle agent process_restart
{
  processes:
      "/usr/bin/daemon"
      restart_class => "launch";

  commands:
    launch::
      "/usr/bin/daemon";
}
body common control
{
      bundlesequence => { "process_restart" };
}

bundle agent process_restart
{
  vars:
      "component" slist => {
                             "cf-monitord",
                             "cf-serverd",
                             "cf-execd"
      };

  processes:
      "$(component)"
      restart_class => canonify("start_$(component)");

  commands:
      "/var/cfengine/bin/$(component)"
      ifvarclass => canonify("start_$(component)");
}

body common control
{
      bundlesequence  => { "testbundle"  };
}

bundle agent testbundle
{
  processes:

      "cfservd"
      process_count   => up("cfservd");

    cfservd_out_of_control::
      "cfservd"
      signals         => { "stop" , "term" },
      restart_class   => "start_cfserv";

  commands:
    start_cfserv::
      "/usr/local/sbin/cfservd";
}

body process_count up(s)
{
      match_range => "1,10"; # or irange("1","10");
      out_of_range_define => { "$(s)_out_of_control" };
}
Kill process
body common control
{
      bundlesequence => { "test" };
}

bundle agent test
{
  processes:

      "sleep"
      signals => { "term", "kill" };
}
Restart process

A basic pattern for restarting processes:

body common control
{
      bundlesequence => { "process_restart" };
}



bundle agent process_restart
{
  processes:

      "/usr/bin/daemon"
      restart_class => "launch";

  commands:

    launch::
      "/usr/bin/daemon";

}

This can be made more sophisticated to handle generic lists:

body common control
{
      bundlesequence => { "process_restart" };
}



bundle agent process_restart
{
  vars:

      "component" slist => {
                             "cf-monitord",
                             "cf-serverd",
                             "cf-execd"
      };
  processes:

      "$(component)"
      restart_class => canonify("start_$(component)");

  commands:

      "/var/cfengine/bin/$(component)"
      ifvarclass => canonify("start_$(component)");

}

Why? Separating this into two parts gives a high level of control and conistency to CFEngine. There are many options for command execution, like the ability to run commands in a sandbox or as `setuid'. These should not be reproduced in processes.

Mount a filesystem
body common control
{
      bundlesequence => { "mounts" };
}

bundle agent mounts
{
  storage:
      "/mnt" mount  => nfs("slogans.iu.hio.no","/home");
}

body mount nfs(server,source)
{
      mount_type => "nfs";
      mount_source => "$(source)";
      mount_server => "$(server)";
      #mount_options => { "rw" };
      edit_fstab => "true";
      unmount => "true";
}
Manage a system process
Ensure running
Ensure not running
Prune processes
Ensure running

The simplest example might look like this:

bundle agent restart_process
{
  processes:

      "httpd"

      comment => "Make sure apache web server is running",
      restart_class => "restart_httpd";

  commands:

    restart_httpd::

      "/etc/init.d/apache2 restart";

}

This example shows how the CFEngine components could be started using a pattern.

bundle agent CFEngine_processes
{
  vars:

      "components" slist => { "cf-execd", "cf-monitord", "cf-serverd", "cf-hub" };

  processes:

      "$(components)"

      comment => "Make sure server parts of CFEngine are running",
      restart_class => canonify("start_$(component)");

  commands:

      "$(sys.workdir)/bin/$(component)"

      comment => "Make sure server parts of CFEngine are running",
      ifvarclass => canonify("start_$(components)");

}
Ensure not running
bundle agent restart_process
{
  vars:

      "killprocs" slist => { "snmpd", "gameserverd", "irc", "crack" };

  processes:

      "$(killprocs)"

      comment => "Ensure processes are not running",
      signals => { "term", "kill" };
}
Prune processes

This example kills processes owned by a particular user that have exceeded 100000 bytes of resident memory.

body common control
{
      bundlesequence  => { "testbundle"  };
}


bundle agent testbundle
{
  processes:

      ".*"

      process_select  => big_processes("mark"),
      signals => { "term" };
}


body process_select big_processes(o)
{
      process_owner => { $(o) };
      rsize => irange("100000","900000");
      process_result => "rsize.process_owner";
}
Set up HPC clusters

HPC cluster machines are usually all identical, so the CFEngine configuration is very simple. HPC clients value CPU and memory resources, so we can shut down unnecessary services to save CPU. We can also change the scheduling rate of CFEngine to run less frequently, and save a little:

body executor control
{
      splaytime => "1";
      mailto => "cfengine@example.com";
      smtpserver => "localhost";
      mailmaxlines => "30";
      # Once per hour, on the hour

      schedule     => { "Min00_05" };
}

bundle agent services_disable
{
  vars:
      # list all of xinetd services (case sensitive)

      "xinetd_services" slist => {
                                   "imap",
                                   "imaps",
                                   "ipop2",
                                   "ipop3",
                                   "krb5-telnet",
                                   "klogin",
                                   "kshell",
                                   "ktalk",
                                   "ntalk",
                                   "pop3s",
      };
  methods:
      # perform the actual disable all xinetd services according to the list above

      "any"  usebundle => disable_xinetd("$(xinetd_services)");

  processes:
      "$(xinetd_services)"
      signals => { "kill" };
}

bundle agent disable_xinetd(name)
{
  vars:
      "status" string => execresult("/sbin/chkconfig --list $(name)", "useshell");

  classes:
      "on"  expression => regcmp(".*on.*","$(status)");

  commands:
    on::
      "/sbin/chkconfig $(name) off",
      comment => "disable $(name) service";

  reports:
    on::
      "disable $(name) service.";
}
Set up name resolution

There are many ways to do name resolution setup1 We write a reusable bundle using the editing features.

A simple and straightforward approach is to maintain a separate modular bundle for this task. This avoids too many levels of abstraction and keeps all the information in one place. We implement this as a simple editing promise for the /etc/resolv.conf file.

bundle agent system_files
{
  files:
      "$(sys.resolv)"  # test on "/tmp/resolv.conf" #
      comment       => "Add lines to the resolver configuration",
      create        => "true",
      edit_line     => resolver,
      edit_defaults => std_edits;
      # ...other system files ...

}

bundle edit_line resolver
{
  delete_lines:
      # delete any old name servers or junk we no longer need

      "search.*";
      "nameserver 80.65.58.31";
      "nameserver 80.65.58.32";
      "nameserver 82.103.128.146";
      "nameserver 78.24.145.4";
      "nameserver 78.24.145.5";
      "nameserver 128.39.89.10";

  insert_lines:
      "search mydomain.tld" location => start;
    special_net::
      "nameserver 128.39.89.8";
      "nameserver 128.39.74.66";
    !special_net::
      "nameserver 128.38.34.12";
    any::
      "nameserver 212.112.166.18";
      "nameserver 212.112.166.22";
}

A second approach is to try to conceal the operational details behind a veil of abstraction.

bundle agent system_files
{
  vars:
      "searchlist"  string => "iu.hio.no CFEngine.com";
      "nameservers" slist => {
                               "128.39.89.10",
                               "128.39.74.16",
                               "192.168.1.103"
      };

  files:
      "$(sys.resolv)"  # test on "/tmp/resolv.conf" #
      create        => "true",
      edit_line     => doresolv("$(s)","@(this.n)"),
      edit_defaults => empty;
      # ....

}

bundle edit_line doresolv(search,names)
{
  insert_lines:
      "search $(search)";
      "nameserver $(names)";
}

bundle agent system_files { # ...

files: "/etc/hosts" comment => "Add hosts to the /etc/hosts file", edit_line => fix_etc_hosts; }

bundle edit_line fix_etc_hosts { vars: "names[127.0.0.1]" string => "localhost localhost.CFEngine.com"; "names[128.39.89.12]" string => "myhost myhost.CFEngine.com"; "names[128.39.89.13]" string => "otherhost otherhost.CFEngine.com"; # etc

  "i" slist => getindices("names");

insert_lines: "$(i) $(names[$(i)])"; } ```

DNS is not the only name service, of course. Unix has its older /etc/hosts file which can also be managed using file editing. We simply append this to the system_files bundle.

bundle agent system_files
{
  vars:
      "searchlist"  string => "iu.hio.no CFEngine.com";
      "nameservers" slist => {
                               "128.39.89.10",
                               "128.39.74.16",
                               "192.168.1.103"
      };

  files:
      "$(sys.resolv)"  # test on "/tmp/resolv.conf" #
      create        => "true",
      edit_line     => doresolv("$(s)","@(this.n)"),
      edit_defaults => empty;
      # ....

}

bundle edit_line doresolv(search,names)
{
  insert_lines:
      "search $(search)";
      "nameserver $(names)";
}

bundle agent system_files { # ...

files: "/etc/hosts" comment => "Add hosts to the /etc/hosts file", edit_line => fix_etc_hosts; }

bundle edit_line fix_etc_hosts { vars: "names[127.0.0.1]" string => "localhost localhost.CFEngine.com"; "names[128.39.89.12]" string => "myhost myhost.CFEngine.com"; "names[128.39.89.13]" string => "otherhost otherhost.CFEngine.com"; # etc

  "i" slist => getindices("names");

insert_lines: "$(i) $(names[$(i)])"; } ```

Set up sudo

Setting up sudo is straightforward, and is best managed by copying trusted files from a repository.

bundle agent system_files
{
  vars:
      "masterfiles" string => "/subversion_projects/masterfiles";
      # ...

  files:
      "/etc/sudoers"
      comment => "Make sure the sudo configuration is secure and up to date",
      perms => mog("440","root","root"),
      copy_from => secure_cp("$(masterfiles)/sudoers","$(policy_server)");
}
Environments (virtual)
body common control
{
      bundlesequence  => { "my_vm_cloud" };
}

bundle agent my_vm_cloud
{
  vars:
      "vms[atlas]" slist => { "guest1", "guest2", "guest3" };

  environments:
    scope||any::  # These should probably be in class "any" to ensure uniqueness
      "$(vms[$(sys.host)])"
      environment_resources => virt_xml("$(xmlfile[$(this.promiser)])"),
      environment_interface => vnet("eth0,192.168.1.100/24"),
      environment_type      => "test",
      environment_host      => "atlas";
      # default environment_state => "create" on host, and "suspended elsewhere"
}

body environment_resources virt_xml(specfile)
{
      env_spec_file => "$(specfile)";
}

body environment_interface vnet(primary)
{
      env_name      => "$(this.promiser)";
      env_addresses => { "$(primary)" };
    host1::
      env_network => "default_vnet1";
    host2::
      env_network => "default_vnet2";
}
Environment variables
body common control
{
      bundlesequence  => { "my_vm_cloud" };
}

bundle agent my_vm_cloud
{
  environments:
      "centos5"
      environment_resources => virt_xml,
      environment_type      => "xen",
      environment_host      => "ursa-minor";
      # default environment_state => "create" on host, and "suspended elsewhere"
}

body environment_resources virt_xml
{
      env_spec_file => "/srv/xen/centos5-libvirt-create.xml";
}
Tidying garbage files

Emulating the `tidy' feature of CFEngine 2.

body common control
{
    any::
      bundlesequence  => { "testbundle" };
}

bundle agent testbundle
{
  files:
      "/tmp/test"
      delete => tidy,
      file_select => zero_age,
      depth_search => recurse("inf");
}

body depth_search recurse(d)
{
      #include_basedir => "true";
      depth => "$(d)";
}

body delete tidy
{
      dirlinks => "delete";
      rmdirs   => "false";
}

body file_select zero_age
{
      mtime     => irange(ago(1,0,0,0,0,0),now);
      file_result => "mtime";
}

System File Examples

Editing password or group files

To change the password of a system, we need to edit a file. A file is a complex object – once open there is a new world of possible promises to make about its contents. CFEngine has bundles of promises that are specially for editing.

body common control
{
      inputs => { "$(sys.libdir)/stdlib.cf" };
      bundlesequence => { "edit_passwd" };
}
bundle agent edit_passwd
{
  vars:
      "userset" slist => { "user1", "user2", "user3" };

  files:
      "/etc/passwd"
      edit_line => set_user_field("mark","7","/set/this/shell");

      "/etc/group"
      edit_line => append_user_field("root","4","@(main.userset)");
}
Editing password or group files custom

In this example the bundles from the Community Open Promise-Body Library are included directly in the policy instead of being input as a separate file.

body common control
{
      bundlesequence => { "addpasswd" };
}

bundle agent addpasswd
{
  vars:

      # want to set these values by the names of their array keys

      "pwd[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
      "pwd[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
      "pwd[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";

  files:

      "/tmp/passwd"
      create => "true",
      edit_line => append_users_starting("addpasswd.pwd");

}


bundle edit_line append_users_starting(v)
{
  vars:
      "index"        slist => getindices("$(v)");

  classes:
      "add_$(index)" not => userexists("$(index)");

  insert_lines:
      "$($(v)[$(index)])",
      ifvarclass => "add_$(index)";
}

bundle edit_line append_groups_starting(v)
{
  vars:
      "index"        slist => getindices("$(v)");

  classes:
      "add_$(index)" not => groupexists("$(index)");

  insert_lines:
      "$($(v)[$(index)])",
      ifvarclass => "add_$(index)";
}
Log rotation
body common control
{
      bundlesequence  => { "testbundle" };
}


bundle agent testbundle

{
  files:
      "/home/mark/tmp/rotateme"
      rename => rotate("4");
}


body rename rotate(level)
{
      rotate => "$(level)";
}
Garbage collection
body common control
{
      bundlesequence => { "garbage_collection" };
      inputs => { "cfengine_stdlib.cf" };
}


bundle agent garbage_collection
{
  files:

    Sunday::

      "$(sys.workdir)/nova_repair.log"

      comment => "Rotate the promises repaired logs each week",
      rename => rotate("7"),
      action => if_elapsed("10000");

      "$(sys.workdir)/nova_notkept.log"

      comment => "Rotate the promises not kept logs each week",
      rename => rotate("7"),
      action => if_elapsed("10000");

      "$(sys.workdir)/promise.log"

      comment => "Rotate the promises not kept logs each week",
      rename => rotate("7"),
      action => if_elapsed("10000");

    any::

      "$(sys.workdir)/outputs"

      comment => "Garbage collection of any output files",
      delete => tidy,
      file_select => days_old("3"),
      depth_search => recurse("inf");

      "$(sys.workdir)/"

      comment => "Garbage collection of any output files",
      delete => tidy,
      file_select => days_old("14"),
      depth_search => recurse("inf");

      # Other resources


      "/tmp"

      comment => "Garbage collection of any temporary files",
      delete => tidy,
      file_select => days_old("3"),
      depth_search => recurse("inf");

      "/var/log/apache2/.*bz"

      comment => "Garbage collection of rotated log files",
      delete => tidy,
      file_select => days_old("30"),
      depth_search => recurse("inf");

      "/var/log/apache2/.*gz"

      comment => "Garbage collection of rotated log files",
      delete => tidy,
      file_select => days_old("30"),
      depth_search => recurse("inf");

      "/var/log/zypper.log"

      comment => "Prevent the zypper log from choking the disk",
      rename => rotate("0"),
      action => if_elapsed("10000");

}
Manage a system file
Simple template
Simple versioned template
Macro template
Custom editing
Simple template
bundle agent hand_edited_config_file
{
  vars:
      "file_template" string =>
      "

127.0.0.1       localhost
::1             localhost ipv6-localhost ipv6-loopback
fe00::0         ipv6-localnet
ff00::0         ipv6-mcastprefix
ff02::1         ipv6-allnodes
ff02::2         ipv6-allrouters
ff02::3         ipv6-allhosts
10.0.0.100      host1.domain.tld host1
10.0.0.101      host2.domain.tld host2
10.0.0.20       host3.domain.tld host3
10.0.0.21       host4.domain.tld host4
";
      ##############################################################

  files:
      "/etc/hosts"
      comment => "Define the content of all host files from this master source",
      create => "true",
      edit_line => append_if_no_lines("$(file_template)"),
      edit_defaults => empty,
      perms => mo("$(mode)","root"),
      action => if_elapsed("60");
}
Simple versioned template

The simplest approach to managing a file is to maintain a master copy by hand, keeping it in a version controlled repository (e.g. svn), and installing this version on the end machine.

We'll assume that you have a version control repository that is located on some independent server, and has been checked out manually once (with authentication) in /mysite/masterfiles.

bundle agent hand_edited_config_file
{
  vars:

      "masterfiles"   string => "/mysite/masterfiles";
      "policy_server" string => "policy_host.domain.tld";

  files:

      "/etc/hosts"
      comment => "Synchronize hosts with a hand-edited template in svn",
      perms => m("644"),
      copy_from => remote_cp("$(masterfiles)/trunk/hosts_master","$(policy_server)");

  commands:

      "/usr/bin/svn update"
      comment => "Update the company document repository including manuals to a local copy",
      contain => silent_in_dir("$(masterfiles)/trunk"),
      ifvarclass => canonify("$(policy_server)");
}
Macro template

The next simplest approach to file management is to add variables to the template that will be expanded into local values at the end system, e.g. using variables like ‘$(sys.host)’ for the name of the host within the body of the versioned template.

bundle agent hand_edited_template
{
  vars:

      "masterfiles"   string => "/mysite/masterfiles";
      "policy_server" string => "policy_host.domain.tld";

  files:

      "/etc/hosts"

      comment => "Synchronize hosts with a hand-edited template in svn",
      perms => m("644"),
      create => "true",
      edit_line => expand_template("$(masterfiles)/trunk/hosts_master"),
      edit_defaults => empty,
      action => if_elapsed("60");

  commands:

      "/usr/bin/svn update"

      comment => "Update the company document repository including manuals to a local copy",
      contain => silent_in_dir("$(masterfiles)/trunk"),
      ifvarclass => canonify("$(policy_server)");

}

The macro template file may contain variables, as below, that get expanded by CFEngine.

bundle agent hand_edited_template
{
  vars:

      "masterfiles"   string => "/mysite/masterfiles";
      "policy_server" string => "policy_host.domain.tld";

  files:

      "/etc/hosts"

      comment => "Synchronize hosts with a hand-edited template in svn",
      perms => m("644"),
      create => "true",
      edit_line => expand_template("$(masterfiles)/trunk/hosts_master"),
      edit_defaults => empty,
      action => if_elapsed("60");

  commands:


      "/usr/bin/svn update"

      comment => "Update the company document repository including manuals to a local copy",
      contain => silent_in_dir("$(masterfiles)/trunk"),
      ifvarclass => canonify("$(policy_server)");

}

127.0.0.1 localhost $(sys.host) ::1 localhost ipv6-localhost ipv6-loopback fe00::0 ipv6-localnet ff00::0 ipv6-mcastprefix ff02::1 ipv6-allnodes ff02::2 ipv6-allrouters ff02::3 ipv6-allhosts 10.0.0.100 host1.domain.tld host1 10.0.0.101 host2.domain.tld host2 10.0.0.20 host3.domain.tld host3 10.0.0.21 host4.domain.tld host4

$(definitions.more_hosts) ```

Custom editing

If you do not control the starting state of the file, because it is distributed by an operating system vendor for instance, then editing the final state is the best approach. That way, you will get changes that are made by the vendor, and will ensure your own modifications are kept even when updates arrive.

bundle agent modifying_managed_file
{
  vars:

      "data"   slist => { "10.1.2.3 sirius", "10.1.2.4 ursa-minor", "10.1.2.5 orion"};

  files:

      "/etc/hosts"

      comment => "Append a list of lines to the end of a file if they don't exist",
      perms => m("644"),
      create => "true",
      edit_line => append_if_no_lines("modifying_managed_file.data"),
      action => if_elapsed("60");
}

Another example shows how to set the values of variables using a data-driven approach and methods from the standard library.

body common control
{
      bundlesequence  => { "testsetvar" };
}

bundle agent testsetvar
{
  vars:
      "v[variable_1]" string => "value_1";
      "v[variable_2]" string => "value_2";

  files:
      "/tmp/test_setvar"
      edit_line => set_variable_values("testsetvar.v");
}

Windows Registry Examples

Windows registry
body common control
{
      bundlesequence => { "reg" };
}

bundle agent reg
{
  vars:

      "value" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine","value3");

  reports:
    windows::
      "Value extracted: $(value)";
}
unit_registry_cache.cf
body common control
{
      bundlesequence => {
                          #                   "registry_cache"
                          #                   "registry_restore"
      };
}

bundle agent registry_cache
{
  databases:
    windows::
      "HKEY_LOCAL_MACHINE\SOFTWARE\Adobe"
      database_operation => "cache",
      database_type      => "ms_registry",
      comment => "Save correct registry settings for Adobe products";
}

bundle agent registry_restore
{
  databases:
    windows::
      "HKEY_LOCAL_MACHINE\SOFTWARE\Adobe"
      database_operation => "restore",
      database_type      => "ms_registry",
      comment => "Make sure Adobe products have correct registry settings";
}
unit_registry.cf
body common control
{
      bundlesequence => { "databases" };
}

bundle agent databases
{
  databases:
    windows::
      # Registry has (value,data) pairs in "keys" which are directories

      #  "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS"
      #    database_operation => "create",
      #    database_type     => "ms_registry";

      #  "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
      #    database_operation => "create",
      #    database_rows => { "value1,REG_SZ,new value 1", "value2,REG_SZ,new val 2"} ,
      #    database_type     => "ms_registry";

      "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
      database_operation => "delete",
      database_columns => { "value1", "value2" } ,
      database_type => "ms_registry";

      # "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
      #    database_operation => "cache",   # cache,restore
      #    registry_exclude => { ".*Windows.*CurrentVersion.*", ".*Touchpad.*", ".*Capabilities.FileAssociations.*", ".*Rfc1766.*" , ".*Synaptics.SynTP.*", ".*SupportedDevices.*8086", ".*Microsoft.*ErrorThresholds" },
      #    database_type     => "ms_registry";

      "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS"
      database_operation => "restore",
      database_type      => "ms_registry";
}

File Permissions

ACL file example
body common control
{
      bundlesequence => { "acls" };
}

bundle agent acls
{
  files:
      "/media/flash/acl/test_dir"

      depth_search => include_base,
      acl => template;
}

body acl template
{
      acl_method => "overwrite";
      acl_type => "posix";
      acl_directory_inherit => "parent";
      aces => { "user:*:r(wwx),-r:allow", "group:*:+rw:allow", "mask:x:allow", "all:r"};
}

body acl win
{
      acl_method => "overwrite";
      acl_type => "ntfs";
      acl_directory_inherit => "nochange";
      aces => { "user:Administrator:rw", "group:Bad:rwx(Dpo):deny" };
}

body depth_search include_base
{
      include_basedir => "true";
}
ACL generic example
body common control
{
      bundlesequence => { "acls" };
}

bundle agent acls
{
  files:
      "/media/flash/acl/test_dir"

      depth_search => include_base,
      acl => test;
}

body acl test
{
      acl_type => "generic";
      aces => {"user:bob:rwx", "group:staff:rx", "all:r"};
}

body depth_search include_base
{
      include_basedir => "true";
}
ACL secret example
body common control
{
      bundlesequence => { "acls" };
}

bundle agent acls
{
  files:
    windows::
      "c:\Secret"
      acl => win,
      depth_search => include_base,
      comment => "Secure the secret directory from unauthorized access";
}

body acl win
{
      acl_method => "overwrite";
      aces => { "user:Administrator:rwx" };
}

body depth_search include_base
{
      include_basedir => "true";
}

User Management Examples

Local user management

There are many approaches to managing users. You can edit system files like /etc/passwd directly, you can use commands on some systems like useradd. However the easiest, and preferred way is to use CFEngine's native users type promise.

Ensuring a local user has a specific password

This example shows ensuring that the local users root is managed if there is a specific password hash defined.

body file control
{
  # This policy uses parts of the standard library.
  inputs => { "$(sys.libdir)/users.cf" };
}

bundle agent main
{
  vars:
      # This is the hashed password for 'vagrant'
    debian_8::
      "root_hash"
        string => "$6$1nRTeNoE$DpBSe.eDsuZaME0EydXBEf.DAwuzpSoIJhkhiIAPgRqVKlmI55EONfvjZorkxNQvK2VFfMm9txx93r2bma/4h/";

  users:
    linux::
      "root"
        policy => "present",
        password => hashed_password( $(root_hash) ),
        if => isvariable("root_hash");
}

This policy can be found in /var/cfengine/share/doc/examples/local_user_password.cf and downloaded directly from github.

root@debian-jessie:/core/examples# grep root /etc/shadow
root:!:16791:0:99999:7:::
root@debian-jessie:/core/examples# cf-agent -KIf ./local_user_password.cf
    info: User promise repaired
root@debian-jessie:/core/examples# grep root /etc/shadow
root:$6$1nRTeNoE$DpBSe.eDsuZaME0EydXBEf.DAwuzpSoIJhkhiIAPgRqVKlmI55EONfvjZorkxNQvK2VFfMm9txx93r2bma/4h/:16791:0:99999:7:::
Ensuring local users are present

This example shows ensuring that the local users jack and jill are present on all linux systems using the native users type promise.

body file control
{
  # This policy uses parts of the standard library.
  inputs => { "$(sys.libdir)/files.cf" };
}

bundle agent main
{
  vars:
    "users" slist => { "jack", "jill" };
    "skel" string => "/etc/skel";

  users:
    linux::
      "$(users)"
        home_dir => "/home/$(users)",
        policy => "present",
        home_bundle => home_skel( $(users), $(skel) );
}

bundle agent home_skel(user, skel)
{
  files:
    "/home/$(user)/."
      create => "true",
      copy_from => seed_cp( $(skel) ),
      depth_search => recurse( "inf" );
}

This policy can be found in /var/cfengine/share/doc/examples/local_users_present.cf and downloaded directly from github.

Lets check the environment to see that the users do not currently exist.

root@debian-jessie:/CFEngine/core/examples# egrep "jack|jill" /etc/passwd
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
ls: cannot access /home/jack: No such file or directory
ls: cannot access /home/jill: No such file or directory

Let's run the policy and inspect the state of the system afterwards.

root@debian-jessie:/core/examples# cf-agent -KIf ./users_present.cf
    info: Created directory '/home/jack/.'
    info: Copying from 'localhost:/etc/skel/.bashrc'
    info: Copying from 'localhost:/etc/skel/.profile'
    info: Copying from 'localhost:/etc/skel/.bash_logout'
    info: User promise repaired
    info: Created directory '/home/jill/.'
    info: Copying from 'localhost:/etc/skel/.bashrc'
    info: Copying from 'localhost:/etc/skel/.profile'
    info: Copying from 'localhost:/etc/skel/.bash_logout'
    info: User promise repaired
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/passwd
jack:x:1001:1001::/home/jack:/bin/sh
jill:x:1002:1002::/home/jill:/bin/sh
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
/home/jack:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root  220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root  675 Dec 22 16:37 .profile

/home/jill:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root  220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root  675 Dec 22 16:37 .profile
Ensuring local users are locked

This example shows ensuring that the local users jack and jill are locked if they are present on linux systems using the native users type promise.

bundle agent main
{
  vars:
    "users" slist => { "jack", "jill" };

  users:
    linux::
      "$(users)"
        policy => "locked";
}

This policy can be found in /var/cfengine/share/doc/examples/local_users_locked.cf and downloaded directly from github.

This output shows the state of the /etc/shadow file before running the example policy:

root@debian-jessie:/core/examples# egrep "jack|jill" /etc/shadow
jack:x:16791:0:99999:7:::
jill:x:16791:0:99999:7:::
root@debian-jessie:/core/examples# cf-agent -KIf ./local_users_locked.cf
    info: User promise repaired
    info: User promise repaired
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/shadow
jack:!x:16791:0:99999:7::1:
jill:!x:16791:0:99999:7::1:
Ensuring local users are absent

This example shows ensuring that the local users jack and jill are absent on linux systems using the native users type promise.

bundle agent main
{
  vars:
    "users" slist => { "jack", "jill" };

  users:
    linux::
      "$(users)"
        policy => "absent";
}

This policy can be found in /var/cfengine/share/doc/examples/local_users_absent.cf and downloaded directly from github.

Before activating the example policy, lets inspect the current state of the system.

root@debian-jessie:/core/examples# egrep "jack|jill" /etc/passwd
jack:x:1001:1001::/home/jack:/bin/sh
jill:x:1002:1002::/home/jill:/bin/sh
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
/home/jack:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root  220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root  675 Dec 22 16:37 .profile

/home/jill:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root  220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root  675 Dec 22 16:37 .profile

From the above output we can see that the local users jack and jill are present, and that they both have home directories.

Now lets activate the example policy and insepect the result.

root@debian-jessie:/core/examples# cf-agent -KIf ./local_users_absent.cf
    info: User promise repaired
    info: User promise repaired
root@debian-jessie:/core/examples# egrep "jack|jill" /etc/passwd
root@debian-jessie:/core/examples# ls -al /home/{jack,jill}
/home/jack:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root  220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root  675 Dec 22 16:37 .profile

/home/jill:
total 20
drwxr-xr-x 2 root root 4096 Dec 22 16:37 .
drwxr-xr-x 5 root root 4096 Dec 22 16:37 ..
-rw-r--r-- 1 root root  220 Dec 22 16:37 .bash_logout
-rw-r--r-- 1 root root 3515 Dec 22 16:37 .bashrc
-rw-r--r-- 1 root root  675 Dec 22 16:37 .profile

From the above output we can see that the local users jack and jill were removed from the system as desired. Note that their home directories remain, and if we wanted them to be purged we would have to have a separate promise to perform that cleanup.

Local group management

CFEngine does not currently have a native groups type promsie so you will need to either edit the necessary files using files type promises, or arrange for the proper commands to be run in order to create or delete groups.

Ensure a local group is present

Add lines to the password file, and users to group if they are not already there.

This example uses the native operating system commands to show ensuring that a group is present.

body file control
{
  # This policy uses parts of the standard library.
  inputs => { "$(sys.libdir)/paths.cf" };
}

bundle agent main
{
  classes:
      "group_cfengineers_absent"
        not => groupexists("cfengineers");

  commands:
    linux.group_cfengineers_absent::
      "$(paths.groupadd)"
        args => "cfengineers";
}

This policy can be found in /var/cfengine/share/doc/examples/local_group_present.cf and downloaded directly from github.

First lets inspect the current state of the system.

root@debian-jessie:/core/examples# grep cfengineers /etc/group

Now lets activate the example policy and check the resulting state of the system.

root@debian-jessie:/core/examples# cf-agent -KIf ./local_group_present.cf
    info: Executing 'no timeout' ... '/usr/sbin/groupadd cfengineers'
    info: Completed execution of '/usr/sbin/groupadd cfengineers'
root@debian-jessie:/CFEngine/core2.git/examples# grep cfengineers /etc/group
cfengineers:x:1001:
Ensureing a user is a member of a secondary group

This example shows using the native users type promise to ensure that a user is a member of a particular group.

bundle agent main
{
  users:
    linux::
      "jill"
        policy => "present",
        groups_secondary => { "cfengineers" };
}

This policy can be found in /var/cfengine/share/doc/examples/local_user_secondary_group_member.cf and downloaded directly from github.

First lets inspect the current state of the system

root@debian-jessie:/core/examples# grep jill /etc/passwd
root@debian-jessie:/core/examples# grep jill /etc/group

Now lets actiavte the example policy and inspect the resulting state.

root@debian-jessie:/core/examples# cf-agent -KIf ./local_user_secondary_group_member.cf
    info: User promise repaired
root@debian-jessie:/core/examples# grep jill /etc/passwd
jill:x:1001:1002::/home/jill:/bin/sh
root@debian-jessie:/core/examples# grep jill /etc/group
cfengineers:x:1001:jill
jill:x:1002:

It's important to remember we made no promise about the presence of the cfengineers group in the above example. We can see what would happen when the cfengineers group was not present.

root@debian-jessie:/core/examples# grep cfengineers /etc/group
root@debian-jessie:/core/examples# cf-agent -KIf ./local_user_secondary_group_member.cf
usermod: group 'cfengineers' does not exist
   error: Command returned error while modifying user 'jill'. (Command line: '/usr/sbin/usermod -G "cfengineers" jill')
    info: User promise not kept
Get a list of users
body common control
{
      bundlesequence  => { test };
}

bundle agent test
{
  vars:
      "allusers" slist => getusers("zenoss,mysql,at","12,0");
  reports:
    linux::
      "Found user $(allusers)";
}

Tutorials

Familarize yourself with CFEngine by following these step by step tutorials.

Additional tutorials to help you get started including screen casts can be found in the CFEngine Learning Center.


JSON and YAML Support in CFEngine

Introduction

JSON is a well-known data language. It even has a specification (See http://json.org).

YAML is another well-known data language. It has a longer, much more complex specification (See http://yaml.org).

CFEngine has core support for JSON and YAML. Let's see what it can do.

Problem statement

We'd like to read, access, and merge JSON-sourced data structures: they should be weakly typed, arbitrarily nested, with consistent quoting and syntax.

We'd like to read, access, and merge YAML-sourced data structures just like JSON-sourced, to keep policy and internals simple.

In addition, we must not break backward compatibility with CFEngine 3.5 and older, so we'd like to use the standard CFEngine array a[b] syntax.

Data containers

A new data type, the data container, was introduced in 3.6.

It's simply called data. The documentation with some examples is at https://cfengine.com/docs/master/reference-promise-types-vars.html#data-container-variables

Reading JSON

There are many ways to read JSON data; here are a few:

  • readjson(): read from a JSON file, e.g. "mydata" data => readjson("/my/file", 100k);
  • parsejson(): read from a JSON string, e.g. "mydata" data => parsejson('{ "x": "y" }');
  • data_readstringarray() and data_readstringarrayidx(): read text data from a file, split it on a delimiter, and make them into structured data.
  • mergedata(): merge data containers, slists, and classic CFEngine arrays, e.g. "mydata" data => mergedata(container1, slist2, array3);

mergedata in particular is very powerful. It can convert a slist or a classic CFEngine array to a data container easily: "mydata" data => mergedata(myslist);

Reading YAML

There are two ways to read YAML data:

  • readyaml(): read from a YAML file, e.g. "mydata" data => readyaml("/my/file.yaml", 100k);
  • parseyaml(): read from a YAML string, e.g. "mydata" data => parseyaml('- arrayentry1');

Since these functions return data containers, everything about JSON-sourced data structures applies to YAML-sourced data structures as well.

Accessing JSON

To access JSON data, you can use:

  • the nth() function to access an array element, e.g. "myx" string => nth(container1, 0);
  • the nth function to access a map element, e.g. "myx" string => nth(container1, "x");
  • the a[b] notation, e.g. "myx" string => "$(container1[x])";. You can nest, e.g. a[b][c][0][d]. This only works if the element is something that can be expanded in a string. So a number or a string work. A list of strings or numbers works. A key-value map under x won't work.
  • the getindices() and getvalues() functions, just like classic CFEngine arrays
A full example

This example can be saved and run. It will load a key-value map where the keys are class names and the values are hostname regular expressions or class names.

  • if your host name is c or b or the classes c or b are defined, the dev class will be defined
  • if your host name is flea or the class flea is defined, the prod class will be defined
  • if your host name is a or the class a is defined, the qa class will be defined
  • if your host name is linux or the class linux is defined, the private class will be defined

Easy, right?

body common control
{
      bundlesequence => { "run" };
}

bundle agent run
{
  vars:
      "bykey" data => parsejson('{ "dev": ["c", "b"], "prod": ["flea"], "qa": ["a"], "private": ["linux"] }');

      "keys" slist => getindices("bykey");

  classes:
      # define the class from the key name if any of the items under the key match the host name
      "$(keys)" expression => regcmp("$(bykey[$(keys)])", $(sys.host));

      # define the class from the key name if any of the items under the key are a defined class
      "$(keys)" expression => classmatch("$(bykey[$(keys)])");

  reports:
      "keys = $(keys)";
      "I am in class $(keys)" ifvarclass => $(keys);
}

So, where's the magic? Well, if you're familiar with classic CFEngine arrays, you will be happy to hear that the exact same syntax works with them. In other words, data containers don't change how you use CFEngine. You still use getindices to get the keys, then iterate through them and look up values.

Well, you can change

      "bykey" data => parsejson('{ "dev": ["c", "b"], "prod": ["flea"], "qa": ["a"], "private": ["linux"] }');

with

      "bykey" data => data_readstringarray(...);

and read the same container from a text file. The file should be formatted like this to produce the same data as above:

dev c b
prod flea
qa a
private linux

You can also use

      "bykey" data => readjson(...);

and read the same container from a JSON file.

Summary

Using JSON and YAML from CFEngine is easy and does not change how you use CFEngine. Try it out and see for yourself!


Distribute files from a central location

CFEngine can manage many machines simply by distributing policies to all its hosts. This tutorial describes how to distribute files to hosts from a central policy server location. For this example, we will distribute software patches.

Files are centrally stored on the policy server (hub). In our example, they are stored in /storage/patches. These patch files must also exist on the agent host (client) in /storage/deploy/patches. To do this, perform the following instructions:

Check out masterfiles from your central repository

CFEngine stores the master copy of all policy in the /var/cfengine/masterfiles directory. Ensure that you are working with the latest version of your masterfiles.

git clone url

or

git pull origin master
Make policy changes
Define locations

Before files can be copied we must know where files should be copied from and where files should be copied to. If these locations are used by multiple components, then defining them in a common bundle can reduce repetition. Variables and classes that are defined in common bundles are accessible by all CFEngine components. This is especially useful in the case of file copies because the same variable definition can be used both by the policy server when granting access and by the agent host when performing the copy.

The policy framework includes a common bundle called def. In this example, we will add two variables--dir_patch_store and dir_patch_deploy--to this existing bundle. These variables provide path definitions for storing and deploying patches.

Add the following variable information to the masterfiles/def.cf file:

"dir_patch_store"
  string => "/storage/patches",
  comment => "Define patch files source location",
  handle => "common_def_vars_dir_patch_store";

"dir_patch_deploy"
  string => "/storage/deploy/patches",
  comment => "Define patch files deploy location",
  handle => "common_def_vars_dir_patch_deploy";

}

These common variables can be referenced from the rest of the policy by using their fully qualified names, $(def.dir_patch_store) and $(def.dir_patch_deploy)

Grant file access

Access must be granted before files can be copied. The right to access a file is provided by cf-serverd, the server component of CFEngine. Enter access information using the access promise type in the bundle server access_rules section. This section is located in controls/cf_serverd.cf in the policy framework.

For our example, add the following information to controls/cf_serverd.cf:

"$(def.dir_patch_store)"
  handle => "server_access_grant_locations_files_patch_store_for_hosts",
  admit => { ".*$(def.domain)", @(def.acl) },
  comment => "Hosts need to download patch files from the central location";
Create a custom library for reusable synchronization policy

You might need to frequently synchronize or copy a directory structure from the policy server to an agent host. Thus, identifying reusable parts of policy and abstracting them for later use is a good idea. This information is stored in a custom library.

Create a custom library called lib/custom/files.cf. Add the following content:

    bundle agent sync_from_policyserver(source_path, dest_path)
    # @brief Sync files from the policy server to the agent
    #
    # @param source_path  Location on policy server to copy files from
    # @param dest_path Location on agent host to copy files to
    {
      files:
        "$(dest_path)/."
          handle       => "sync_from_policy_server_files_dest_path_copy_from_source_path_sys_policy_hub",
          copy_from    => sync_cp("$(source_path)", "$(sys.policy_hub)"),
          depth_search => recurse("inf"),
          comment      => "Ensure files from $(sys.policy_hub):$(source_path) exist in $(dest_path)";
    }

This reusable policy will be used to synchronize a directory on the policy server to a directory on the agent host.

Create a patch policy

Organize in a way that makes the most sense to you and your team. We recommend organizing policy by services.

Create services/patching.cf with the following content:

    # Patching Policy

    bundle agent patching
    # @brief Ensure various aspects of patching are handeled

    # We can break down the various parts of patching into separate bundles. This
    # allows us to become less overwhelmed by details if numerous specifics
    # exist in one or more aspect for different host classifications.
    {
      methods:

        "Patch Distribution"
          handle    => "patching_methods_patch_distribution",
          usebundle => "patch_distribution",
          comment   => "Ensure patches are properly distributed";
    }

    bundle agent patch_distribution
    # @brief Ensures that our patches are distributed to the proper locations
    {
      files:
        "$(def.dir_patch_deploy)/."
          handle  => "patch_distribution_files_def_dir_patch_deploy_exists",
          create  => "true",
          comment => "If the destination directory does not exist, we have no place
                      to which to copy the patches.";

      methods:

        "Patches"
          handle    => "patch_distribution_methods_patches_from_policyserver_def_dir_patch_store_to_def_dir_patch_deploy",
          usebundle => sync_from_policyserver("$(def.dir_patch_store)", "$(def.dir_patch_deploy)"),
          comment   => "Patches need to be present on host systems so that we can use
                       them. By convention we use the policy server as the central
                       distribution point.";
    }

The above policy contains two bundles. We have separated a top-level patching bundle from a more specific patch_distribution bundle. This is an illustration of how to use bundles in order to abstract details. You might, for example, have some hosts that you don’t want to fully synchronize so you might use a different method or copy from a different path. Creating numerous bundles allows you to move those details away from the top level of what is involved in patching. If people are interested in what is involved in patch distribution, they can view that bundle for specifics.

Integrate the policy

Now that all the pieces of the policy are in place, they must be integrated into the policy so they can be activated. Add each policy file to the inputs section which is found under body common control. Once the policy file is included in inputs, the bundle can be activated. Bundles can be activated by adding them to either the bundlesequence or they can be called as a methods type promise.

Add the following entries to promises.cf under body common control -> inputs:

"lib/custom/files.cf",
"services/patching.cf",

and the following to promises.cf under body common control -> bundlesequence:

"patching",

Now that all of the policy has been edited and is in place, check for syntax errors by running cf-promises -f ./promises.cf. This promise is activated from the service_catalogue bundle.

Commit Changes
Set up trackers in the Mission Portal (Enterprise Users Only)

Before committing the changes to your repository, log in to the Mission Portal and set up a Tracker so that you can see the policy as it goes out. To do this, perform the following:

Navigate to the Hosts section. Select All hosts. Select the Events tab, located in the right-hand panel. Click Add new tracker.

Mission Portal Host Event

Name it Patch Failure. Set the Report Type to Promise not Kept. Under Watch, enter .patch. Set the Start Time to Now and then click Done to close the Start Time window. Click Start to save the new tracker. This tracker watches for any promise handle that includes the string patch where a promise is not kept.

Add New Tracker

Add another tracker called Patch Repaired. Set the Report Type to Promise Repaired. Enter the same values as above for Watch and Start Time. Click Start to save the new tracker. This tracker allows you to see how the policy reacts as it is activated on your infrastructure.

Deploy changes (Enterprise and Community Users)

Always inspect what you expect. git status shows the status of your current branch.

git status

Inspect the changes contained in each file. Once satisfied, add them to git's commit staging area.

git diff file
git add file

Iterate over using git diff, add, and status until all of the changes that you expected are listed as Changes to be committed. Check the status once more before you commit the changes.

git status

Commit the changes to your local repository.

git commit

Push the changes to the central repository so they can be pulled down to your policy server for distribution.

git push origin master

Reporting and Remediation of Security Vulnerabilities

Prerequisites
  • CFEngine 3.6 Enterprise Hub
  • At least one client vulnerable to CVE-2014-6271
Overview

Remediating security vulnerabilities is a common issue. Sometimes you want to know the extent to which your estate is affected by a threat. Identification of affected systems can help you prioritize and plan remediation efforts. In this tutorial you will learn how to inventory your estate and build alerts to find hosts that are affected by the #shellshock exploit. After identifying the affected hosts you will patch a subset of the hosts and then be able to see the impact on your estate. The same methodology can be applied to other issues.

Note: The included policy does not require CFEngine Enterprise. Only the reporting functionality (Mission Portal) requires the Enterprise version.

Inventory CVE-2013-6271

Writing inventory policy with CFEngine is just like any other CFEngine policy, except for the addition of special meta attributes used to augment the inventory interface. First you must know how to collect the information you want. In this case we know that a vulnerable system will have the word vulnerable listed in the output of the command env x='() { :;}; echo vulnerable' $(bash) -c 'echo testing CVE-2014-6271'.

This bundle will check if the host is vulnerable to the CVE, define a class CVE_2014_6217 if it is vulnerable and augment Mission Portals Inventory interface in CFEngine Enterprise.

bundle agent inventory_CVE_2014_6271
{
  meta:
    "description" string => "Remote exploit vulnerability in bash http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-6271";
    "tags" slist => { "autorun" };

  vars:
    "env" string => "$(paths.env)";
    "bash" string => "/bin/bash";
    "echo" string => "$(paths.echo)";

    "test_result" string => execresult("$(env) x='() { :;}; $(echo) vulnerable' $(bash) -c 'echo testing CVE-2014-6271'", "useshell");

    CVE_2014_6271::
      "vulnerable"
        string => "CVE-2014-6271",
        meta => { "inventory", "attribute_name=Vulnerable CVE(s)" };

  classes:
    "CVE_2014_6271"
      expression => regcmp( "vulnerable.*", "$(test_result)" ),
      scope => "namespace",
      persistence => "10",
      comment => "We persist the class for 2 agent runs so that bundles
         activated before this bundle can use the class on the next
                 agent execution to coordinate things like package updates.";

  reports:
    DEBUG|DEBUG_cve_2014_6217::
      "Test Result: $(test_result)";

    CVE_2014_6271.(inform_mode|verbose_mode)::
      "Tested Vulnerable for CVE-2014-6271: $($(this.bundle)_meta.description)";
}
What does this inventory policy do?

Meta type promises are used to attach additional information to bundles. We have set 'description' so that future readers of the policy will know what the policy is for and how to get more information on the vulnerability. For the sake of simplicity in this example set 'autorun' as a tag to the bundle. This makes the bundle available for automatic activation when using the autorun feature in the Masterfiles Policy Framework.

Next we set the paths to the binaries that we will use to exeucte our test command. As of this writing the paths for 'env' and 'echo' are both in the standard libraries paths bundle, but 'bash' is not. Note that you may need to adjust the path to bash for your platforms. Then we run our test command and place the command output into the 'test_result' variable. Since we have no CVE_2014_6271 class defined yet, the next promise to set the variable 'vulnerable' to 'CVE-2014-6271' will be skipped on the first pass. Then the classes type promise is evaluated and defines the class CVE_2014_6271 if the output matches the regular expression 'vulnerable.*'. Finally the reports are evaluated before starting the second pass. If the class 'DEBUG' or 'DEBUG_inventory_CVE_2014_6271' is set the test command output will be shown, and if the vulnerability is present agent is running in inform or verbose mode message indicating the host is vulnerable along with the description will be output.

On the second pass only that variable 'vulnerable' will be set with the value 'CVE-2014-6271' if the host is vulnerable. Note how this variable tagged with 'inventory' and 'attribute_name='. These are special meta tags that CFEngine Enterprise uses in order to display information.

Deploy the policy

As noted previously, in this example we will use autorun for simplicity. Please ensure that the class "services_autorun" is defined. The easiest way to do this is to change "services_autorun" expression => "!any"; to "services_autorun" expression => "any"; in def.cf.

Once you have autorun enabled you need only save the policy into services/autorun/inventory_CVE_2014_6271.cf.

Report on affected system inventory

Within 20 minutes of deploying the policy you should be able to see results in the Inventory Reporting interface.

A new Inventory attribute 'Vulnerable CVE(s)' is available. A new Inventory attribute 'Vulnerable CVE(s)' is available

Report showing CVEs that each host is vulnerable to.

Report showing CVEs that each host is vulnerable to

Chart the Vulnerable CVE(s) and get a visual breakdown. Chart the Vulnerable CVE(s) and get a visual breakdown Chart the Vulnerable CVE(s) and get a visual breakdown - pie Chart the Vulnerable CVE(s) and get a visual breakdown - column

Build Dashboard Widget with Alerts

Let's add alerts for CVE(s) to the dashboard. Let's add alerts for CVE(s) to the dashboard

Give the dashboard widget a name. Give the dashboard widget a name

Configure an general CVE alert for the dashboard. Configure an general CVE alert for the dashboard

Add an additional alert for this specific CVE. Add an additional alert for this specific CVE

See the dashboard alert in action. See an the dashboard alert in action - visualization See an the dashboard alert in action - details See an the dashboard alert in action - alert details 1 See an the dashboard alert in action - specifc alert details

Remediate Vulnerabilities

Now that we know the extent of exposure lets ensure bash gets updated on some of the affected systems. Save the following policy into services/autorun/remediate_CVE_2014_6271.cf

bundle agent remediate_CVE_2014_6271
{
  meta:
    "tags" slist => { "autorun" };

  classes:
    "allow_update" or => { "hub", "host001" };

  methods:
    allow_update.CVE_2014_6271::
      "Upgrade_Bash"
        usebundle => package_latest("bash");
}
What does this remediation policy do?

For simplicity of the example this policy defines the class allow_update on hub and host001, but you could use any class that makes sense to you. If the allow_update class is set, and the class CVE_2014_6271 is defined (indicating the host is vulnerable) then the policy ensures that bash is updated to the latest version available.

Report on affected systems inventory after remediation

Within 20 minutes or so of the policy being deployed you will be able to report on the state of remediation.

See the remediation efforts relfected in the dashboard. See the remediation efforts relfected in the dashboard

Drill down into the dashboard and alert details. Drill down into the dashboard and alert details - widget alerts Drill down into the dashboard and alert details - alert detail

Run an Inventory report to see hosts and their CVE status. Run an Inventory report to see hosts and their CVE status

Chart the Vulnerable CVE(s) and get a visual breakdown. Chart the Vulnerable CVE(s) and get a visual breakdown - pie Chart the Vulnerable CVE(s) and get a visual breakdown - bar

Summary

In this tutorial you have learned how to use the reporting and inventory features of CFEngine Enterprise to discover and report on affected systems before and after remediation efforts.


Create, Modify, and Delete Files

Prerequisites
body common control {

    inputs => {
       "libraries/cfengine_stdlib.cf",
    };
}

Note: This change is not necessary for supporting each of the examples in this tutorial. It will be included only in those examples that require it.

List Files

Note: The following workflow assumes the directory /home/user already exists. If it does not either create the directory or adjust the example to a path of your choosing.

  1. Create a file /var/cfengine/masterfiles/file_test.cf that includes the following text:

    bundle agent list_file
    {
    
      vars:
          "ls" slist => lsdir("/home/user","test_plain.txt","true");
    
      reports:
          "ls: $(ls)";
    
    }
    
  2. Run the following command to remove any existing test file at the location we wish to use for testing this example:

    rm /home/user/test_plain.txt
    
  3. Test to ensure there is no file /home/user/test_plain.txt, using the following command (the expected result is that there should be no file listed at the location /home/user/test_plain.txt):

    ls /home/user/test_plain.txt
    
  4. Run the following command to instruct CFEngine to see if the file exists (the expected result is that no report will be generated (because the file does not exist):

    /var/cfengine/bin/cf-agent --no-lock --file /var/cfengine/masterfiles/file_test.cf --bundlesequence list_file
    
  5. Create a file for testing the example, using the following command:

    touch /home/user/test_plain.txt
    
  6. Run the following command to instruct CFEngine to search for the file (the expected result is that a report will be generated, because the file exists):

    /var/cfengine/bin/cf-agent --no-lock --file /var/cfengine/masterfiles/file_test.cf --bundlesequence list_file
    
  7. Double check the file exists, using the following command (the expected result is that there will be a file listed at the location /home/user/test_plain.txt):

    ls /home/user/test_plain.txt
    
  8. Run the following command to remove the file:

    rm /home/user/test_plain.txt
    
Create a File
bundle agent testbundle
{

  files:
      "/home/user/test_plain.txt"
      perms => system,
      create => "true";
}

bundle agent list_file
{

  vars:
      "ls" slist => lsdir("/home/user","test_plain.txt","true");

  reports:
      "ls: $(ls)";

}


bundle agent list_file_2
{

  vars:
      "ls" slist => lsdir("/home/user","test_plain.txt","true");

  reports:
      "ls: $(ls)";

}



body perms system
{
      mode  => "0640";
}

ls /home/user/test_plain.txt

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2

ls /home/user/test_plain.txt

rm /home/user/test_plain.txt

Delete a File
body common control {

    inputs => {
       "libraries/cfengine_stdlib.cf",
    };
}

bundle agent testbundle
{

  files:
      "/home/user/test_plain.txt"
      perms => system,
      create => "true";
}

bundle agent test_delete
{

  files:
      "/home/user/test_plain.txt"
      delete => tidy;
}


bundle agent list_file
{

  vars:
      "ls" slist => lsdir("/home/user","test_plain.txt","true");

  reports:
      "ls: $(ls)";

}


bundle agent list_file_2
{

  vars:
      "ls" slist => lsdir("/home/user","test_plain.txt","true");

  reports:
      "ls: $(ls)";

}



body perms system
{
      mode  => "0640";
}

rm /home/user/test_plain.txt

ls /home/user/test_plain.txt

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,test_delete,list_file_2

ls /home/user/test_plain.txt

rm /home/user/test_plain.txt

(last command will throw an error because the file doesn't exist!)

Modify a File

rm /home/user/test_plain.txt

ls /home/user/test_plain.txt

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2

body common control {

    inputs => {
       "libraries/cfengine_stdlib.cf",
    };
}

bundle agent testbundle
{

  files:
      "/home/user/test_plain.txt"
      perms => system,
      create => "true";
}

bundle agent test_delete
{

  files:
      "/home/user/test_plain.txt"
      delete => tidy;
}


bundle agent list_file
{

  vars:
      "ls" slist => lsdir("/home/user","test_plain.txt","true");

  reports:
      "ls: $(ls)";

}


bundle agent list_file_2
{

  vars:
      "ls" slist => lsdir("/home/user","test_plain.txt","true");

  reports:
      "ls: $(ls)";

}

# Finds the file, if exists calls bundle to edit line

bundle agent outer_bundle_1
{
    files:

       "/home/user/test_plain.txt"
       create    => "false",
       edit_line => inner_bundle_1;
}

# Finds the file, if exists calls bundle to edit line

bundle agent outer_bundle_2
{
    files:

       "/home/user/test_plain.txt"
       create    => "false",
       edit_line => inner_bundle_2;
}

# Inserts lines

bundle edit_line inner_bundle_1
{
  vars:

    "msg" string => "Helloz to World!";

  insert_lines:
    "$(msg)";

}

# Replaces lines

bundle edit_line inner_bundle_2
{
   replace_patterns:

   "Helloz to World!"
      replace_with => hello_world;

}

body replace_with hello_world
{
   replace_value => "Hello World";
   occurrences => "all";
}


body perms system
{
      mode  => "0640";
}

/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,test_delete,list_file_2

ls /home/user/test_plain.txt

rm /home/user/test_plain.txt

Copy a File and Edit its Text
body common control {

    inputs => {
       "libraries/cfengine_stdlib.cf",
    };
}

bundle agent testbundle
{

  files:
      "/home/ichien/test_plain.txt"
      perms => system,
      create => "true";

  reports:
    "test_plain.txt has been created";
}

bundle agent test_delete
{

  files:
      "/home/ichien/test_plain.txt"
      delete => tidy;
}

bundle agent do_files_exist

{
  vars:

      "mylist" slist => { "/home/ichien/test_plain.txt", "/home/ichien/test_plain_2.txt" };

  classes:

      "exists" expression => filesexist("@(mylist)");

  reports:

    exists::

      "test_plain.txt and test_plain_2.txt files exist";

    !exists::

      "test_plain.txt and test_plain_2.txt files do not exist";
}



bundle agent do_files_exist_2

{
  vars:

      "mylist" slist => { "/home/ichien/test_plain.txt", "/home/ichien/test_plain_2.txt" };

  classes:

      "exists" expression => filesexist("@(mylist)");

  reports:

    exists::

      "test_plain.txt and test_plain_2.txt files both exist";

    !exists::

      "test_plain.txt and test_plain_2.txt files do not exist";
}



bundle agent list_file_1
{

  vars:
      "ls1" slist => lsdir("/home/ichien","test_plain.txt","true");
      "ls2" slist => lsdir("/home/ichien","test_plain_2.txt","true");

      "file_content_1" string => readfile( "/home/ichien/test_plain.txt" , "33" );
      "file_content_2" string => readfile( "/home/ichien/test_plain_2.txt" , "33" );
  reports:
      #"ls1: $(ls1)";
      #"ls2: $(ls2)";

      "Contents of /home/ichien/test_plain.txt = $(file_content_1)";
      "Contents of /home/ichien/test_plain_2.txt = $(file_content_2)";
}


bundle agent list_file_2
{

  vars:
      "ls1" slist => lsdir("/home/ichien","test_plain.txt","true");
      "ls2" slist => lsdir("/home/ichien","test_plain_2.txt","true");
      "file_content_1" string => readfile( "/home/ichien/test_plain.txt" , "33" );
      "file_content_2" string => readfile( "/home/ichien/test_plain_2.txt" , "33" );

  reports:
      #"ls1: $(ls1)";
      #"ls2: $(ls2)";
      "Contents of /home/ichien/test_plain.txt = $(file_content_1)";
      "Contents of /home/ichien/test_plain_2.txt = $(file_content_2)";

}

bundle agent outer_bundle_1
{
    files:

       "/home/ichien/test_plain.txt"
       create    => "false",
       edit_line => inner_bundle_1;
}

# Copies file
bundle agent copy_a_file
{
  files:

      "/home/ichien/test_plain_2.txt"
      copy_from => local_cp("/home/ichien/test_plain.txt");

  reports:
     "test_plain.txt has been copied to test_plain_2.txt";
}

bundle agent outer_bundle_2
{
    files:

       "/home/ichien/test_plain_2.txt"
       create    => "false",
       edit_line => inner_bundle_2;
}


bundle edit_line inner_bundle_1
{
  vars:

    "msg" string => "Helloz to World!";

  insert_lines:
    "$(msg)";

  reports:
    "inserted $(msg) into test_plain.txt";

}

bundle edit_line inner_bundle_2
{
   replace_patterns:

   "Helloz to World!"
      replace_with => hello_world;

   reports:
      "Text in test_plain_2.txt has been replaced";

}

body replace_with hello_world
{
   replace_value => "Hello World";
   occurrences => "all";
}

body perms system
{
      mode  => "0640";
}
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence test_delete,do_files_exist,testbundle,outer_bundle_1,copy_a_file,do_files_exist_2,list_file_1,outer_bundle_2,list_file_2

Tags for variables, classes, and bundles

Introduction

meta tags can be attached to any promise type using the meta attribute. These tags are useful for cross-referencing related promises. bundles, vars and classes can be identified and leveraged in different ways within policy using these tags.

Problem statement

We'd like to apply tags to variables and classes for many purposes, from stating their provenance (whence they came, why they exist, and how they can be used) to filtering them based on tags.

We'd also like to be able to include all the files in a directory and then run all the discovered bundles if they are tagged appropriately.

Syntax

Tagging variables and classes is easy with the meta attribute. Here's an example that sets the inventory tag on a variable and names the attribute that it represents. This one is actually built into the standard MPF inventory policy, so it's available out of the box in either Community or Enterprise.

bundle agent cfe_autorun_inventory_listening_ports
{
  vars:
      "ports" -> { "ENT-150" }
        slist => sort( "mon.listening_ports", "int"),
        meta => { "inventory", "attribute_name=Ports listening" },
        ifvarclass => some("[0-9]+", "mon.listening_ports"),
        comment => "We only want to inventory the listening ports if we have
                    values that make sense.";
}

In the Enterprise Mission Portal, you can then make a report for "Ports listening" across all your machines. For more details, see Enterprise Reporting

Class tags work exactly the same way, you just apply them to a classes promise with the meta attribute.

Tagging bundles is different because you have to use the meta promise type (different from the meta attribute).

An example is easiest:

bundle agent run_deprecated
{
  meta:
      "tags" slist => { "deprecated" };
}

This declares an agent bundle with a single tag.

Functions

Several new functions exist to give you access to variable and class tags, and to find classes and variables with tags.

  • classesmatching: this used to be somewhat available with the allclasses.txt file. You can now call a function to get all the defined classes, optionally filtering by name and tags. See classesmatching

  • getvariablemetatags: get the tags of a variable as an slist. See getvariablemetatags

  • variablesmatching: just like classesmatching but for variables. See variablesmatching

  • variablesmatching_as_data: like variablesmatching but the matching variables and values are returned as a merged data container. See variablesmatching_as_data

  • getclassmetatags: get the tags of a class as an slist. See getclassmetatags

  • bundlesmatching: find the bundles matching some tags. See bundlesmatching (the example shows how you'd find a deprecated bundle like run_deprecated earlier).

Module protocol

The module protocol has been extended to support tags. You set the tags on a line and they persist for every subsequent variable or class.

^meta=inventory
+x
=a=100
^meta=report,attribute_name=My vars
+y
=n=100

This will create class x and variable a with tag inventory.

Then it will create class y and variable b with tags report and attribute_name=My vars.

Enterprise Reporting with tags

In CFEngine Enterprise, you can build reports based on tagged variables and classes.

Please see Enterprise Reporting for a full tutorial, including troubleshooting possible errors. In short, this is an extremely easy way to categorize various data accessible to the agent.

Dynamic bundlesequence

Dynamic bundlesequences are extremely easy. First you find all the bundles whos name matches a regular expression and N tags.

vars:
  "bundles" slist => bundlesmatching("regex", "tag1", "tag2", ...);

Then every bundle matching the regular expression regex and all the tags will be found and run.

methods:
  "run $(bundles)" usebundle => $(bundles);

Note that the discovered bundle names will have the namespace prefix, e.g. default:mybundle. The regular expression has to match that. So mybundle as the regular expression would not work. See bundlesmatching for another detailed example.

In fact we found this so useful we implemented services autorun in the masterfiles policy framework.

There is only one thing to beware. All the bundles have to have the same number of arguments (0 in the case shown). Otherwise you will get a runtime error and CFEngine will abort. We recommend only using 0-argument bundles in a dynamic sequence to reduce this risk.

Summary

Tagging variables and classes and bundles in CFEngine is easy and allows more dynamic behavior than ever before. Try it out and see for yourself how it will change the way you use and think about system configuration policy and CFEngine.


Masterfiles Policy Framework Upgrade

Introduction

Upgrading the Masterfiles Policy Framework (MPF) is an optional but highly recommended first step when upgrading CFEngine.

Upgrading the MPF is not an exact process as the details highly depend on the specifics of the changes made to the default policy. This tutorial leverages git and shows an example of upgrading a simple policy set based on 3.6.7 to 3.7.4 and can be used as a reference for upgrading your own policy sets.

Prepare a git clone of your working masterfiles

If you are not using git and instead editing directly in $(sys.workdir/masterfiles) you can simply copy your masterfiles into a new directory and initalize a new git repository.

If you're using git already simply clone your repository and skip to the next step.

[root@hub MPF_upgrade]# rsync -a /var/cfengine/masterfiles/ MPF_upgrade/

Then initialize the new git repository and add all the files to it.

[root@hub ~]# cd MPF_upgrade/
[root@hub MPF_upgrade]# git init
Initialized empty Git repository in /root/MPF_upgrade/.git/
[root@hub MPF_upgrade]# git add -A
[root@hub MPF_upgrade]# git commit -m "Before Upgrade"
[master (root-commit) 108c210] Before Upgrade
 78 files changed, 19980 insertions(+)
 create mode 100644 CUSTOM/policy1.cf
 create mode 100644 cf_promises_release_id
 create mode 100644 cf_promises_validated
 create mode 100644 cfe_internal/CFE_cfengine.cf
 create mode 100644 cfe_internal/CFE_hub_specific.cf
 create mode 100644 cfe_internal/CFE_knowledge.cf
 create mode 100644 cfe_internal/cfengine_processes.cf
 create mode 100644 cfe_internal/ha/ha.cf
 create mode 100644 cfe_internal/ha/ha_def.cf
 create mode 100644 cfe_internal/host_info_report.cf
 create mode 100644 controls/3.4/cf_serverd.cf
 create mode 100644 controls/cf_agent.cf
 create mode 100644 controls/cf_execd.cf
 create mode 100644 controls/cf_hub.cf
 create mode 100644 controls/cf_monitord.cf
 create mode 100644 controls/cf_runagent.cf
 create mode 100644 controls/cf_serverd.cf
 create mode 100644 def.cf
 create mode 100644 inventory/any.cf
 create mode 100644 inventory/debian.cf
 create mode 100644 inventory/generic.cf
 create mode 100644 inventory/linux.cf
 create mode 100644 inventory/lsb.cf
 create mode 100644 inventory/macos.cf
 create mode 100644 inventory/os.cf
 create mode 100644 inventory/redhat.cf
 create mode 100644 inventory/suse.cf
 create mode 100644 inventory/windows.cf
 create mode 100644 lib/3.5/bundles.cf
 create mode 100644 lib/3.5/cfe_internal.cf
 create mode 100644 lib/3.5/commands.cf
 create mode 100644 lib/3.5/common.cf
 create mode 100644 lib/3.5/databases.cf
 create mode 100644 lib/3.5/feature.cf
 create mode 100644 lib/3.5/files.cf
 create mode 100644 lib/3.5/guest_environments.cf
 create mode 100644 lib/3.5/monitor.cf
 create mode 100644 lib/3.5/packages.cf
 create mode 100644 lib/3.5/paths.cf
 create mode 100644 lib/3.5/processes.cf
 create mode 100644 lib/3.5/reports.cf
 create mode 100644 lib/3.5/services.cf
 create mode 100644 lib/3.5/storage.cf
 create mode 100644 lib/3.6/bundles.cf
 create mode 100644 lib/3.6/cfe_internal.cf
 create mode 100644 lib/3.6/cfengine_enterprise_hub_ha.cf
 create mode 100644 lib/3.6/commands.cf
 create mode 100644 lib/3.6/common.cf
 create mode 100644 lib/3.6/databases.cf
 create mode 100644 lib/3.6/edit_xml.cf
 create mode 100644 lib/3.6/examples.cf
 create mode 100644 lib/3.6/feature.cf
 create mode 100644 lib/3.6/files.cf
 create mode 100644 lib/3.6/guest_environments.cf
 create mode 100644 lib/3.6/monitor.cf
 create mode 100644 lib/3.6/packages.cf
 create mode 100644 lib/3.6/paths.cf
 create mode 100644 lib/3.6/processes.cf
 create mode 100644 lib/3.6/reports.cf
 create mode 100644 lib/3.6/services.cf
 create mode 100644 lib/3.6/stdlib.cf
 create mode 100644 lib/3.6/storage.cf
 create mode 100644 lib/3.6/users.cf
 create mode 100644 lib/3.6/vcs.cf
 create mode 100644 promises.cf
 create mode 100644 services/autorun.cf
 create mode 100644 services/autorun/custom_policy2.cf
 create mode 100644 services/autorun/hello.cf
 create mode 100644 services/file_change.cf
 create mode 100644 sketches/meta/api-runfile.cf
 create mode 100644 templates/host_info_report.mustache
 create mode 100644 update.cf
 create mode 100644 update/cfe_internal_dc_workflow.cf
 create mode 100644 update/cfe_internal_local_git_remote.cf
 create mode 100644 update/cfe_internal_update_from_repository.cf
 create mode 100644 update/update_bins.cf
 create mode 100644 update/update_policy.cf
 create mode 100644 update/update_processes.cf
[root@hub MPF_upgrade]# git status
# On branch master
nothing to commit, working directory clean

Now we have a git repository that we can start merging in the changes from upstream.

Merge the upstream changes to the MPF into your policy
Remove everything except the .git directory.

By first removing everything we will easily be able so see which files are new, changed, moved or removed upstream.

[root@hub MPF_upgrade]# rm -rf *


[root@hub MPF_upgrade]# git status
On branch master
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    deleted:    CUSTOM/policy1.cf
    deleted:    cf_promises_release_id
    deleted:    cf_promises_validated
    deleted:    cfe_internal/CFE_cfengine.cf
    deleted:    cfe_internal/CFE_hub_specific.cf
    deleted:    cfe_internal/CFE_knowledge.cf
    deleted:    cfe_internal/cfengine_processes.cf
    deleted:    cfe_internal/ha/ha.cf
    deleted:    cfe_internal/ha/ha_def.cf
    deleted:    cfe_internal/host_info_report.cf
    deleted:    controls/3.4/cf_serverd.cf
    deleted:    controls/cf_agent.cf
    deleted:    controls/cf_execd.cf
    deleted:    controls/cf_hub.cf
    deleted:    controls/cf_monitord.cf
    deleted:    controls/cf_runagent.cf
    deleted:    controls/cf_serverd.cf
    deleted:    def.cf
    deleted:    inventory/any.cf
    deleted:    inventory/debian.cf
    deleted:    inventory/generic.cf
    deleted:    inventory/linux.cf
    deleted:    inventory/lsb.cf
    deleted:    inventory/macos.cf
    deleted:    inventory/os.cf
    deleted:    inventory/redhat.cf
    deleted:    inventory/suse.cf
    deleted:    inventory/windows.cf
    deleted:    lib/3.5/bundles.cf
    deleted:    lib/3.5/cfe_internal.cf
    deleted:    lib/3.5/commands.cf
    deleted:    lib/3.5/common.cf
    deleted:    lib/3.5/databases.cf
    deleted:    lib/3.5/feature.cf
    deleted:    lib/3.5/files.cf
    deleted:    lib/3.5/guest_environments.cf
    deleted:    lib/3.5/monitor.cf
    deleted:    lib/3.5/packages.cf
    deleted:    lib/3.5/paths.cf
    deleted:    lib/3.5/processes.cf
    deleted:    lib/3.5/reports.cf
    deleted:    lib/3.5/services.cf
    deleted:    lib/3.5/storage.cf
    deleted:    lib/3.6/bundles.cf
    deleted:    lib/3.6/cfe_internal.cf
    deleted:    lib/3.6/cfengine_enterprise_hub_ha.cf
    deleted:    lib/3.6/commands.cf
    deleted:    lib/3.6/common.cf
    deleted:    lib/3.6/databases.cf
    deleted:    lib/3.6/edit_xml.cf
    deleted:    lib/3.6/examples.cf
    deleted:    lib/3.6/feature.cf
    deleted:    lib/3.6/files.cf
    deleted:    lib/3.6/guest_environments.cf
    deleted:    lib/3.6/monitor.cf
    deleted:    lib/3.6/packages.cf
    deleted:    lib/3.6/paths.cf
    deleted:    lib/3.6/processes.cf
    deleted:    lib/3.6/reports.cf
    deleted:    lib/3.6/services.cf
    deleted:    lib/3.6/stdlib.cf
    deleted:    lib/3.6/storage.cf
    deleted:    lib/3.6/users.cf
    deleted:    lib/3.6/vcs.cf
    deleted:    promises.cf
    deleted:    services/autorun.cf
    deleted:    services/autorun/custom_policy2.cf
    deleted:    services/autorun/hello.cf
    deleted:    services/file_change.cf
    deleted:    sketches/meta/api-runfile.cf
    deleted:    templates/host_info_report.mustache
    deleted:    update.cf
    deleted:    update/cfe_internal_dc_workflow.cf
    deleted:    update/cfe_internal_local_git_remote.cf
    deleted:    update/cfe_internal_update_from_repository.cf
    deleted:    update/update_bins.cf
    deleted:    update/update_policy.cf
    deleted:    update/update_processes.cf

no changes added to commit (use "git add" and/or "git commit -a")
Install the new MPF

The MPF can be obtained from any community package (in $(sys.workdir)/share/CoreBase/), enterprise hub package (in $(sys.workdir)/share/NovaBase/), masterfiles source tarball (requires ./configure and make install ), installed masterfiles tarball (ready for extraction), or directly from github.

We will install the MPF from source obtained directly from github.

Note: You will need automake to install from source.

First clone the masterfiles repository for the version you are installing. And verify you have the correct tag checked out.

Note: Directly checking out a tag as in the example below is only supported in git versions 1.7.9.5 and newer.

[root@hub MPF_upgrade]# cd ..
[root@hub ~]# git clone -b 3.7.4 https://github.com/cfengine/masterfiles
[root@hub ~]# cd masterfiles
[root@hub ~]# git describe
3.7.4

Now we will install the masterfiles from upstream into the directory where we are doing the integration.

First we build and install masterfiles to a temporary location.

./autogen.sh
[root@hub masterfiles]# ./autogen.sh
configure.ac:31: installing `./config.guess'
configure.ac:31: installing `./config.sub'
configure.ac:34: installing `./install-sh'
configure.ac:34: installing `./missing'
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking whether to disable maintainer-specific portions of Makefiles... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for a BSD-compatible install... /usr/bin/install -c

Summary of options:
Core directory       -> not set - tests are disabled
Enterprise directory -> not set - some tests are disabled
Install prefix       -> /var/cfengine

configure: generating makefile targets
configure: creating ./config.status
config.status: creating Makefile
config.status: creating controls/3.5/update_def.cf
config.status: creating controls/3.6/update_def.cf
config.status: creating controls/3.7/update_def.cf
config.status: creating modules/packages/Makefile
config.status: creating promises.cf
config.status: creating tests/acceptance/Makefile
config.status: creating tests/unit/Makefile

DONE: Configuration done. Run "make install" to install CFEngine Masterfiles.
[root@hub masterfiles]# ./configure --prefix /tmp/masterfiles-3.7.4
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking whether to disable maintainer-specific portions of Makefiles... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for a BSD-compatible install... /usr/bin/install -c

Summary of options:
Core directory       -> not set - tests are disabled
Enterprise directory -> not set - some tests are disabled
Install prefix       -> /tmp/masterfiles-3.7.4

configure: generating makefile targets
configure: creating ./config.status
config.status: creating Makefile
config.status: creating controls/3.5/update_def.cf
config.status: creating controls/3.6/update_def.cf
config.status: creating controls/3.7/update_def.cf
config.status: creating modules/packages/Makefile
config.status: creating promises.cf
config.status: creating tests/acceptance/Makefile
config.status: creating tests/unit/Makefile

DONE: Configuration done. Run "make install" to install CFEngine Masterfiles.

Then after running make install we move the installed masterfiles into our integration directory.

[root@hub masterfiles]# mv /tmp/masterfiles-3.7.4/masterfiles/* ../MPF_upgrade
[root@hub masterfiles]# cd ../MPF_upgrade/

Merge differences

Now we can use git status to see an overview of the changes to the repository between our starting point and the new MPF.

[root@hub MPF_upgrade]# git status
On branch master
Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    deleted:    CUSTOM/policy1.cf
    deleted:    cf_promises_release_id
    deleted:    cf_promises_validated
    modified:   cfe_internal/CFE_cfengine.cf
    deleted:    cfe_internal/CFE_hub_specific.cf
    deleted:    cfe_internal/CFE_knowledge.cf
    deleted:    cfe_internal/cfengine_processes.cf
    deleted:    cfe_internal/ha/ha.cf
    deleted:    cfe_internal/ha/ha_def.cf
    deleted:    cfe_internal/host_info_report.cf
    deleted:    controls/3.4/cf_serverd.cf
    deleted:    controls/cf_agent.cf
    deleted:    controls/cf_execd.cf
    deleted:    controls/cf_hub.cf
    deleted:    controls/cf_monitord.cf
    deleted:    controls/cf_runagent.cf
    deleted:    controls/cf_serverd.cf
    deleted:    def.cf
    modified:   inventory/any.cf
    modified:   inventory/linux.cf
    modified:   inventory/lsb.cf
    modified:   lib/3.5/cfe_internal.cf
    modified:   lib/3.5/common.cf
    modified:   lib/3.5/files.cf
    modified:   lib/3.5/packages.cf
    deleted:    lib/3.5/reports.cf
    modified:   lib/3.6/cfe_internal.cf
    modified:   lib/3.6/common.cf
    modified:   lib/3.6/files.cf
    modified:   lib/3.6/packages.cf
    deleted:    lib/3.6/reports.cf
    modified:   lib/3.6/services.cf
    modified:   lib/3.6/stdlib.cf
    modified:   promises.cf
    deleted:    services/autorun.cf
    deleted:    services/autorun/custom_policy2.cf
    deleted:    services/file_change.cf
    modified:   sketches/meta/api-runfile.cf
    modified:   update.cf
    deleted:    update/cfe_internal_dc_workflow.cf
    deleted:    update/cfe_internal_local_git_remote.cf
    deleted:    update/cfe_internal_update_from_repository.cf
    deleted:    update/update_bins.cf
    deleted:    update/update_policy.cf
    deleted:    update/update_processes.cf

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    cfe_internal/core/
    cfe_internal/enterprise/
    cfe_internal/update/
    controls/3.5/
    controls/3.6/
    controls/3.7/
    inventory/freebsd.cf
    lib/3.6/autorun.cf
    lib/3.6/cfe_internal_hub.cf
    lib/3.7/
    services/main.cf

no changes added to commit (use "git add" and/or "git commit -a")

All of the Untracked files are new additions from upstream so they should be safe to take.

[root@hub MPF_upgrade]# git add cfe_internal/core/ \
cfe_internal/enterprise/ \
cfe_internal/update/ \
controls/3.5/ \
controls/3.6/ \
controls/3.7/ \
inventory/freebsd.cf \
lib/3.6/autorun.cf \
lib/3.6/cfe_internal_hub.cf \
lib/3.7/ \
services/main.cf

We can run git status again to see the current overview:

[root@hub MPF_upgrade]# git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    new file:   cfe_internal/core/deprecated/cfengine_processes.cf
    new file:   cfe_internal/core/host_info_report.cf
    new file:   cfe_internal/core/limit_robot_agents.cf
    new file:   cfe_internal/core/log_rotation.cf
    new file:   cfe_internal/core/main.cf
    new file:   cfe_internal/enterprise/CFE_hub_specific.cf
    new file:   cfe_internal/enterprise/CFE_knowledge.cf
    new file:   cfe_internal/enterprise/file_change.cf
    new file:   cfe_internal/enterprise/ha/ha.cf
    new file:   cfe_internal/enterprise/ha/ha_def.cf
    new file:   cfe_internal/enterprise/ha/ha_update.cf
    new file:   cfe_internal/enterprise/main.cf
    new file:   cfe_internal/update/cfe_internal_dc_workflow.cf
    new file:   cfe_internal/update/cfe_internal_local_git_remote.cf
    new file:   cfe_internal/update/cfe_internal_update_from_repository.cf
    new file:   cfe_internal/update/update_bins.cf
    new file:   cfe_internal/update/update_policy.cf
    new file:   cfe_internal/update/update_processes.cf
    new file:   controls/3.5/cf_agent.cf
    new file:   controls/3.5/cf_execd.cf
    new file:   controls/3.5/cf_hub.cf
    new file:   controls/3.5/cf_monitord.cf
    new file:   controls/3.5/cf_runagent.cf
    new file:   controls/3.5/cf_serverd.cf
    new file:   controls/3.5/def.cf
    new file:   controls/3.5/def_inputs.cf
    new file:   controls/3.5/reports.cf
    new file:   controls/3.5/update_def.cf
    new file:   controls/3.5/update_def_inputs.cf
    new file:   controls/3.6/cf_agent.cf
    new file:   controls/3.6/cf_execd.cf
    new file:   controls/3.6/cf_hub.cf
    new file:   controls/3.6/cf_monitord.cf
    new file:   controls/3.6/cf_runagent.cf
    new file:   controls/3.6/cf_serverd.cf
    new file:   controls/3.6/def.cf
    new file:   controls/3.6/def_inputs.cf
    new file:   controls/3.6/reports.cf
    new file:   controls/3.6/update_def.cf
    new file:   controls/3.6/update_def_inputs.cf
    new file:   controls/3.7/cf_agent.cf
    new file:   controls/3.7/cf_execd.cf
    new file:   controls/3.7/cf_hub.cf
    new file:   controls/3.7/cf_monitord.cf
    new file:   controls/3.7/cf_runagent.cf
    new file:   controls/3.7/cf_serverd.cf
    new file:   controls/3.7/def.cf
    new file:   controls/3.7/def_inputs.cf
    new file:   controls/3.7/reports.cf
    new file:   controls/3.7/update_def.cf
    new file:   controls/3.7/update_def_inputs.cf
    new file:   inventory/freebsd.cf
    new file:   lib/3.6/autorun.cf
    new file:   lib/3.6/cfe_internal_hub.cf
    new file:   lib/3.7/autorun.cf
    new file:   lib/3.7/bundles.cf
    new file:   lib/3.7/cfe_internal.cf
    new file:   lib/3.7/cfe_internal_hub.cf
    new file:   lib/3.7/cfengine_enterprise_hub_ha.cf
    new file:   lib/3.7/commands.cf
    new file:   lib/3.7/common.cf
    new file:   lib/3.7/databases.cf
    new file:   lib/3.7/edit_xml.cf
    new file:   lib/3.7/examples.cf
    new file:   lib/3.7/feature.cf
    new file:   lib/3.7/files.cf
    new file:   lib/3.7/guest_environments.cf
    new file:   lib/3.7/monitor.cf
    new file:   lib/3.7/packages.cf
    new file:   lib/3.7/paths.cf
    new file:   lib/3.7/processes.cf
    new file:   lib/3.7/services.cf
    new file:   lib/3.7/stdlib.cf
    new file:   lib/3.7/storage.cf
    new file:   lib/3.7/users.cf
    new file:   lib/3.7/vcs.cf
    new file:   services/main.cf

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

    deleted:    CUSTOM/policy1.cf
    deleted:    cf_promises_release_id
    deleted:    cf_promises_validated
    modified:   cfe_internal/CFE_cfengine.cf
    deleted:    cfe_internal/CFE_hub_specific.cf
    deleted:    cfe_internal/CFE_knowledge.cf
    deleted:    cfe_internal/cfengine_processes.cf
    deleted:    cfe_internal/ha/ha.cf
    deleted:    cfe_internal/ha/ha_def.cf
    deleted:    cfe_internal/host_info_report.cf
    deleted:    controls/3.4/cf_serverd.cf
    deleted:    controls/cf_agent.cf
    deleted:    controls/cf_execd.cf
    deleted:    controls/cf_hub.cf
    deleted:    controls/cf_monitord.cf
    deleted:    controls/cf_runagent.cf
    deleted:    controls/cf_serverd.cf
    deleted:    def.cf
    modified:   inventory/any.cf
    modified:   inventory/linux.cf
    modified:   inventory/lsb.cf
    modified:   lib/3.5/cfe_internal.cf
    modified:   lib/3.5/common.cf
    modified:   lib/3.5/files.cf
    modified:   lib/3.5/packages.cf
    deleted:    lib/3.5/reports.cf
    modified:   lib/3.6/cfe_internal.cf
    modified:   lib/3.6/common.cf
    modified:   lib/3.6/files.cf
    modified:   lib/3.6/packages.cf
    deleted:    lib/3.6/reports.cf
    modified:   lib/3.6/services.cf
    modified:   lib/3.6/stdlib.cf
    modified:   promises.cf
    deleted:    services/autorun.cf
    deleted:    services/autorun/custom_policy2.cf
    deleted:    services/file_change.cf
    modified:   sketches/meta/api-runfile.cf
    modified:   update.cf
    deleted:    update/cfe_internal_dc_workflow.cf
    deleted:    update/cfe_internal_local_git_remote.cf
    deleted:    update/cfe_internal_update_from_repository.cf
    deleted:    update/update_bins.cf
    deleted:    update/update_policy.cf
    deleted:    update/update_processes.cf

Next we want to bring back any of our custom policy files. Keeping your polices organized together helps to make this process easy. The custom policy files in the example policy set are CUSTOM/policy1.cf and services/autorun/custom_policy2.cf. Restore them with git checkout.

[root@hub MPF_upgrade] git checkout CUSTOM/policy1.cf services/autorun/custom_policy2.cf

The files marked as modified in the git status output are files that have changed upstream.

[root@hub MPF_upgrade]# git status | grep modified
    modified:   cfe_internal/CFE_cfengine.cf
    modified:   inventory/any.cf
    modified:   inventory/linux.cf
    modified:   inventory/lsb.cf
    modified:   lib/3.5/cfe_internal.cf
    modified:   lib/3.5/common.cf
    modified:   lib/3.5/files.cf
    modified:   lib/3.5/packages.cf
    modified:   lib/3.6/cfe_internal.cf
    modified:   lib/3.6/common.cf
    modified:   lib/3.6/files.cf
    modified:   lib/3.6/packages.cf
    modified:   lib/3.6/services.cf
    modified:   lib/3.6/stdlib.cf
    modified:   promises.cf
    modified:   sketches/meta/api-runfile.cf
    modified:   update.cf

For any files that you have not modified (like those in lib) simply add them to gits staging area with git add. Carefully review and merge or re-integrate your custom changes on top of the upstream files.

The remaining files in git status marked as deleted are files that have been moved or removed from upstream.

NOTE: It is uncommon for any files to be moved or deleted between patch releases (e.g. 3.7.1 -> 3.7.2).

[root@hub MPF_upgrade]# git status | grep deleted
    deleted:    cf_promises_release_id
    deleted:    cf_promises_validated
    deleted:    cfe_internal/CFE_hub_specific.cf
    deleted:    cfe_internal/CFE_knowledge.cf
    deleted:    cfe_internal/cfengine_processes.cf
    deleted:    cfe_internal/ha/ha.cf
    deleted:    cfe_internal/ha/ha_def.cf
    deleted:    cfe_internal/host_info_report.cf
    deleted:    controls/3.4/cf_serverd.cf
    deleted:    controls/cf_agent.cf
    deleted:    controls/cf_execd.cf
    deleted:    controls/cf_hub.cf
    deleted:    controls/cf_monitord.cf
    deleted:    controls/cf_runagent.cf
    deleted:    controls/cf_serverd.cf
    deleted:    def.cf
    deleted:    lib/3.5/reports.cf
    deleted:    lib/3.6/reports.cf
    deleted:    services/autorun.cf
    deleted:    services/file_change.cf
    deleted:    update/cfe_internal_dc_workflow.cf
    deleted:    update/cfe_internal_local_git_remote.cf
    deleted:    update/cfe_internal_update_from_repository.cf
    deleted:    update/update_bins.cf
    deleted:    update/update_policy.cf
    deleted:    update/update_processes.cf

It's a good idea to review these files as some of them might have contained modifications, especially def.cf and any files under controls. Always keep track of the modifications you make to any of the files that ship with the MPF. Make sure that any necessary customization's to the deleted files are carried through to their new locations.

Once the files are no longer needed you can git rm them.

[root@hub MPF_upgrade]# git rm def.cf cf_promises_release_id cf_promises_validated cfe_internal/CFE_hub_specific.cf cfe_internal/CFE_knowledge.cf cfe_internal/cfengine_processes.cf cfe_internal/ha/ha.cf cfe_internal/ha/ha_def.cf cfe_internal/host_info_report.cf controls/3.4/cf_serverd.cf controls/cf_agent.cf controls/cf_execd.cf controls/cf_hub.cf controls/cf_monitord.cf controls/cf_runagent.cf controls/cf_serverd.cf lib/3.5/reports.cf lib/3.6/reports.cf services/autorun.cf services/file_change.cf update/cfe_internal_dc_workflow.cf update/cfe_internal_local_git_remote.cf update/cfe_internal_update_from_repository.cf update/update_bins.cf update/update_policy.cf update/update_processes.cf
rm 'def.cf'
rm 'cf_promises_release_id'
rm 'cf_promises_validated'
rm 'cfe_internal/CFE_hub_specific.cf'
rm 'cfe_internal/CFE_knowledge.cf'
rm 'cfe_internal/cfengine_processes.cf'
rm 'cfe_internal/ha/ha.cf'
rm 'cfe_internal/ha/ha_def.cf'
rm 'cfe_internal/host_info_report.cf'
rm 'controls/3.4/cf_serverd.cf'
rm 'controls/cf_agent.cf'
rm 'controls/cf_execd.cf'
rm 'controls/cf_hub.cf'
rm 'controls/cf_monitord.cf'
rm 'controls/cf_runagent.cf'
rm 'controls/cf_serverd.cf'
rm 'lib/3.5/reports.cf'
rm 'lib/3.6/reports.cf'
rm 'services/autorun.cf'
rm 'services/file_change.cf'
rm 'update/cfe_internal_dc_workflow.cf'
rm 'update/cfe_internal_local_git_remote.cf'
rm 'update/cfe_internal_update_from_repository.cf'
rm 'update/update_bins.cf'
rm 'update/update_policy.cf'
rm 'update/update_processes.cf'

Review git status and make sure that the policy validates then commit your changes.

[root@hub MPF_upgrade]# git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    deleted:    cf_promises_release_id
    deleted:    cf_promises_validated
    modified:   cfe_internal/CFE_cfengine.cf
    renamed:    cfe_internal/cfengine_processes.cf -> cfe_internal/core/deprecated/cfengine_processes.cf
    renamed:    cfe_internal/host_info_report.cf -> cfe_internal/core/host_info_report.cf
    new file:   cfe_internal/core/limit_robot_agents.cf
    new file:   cfe_internal/core/log_rotation.cf
    new file:   cfe_internal/core/main.cf
    renamed:    cfe_internal/CFE_hub_specific.cf -> cfe_internal/enterprise/CFE_hub_specific.cf
    renamed:    cfe_internal/CFE_knowledge.cf -> cfe_internal/enterprise/CFE_knowledge.cf
    renamed:    services/file_change.cf -> cfe_internal/enterprise/file_change.cf
    new file:   cfe_internal/enterprise/ha/ha.cf
    renamed:    cfe_internal/ha/ha_def.cf -> cfe_internal/enterprise/ha/ha_def.cf
    new file:   cfe_internal/enterprise/ha/ha_update.cf
    new file:   cfe_internal/enterprise/main.cf
    deleted:    cfe_internal/ha/ha.cf
    renamed:    update/cfe_internal_dc_workflow.cf -> cfe_internal/update/cfe_internal_dc_workflow.cf
    renamed:    update/cfe_internal_local_git_remote.cf -> cfe_internal/update/cfe_internal_local_git_remote.cf
    new file:   cfe_internal/update/cfe_internal_update_from_repository.cf
    renamed:    update/update_bins.cf -> cfe_internal/update/update_bins.cf
    renamed:    update/update_policy.cf -> cfe_internal/update/update_policy.cf
    renamed:    update/update_processes.cf -> cfe_internal/update/update_processes.cf
    deleted:    controls/3.4/cf_serverd.cf
    renamed:    controls/cf_agent.cf -> controls/3.5/cf_agent.cf
    new file:   controls/3.5/cf_execd.cf
    renamed:    controls/cf_hub.cf -> controls/3.5/cf_hub.cf
    renamed:    controls/cf_monitord.cf -> controls/3.5/cf_monitord.cf
    renamed:    controls/cf_runagent.cf -> controls/3.5/cf_runagent.cf
    renamed:    controls/cf_serverd.cf -> controls/3.5/cf_serverd.cf
    renamed:    def.cf -> controls/3.5/def.cf
    new file:   controls/3.5/def_inputs.cf
    renamed:    lib/3.5/reports.cf -> controls/3.5/reports.cf
    renamed:    update.cf -> controls/3.5/update_def.cf
    new file:   controls/3.5/update_def_inputs.cf
    new file:   controls/3.6/cf_agent.cf
    new file:   controls/3.6/cf_execd.cf
    new file:   controls/3.6/cf_hub.cf
    new file:   controls/3.6/cf_monitord.cf
    new file:   controls/3.6/cf_runagent.cf
    new file:   controls/3.6/cf_serverd.cf
    new file:   controls/3.6/def.cf
    new file:   controls/3.6/def_inputs.cf
    renamed:    lib/3.6/reports.cf -> controls/3.6/reports.cf
    new file:   controls/3.6/update_def.cf
    new file:   controls/3.6/update_def_inputs.cf
    new file:   controls/3.7/cf_agent.cf
    new file:   controls/3.7/cf_execd.cf
    new file:   controls/3.7/cf_hub.cf
    new file:   controls/3.7/cf_monitord.cf
    new file:   controls/3.7/cf_runagent.cf
    new file:   controls/3.7/cf_serverd.cf
    new file:   controls/3.7/def.cf
    new file:   controls/3.7/def_inputs.cf
    new file:   controls/3.7/reports.cf
    new file:   controls/3.7/update_def.cf
    new file:   controls/3.7/update_def_inputs.cf
    deleted:    controls/cf_execd.cf
    modified:   inventory/any.cf
    new file:   inventory/freebsd.cf
    modified:   inventory/linux.cf
    modified:   inventory/lsb.cf
    modified:   lib/3.5/cfe_internal.cf
    modified:   lib/3.5/common.cf
    modified:   lib/3.5/files.cf
    modified:   lib/3.5/packages.cf
    renamed:    services/autorun.cf -> lib/3.6/autorun.cf
    modified:   lib/3.6/cfe_internal.cf
    renamed:    lib/3.6/cfe_internal.cf -> lib/3.6/cfe_internal_hub.cf
    modified:   lib/3.6/common.cf
    modified:   lib/3.6/files.cf
    modified:   lib/3.6/packages.cf
    modified:   lib/3.6/services.cf
    modified:   lib/3.6/stdlib.cf
    new file:   lib/3.7/autorun.cf
    new file:   lib/3.7/bundles.cf
    new file:   lib/3.7/cfe_internal.cf
    new file:   lib/3.7/cfe_internal_hub.cf
    new file:   lib/3.7/cfengine_enterprise_hub_ha.cf
    new file:   lib/3.7/commands.cf
    new file:   lib/3.7/common.cf
    new file:   lib/3.7/databases.cf
    new file:   lib/3.7/edit_xml.cf
    new file:   lib/3.7/examples.cf
    new file:   lib/3.7/feature.cf
    new file:   lib/3.7/files.cf
    new file:   lib/3.7/guest_environments.cf
    new file:   lib/3.7/monitor.cf
    new file:   lib/3.7/packages.cf
    new file:   lib/3.7/paths.cf
    new file:   lib/3.7/processes.cf
    new file:   lib/3.7/services.cf
    new file:   lib/3.7/stdlib.cf
    new file:   lib/3.7/storage.cf
    new file:   lib/3.7/users.cf
    new file:   lib/3.7/vcs.cf
    modified:   promises.cf
    new file:   services/main.cf
    modified:   sketches/meta/api-runfile.cf
    modified:   update.cf
    deleted:    update/cfe_internal_update_from_repository.cf

[root@hub MPF_upgrade]# cf-promises -cf ./promises.cf
[root@hub MPF_upgrade]# cf-promises -cf ./update.cf
[root@hub MPF_upgrade]# git commit -m "After Policy Upgrade"
 100 files changed, 12521 insertions(+), 1493 deletions(-)
 delete mode 100644 cf_promises_release_id
 delete mode 100644 cf_promises_validated
 rewrite cfe_internal/CFE_cfengine.cf (88%)
 rename cfe_internal/{ => core/deprecated}/cfengine_processes.cf (95%)
 rename cfe_internal/{ => core}/host_info_report.cf (98%)
 create mode 100644 cfe_internal/core/limit_robot_agents.cf
 create mode 100644 cfe_internal/core/log_rotation.cf
 create mode 100644 cfe_internal/core/main.cf
 rename cfe_internal/{ => enterprise}/CFE_hub_specific.cf (85%)
 rename cfe_internal/{ => enterprise}/CFE_knowledge.cf (100%)
 rename {services => cfe_internal/enterprise}/file_change.cf (58%)
 create mode 100644 cfe_internal/enterprise/ha/ha.cf
 rename cfe_internal/{ => enterprise}/ha/ha_def.cf (54%)
 create mode 100644 cfe_internal/enterprise/ha/ha_update.cf
 create mode 100644 cfe_internal/enterprise/main.cf
 delete mode 100644 cfe_internal/ha/ha.cf
 rename {update => cfe_internal/update}/cfe_internal_dc_workflow.cf (100%)
 rename {update => cfe_internal/update}/cfe_internal_local_git_remote.cf (100%)
 create mode 100644 cfe_internal/update/cfe_internal_update_from_repository.cf
 rename {update => cfe_internal/update}/update_bins.cf (97%)
 rename {update => cfe_internal/update}/update_policy.cf (92%)
 rename {update => cfe_internal/update}/update_processes.cf (92%)
 delete mode 100644 controls/3.4/cf_serverd.cf
 rename controls/{ => 3.5}/cf_agent.cf (80%)
 create mode 100644 controls/3.5/cf_execd.cf
 rename controls/{ => 3.5}/cf_hub.cf (100%)
 rename controls/{ => 3.5}/cf_monitord.cf (100%)
 rename controls/{ => 3.5}/cf_runagent.cf (100%)
 rename controls/{ => 3.5}/cf_serverd.cf (87%)
 rename def.cf => controls/3.5/def.cf (74%)
 create mode 100644 controls/3.5/def_inputs.cf
 rename {lib => controls}/3.5/reports.cf (80%)
 rename update.cf => controls/3.5/update_def.cf (59%)
 create mode 100644 controls/3.5/update_def_inputs.cf
 create mode 100644 controls/3.6/cf_agent.cf
 create mode 100644 controls/3.6/cf_execd.cf
 create mode 100644 controls/3.6/cf_hub.cf
 create mode 100644 controls/3.6/cf_monitord.cf
 create mode 100644 controls/3.6/cf_runagent.cf
 create mode 100644 controls/3.6/cf_serverd.cf
 create mode 100644 controls/3.6/def.cf
 create mode 100644 controls/3.6/def_inputs.cf
 rename {lib => controls}/3.6/reports.cf (78%)
 create mode 100644 controls/3.6/update_def.cf
 create mode 100644 controls/3.6/update_def_inputs.cf
 create mode 100644 controls/3.7/cf_agent.cf
 create mode 100644 controls/3.7/cf_execd.cf
 create mode 100644 controls/3.7/cf_hub.cf
 create mode 100644 controls/3.7/cf_monitord.cf
 create mode 100644 controls/3.7/cf_runagent.cf
 create mode 100644 controls/3.7/cf_serverd.cf
 create mode 100644 controls/3.7/def.cf
 create mode 100644 controls/3.7/def_inputs.cf
 create mode 100644 controls/3.7/reports.cf
 create mode 100644 controls/3.7/update_def.cf
 create mode 100644 controls/3.7/update_def_inputs.cf
 delete mode 100644 controls/cf_execd.cf
 create mode 100644 inventory/freebsd.cf
 rename {services => lib/3.6}/autorun.cf (50%)
 rewrite lib/3.6/cfe_internal.cf (67%)
 rename lib/3.6/{cfe_internal.cf => cfe_internal_hub.cf} (77%)
 create mode 100644 lib/3.7/autorun.cf
 create mode 100644 lib/3.7/bundles.cf
 create mode 100644 lib/3.7/cfe_internal.cf
 create mode 100644 lib/3.7/cfe_internal_hub.cf
 create mode 100644 lib/3.7/cfengine_enterprise_hub_ha.cf
 create mode 100644 lib/3.7/commands.cf
 create mode 100644 lib/3.7/common.cf
 create mode 100644 lib/3.7/databases.cf
 create mode 100644 lib/3.7/edit_xml.cf
 create mode 100644 lib/3.7/examples.cf
 create mode 100644 lib/3.7/feature.cf
 create mode 100644 lib/3.7/files.cf
 create mode 100644 lib/3.7/guest_environments.cf
 create mode 100644 lib/3.7/monitor.cf
 create mode 100644 lib/3.7/packages.cf
 create mode 100644 lib/3.7/paths.cf
 create mode 100644 lib/3.7/processes.cf
 create mode 100644 lib/3.7/services.cf
 create mode 100644 lib/3.7/stdlib.cf
 create mode 100644 lib/3.7/storage.cf
 create mode 100644 lib/3.7/users.cf
 create mode 100644 lib/3.7/vcs.cf
 create mode 100644 services/main.cf
 rewrite update.cf (78%)
 delete mode 100644 update/cfe_internal_update_from_repository.cf

Now your Masterfiles Policy Framework is upgraded and ready to be tested.


Custom Inventory

This tutorial will show you how to add custom inventory attributes that can be leveraged in policy and reported on in the CFEngine Enterprise Mission Portal. For a more detailed overview on how the inventory system works please reference CFEngine 3 inventory modules.

Overview

This tutorial provides instructions for the following:

Choose an Attribute to Inventory

Writing inventory policy is incredibly easy. Simply add the inventory and attribute_name= tags to any variable or namespace scoped classes.

In this tutorial we will add Owner information into the inventory. In this example we will use a simple shared flat file data source /vagrant/inventory_owner.csv.

On your hosts create /vagrant/inventory_owner.csv with the following contnet:

hub, Operations Team <ops@cfengine.com>
host001, Development <dev@cfengine.com>

Note: I am using the [CFEngine Enterprise Vagrant Environment][Installing Enterprise Vagrant Environemt] and files located in the vagrant project directory are automatically available to all hosts.

Create and Deploy Inventory Policy

Now that each of your hosts has access to a data source that provides the Owner information we will write an inventory policy to report that information.

Create /var/cfengine/masterfiles/services/tutorials/inventory/owner.cf with the following content:

bundle common tutorials_inventory_owner
# @brief Inventory Owner information
# @description Inventory owner information from `/vagrant/inventory_owner.csv`.
{
  classes:
    "have_owner"
      expression => isvariable("my_owner");

  vars:
    "data_source" string => "/vagrant/inventory_owner.csv";
    "owners" data => data_readstringarray( $(data_source), "", ", ", 100, 512 );

    "my_owner"
      string  => "$(owners[$(sys.uqhost)][0])",
      meta    => { "inventory", "attribute_name=Owner" },
      comment => "We need to tag the owner information so that it is correctly
                  reported via the UI.";

  reports:
    inform_mode.have_owner::
      "$(this.bundle): Discovered Owner='$(my_owner)'";
}

Note: For the simplicity of this tutorial we assume that masterfiles is not configured for policy updates from a git repository. If it is, please add the policy to your repository and ensure it gets to its final destination as needed.

This policy will not be activated until it has been included in inputs. For simplicity we will be adding it to common control, but note that files can include other files through the use of inputs in file control.

Add 'services/tutorials/inventory/owner.cf' to inputs and 'tutorials_inventory_owner' to the bundlesequence in common control found in /var/cfengine/masterfiles/promises.cf as shown below.

body common control
{
    bundlesequence =>  {
                         "inventory_control",
                         @(inventory.bundles),
                         tutorials_inventory_owner,
                         def,
                         cfe_internal_hub_vars,
# snipped for brevity
                         service_catalogue,
    };

    inputs => {
               # File definition for global variables and classes
                "def.cf",

              # Inventory policy
                @(inventory.inputs),
                "services/tutorials/inventory/owner.cf",
# snipped for brevity
                # List of services here
                  "services/init_msg.cf",
                  "services/file_change.cf",

      };
}

When you finish its a good idea to run cf-promises (syntax analyzer) to ensure the policy validates. You can also perform a manual policy run and check that the correct owner is discovered.

Syntax Check:

[root@hub ~]# cf-promises -f /var/cfengine/masterfiles/promises.cf
[root@hub ~]#

No output and return code 0 indicate the policy was successfully validated.

Manual Policy Run:

[root@hub ~]# cf-agent -KIf /var/cfengine/masterfiles/promises.cf -b tutorials_inventory_owner
2014-06-16T19:24:58+0000     info: Using command line specified bundlesequence
R: tutorials_inventory_owner: Discovered Owner='Operations Team <ops@cfengine.com>'

Here I have run the policy without locks (-K) in inform mode (-I), using a specific policy entry (-f) and activating only a specific bundle (-b). The inform output helps me confirm that my hubs owner is discovered from our CSV properly.

Run Reports

Once you have integrated the policy into promises.cf it will be distributed and run by all agents. Once the hub has had a chance to collect reports the 'Owner' attribute will be available to select as a Table column for Inventory Reports. Custom attributes appear under the 'User defined' section.

Note: It may take up to 15 minutes for your custom inventory attributes to be collected and made available for reporting.

custom inventory attribute

You will see the host owner as shown in the following report.

custom inventory report


File Comparison

  1. Add the policy contents (also can be downloaded from file_compare_test.cf) to a new file, such as /var/cfengine/masterfiles/file_test.cf.
  2. Run the following commands as root on the command line:

    export AOUT_BIN="a.out"
    export GCC_BIN="/usr/bin/gcc"
    export RM_BIN="/bin/rm"
    export WORK_DIR=$HOME
    export CFE_FILE1="test_plain_1.txt"
    export CFE_FILE2="test_plain_2.txt"
    
    /var/cfengine/bin/cf-agent /var/cfengine/masterfiles/file_test.cf --bundlesequence robot,global_vars,packages,create_aout_source_file,create_aout,test_delete,do_files_exist_1,create_file_1,outer_bundle_1,copy_a_file,do_files_exist_2,list_file_1,stat,outer_bundle_2,list_file_2
    

Here is the order in which bundles are called in the command line above (some other support bundles are contained within file_test.cf but are not included here):

  1. robot - demonstrates use of reports.
  2. global_vars - sets up some global variables for later use.
  3. packages - installs packages that will be used later on.
  4. create_aout_source_file - creates a source file.
  5. create_aout - compiles the source file.
  6. test_delete - deletes a file.
  7. do_files_exist_1 - checks the existence of files.
  8. create_file_1 - creates a file.
  9. outer_bundle_1 - adds text to a file.
  10. copy_a_file - copies the file.
  11. do_files_exist_2 - checks the existence of both files.
  12. list_file_1 - shows the contents of each file.
  13. stat - uses the stat command and the aout application to compare modified times of both files.
  14. outer_bundle_2 - modifies the contents of the second file.
  15. list_file_2 - shows the contents of both files and uses CFEngine functionality to compare the modified time for each file.
robot

Demonstrates use of reports, using an ascii art representation of the CFEngine robot.

global_vars

Sets up some global variables that are used frequently by other bundles.

bundle common global_vars
{
    vars:

      "gccexec" string => getenv("GCC_BIN",255);
      "rmexec" string => getenv("RM_BIN",255);

      "aoutbin" string => getenv("AOUT_BIN",255);
      "workdir" string => getenv("WORK_DIR",255);

      "aoutexec" string => "$(workdir)/$(aoutbin)";

      "file1name" string => getenv("CFE_FILE1",255);
      "file2name" string => getenv("CFE_FILE2",255);

      "file1" string => "$(workdir)/$(file1name)";
      "file2" string => "$(workdir)/$(file2name)";

    classes:
      "gclass" expression => "any";

}
packages

Ensures that the gcc package is installed, for later use by the create_aout bundle.

    bundle agent packages
    {
      vars:

          "match_package" slist => {
            "gcc"
          };

      packages:
          "$(match_package)"
          package_policy => "add",
          package_method => yum;

      reports:

        gclass::
            "Package gcc installed";
            "*********************************";

    }
create_aout_source_file

Creates the c source file that will generate a binary application in create_aout.

bundle agent create_aout_source_file
{

  # This bundle creates the source file that will be compiled in bundle agent create_aout.
  # See that bunlde's comments for more information.

  vars:

    # An slist is used here instead of a straight forward string because it doesn't seem possible to create
    # line endings using \n when using a string to insert text into a file.

    "c" slist => {"#include <stdlib.h>","#include <stdio.h>","#include <sys/stat.h>","#include <string.h>","void main()","{char file1[255];strcpy(file1,\"$(global_vars.file1)\");char file2[255];strcpy(file2,\"$(global_vars.file2)\");struct stat time1;int i = lstat(file1, &time1);struct stat time2;int j = lstat(file2, &time2);if (time1.st_mtime < time2.st_mtime){printf(\"Newer\");}else{if(time1.st_mtim.tv_nsec < time2.st_mtim.tv_nsec){printf(\"Newer\");}else{printf(\"Not newer\");}}}"};

  files:
      "$(global_vars.workdir)/a.c"
      perms => system,
      create => "true",
      edit_line => Insert("@(c)");

  reports:
    "The source file $(global_vars.workdir)/a.c has been created. It will be used to compile the binary a.out, which will provide more accurate file stats to compare two files than the built in CFEngine functionality for comparing file stats, including modification time. This information will be used to determine of the second of the two files being compared is newer or not.";
    "*********************************";

}
create_aout

This bundle creates a binary application from the source in create_aout_source_file that uses the stat library to compare two files, determine if the modified times are different, nd whether the second file is newer than the first.

The difference between this application and using CFEngine's built in support for getting file stats is that normally the accuracy is only to the second of the modified file time but in order to better compare two files requires parts of a second as well. The stat library provides the extra support for retrieving the additional information required.

bundle agent create_aout
{

    classes:

    "doesfileacexist" expression => fileexists("$(global_vars.workdir)/a.c");
    "doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");

  vars:

    # Removes any previous binary
    "rmaout" string => execresult("$(global_vars.rmexec) $(global_vars.aoutexec)","noshell");

    doesfileacexist::
    "compilestr" string => "$(global_vars.gccexec) $(global_vars.workdir)/a.c -o $(global_vars.aoutexec)";
    "gccaout" string => execresult("$(compilestr)","noshell");

  reports:
    doesfileacexist::
      "gcc output: $(gccaout)";
      "Creating aout using $(compilestr)";
    !doesfileacexist::
      "Cannot compile a.out, $(global_vars.workdir)/a.c does not exist.";   
    doesaoutexist::
      "The binary application aout has been compiled from the source in the create_aout_source_file bundle. It uses the stat library to compare two files, determine if the modified times are different, and whether the second file is newer than the first. The difference between this application and using CFEngine's built in support for getting file stats (e.g. filestat, isnewerthan), which provides file modification time accurate to a second. However, in order to better compare two files might sometimes require parts of a second as well. The stat library provides the extra support for retrieving the additional information required to get better accuracy (down to parts of a second), and is utilized by the binary application a.out that is compiled within the create_aout bundle.";
      "*********************************";

}
test_delete

Deletes any previous copy of the test files used in the example.

bundle agent test_delete
{

  files:
      "$(global_vars.file1)"
      delete => tidy;
}
do_files_exist_1

Verifies whether the test files exist or not.

bundle agent do_files_exist_1

{

  classes:

    "doesfile1exist" expression => fileexists("$(global_vars.file1)");
    "doesfile2exist" expression => fileexists("$(global_vars.file2)");

  methods:

    doesfile1exist::

    "any" usebundle => delete_file("$(global_vars.file1)"); 
    doesfile2exist::
    "any" usebundle => delete_file("$(global_vars.file2)"); 
  reports:

    !doesfile1exist::
      "$(global_vars.file1) does not exist.";
    doesfile1exist::
      "$(global_vars.file1) did exist. Call to delete it was made.";    

    !doesfile2exist::
      "$(global_vars.file2) does not exist.";
    doesfile2exist::
      "$(global_vars.file2) did exist. Call to delete it was made.";    

}
create_file_1

Creates the first test file, as an empty file.

bundle agent create_file_1
{

  files:
      "$(global_vars.file1)"
      perms => system,
      create => "true";

  reports:
    "$(global_vars.file1) has been created";
}
outer_bundle_1

Adds some text to the first test file.

bundle agent outer_bundle_1
{
    files:

       "$(global_vars.file1)"
       create    => "false",
       edit_line => inner_bundle_1;
}
copy_a_file

Makes a copy of the test file.

bundle agent copy_a_file
{
  files:

      "$(global_vars.file2)"
      copy_from => local_cp("$(global_vars.file1)");

  reports:
     "$(global_vars.file1) has been copied to $(global_vars.file2)";
}
do_files_exist_2

Verifies that both test files exist.

bundle agent do_files_exist_2
{
    methods:

    "any" usebundle => does_file_exist($(global_vars.file1));
    "any" usebundle => does_file_exist($(global_vars.file2));
}
list_file_1

Reports the contents of each test file.

bundle agent list_file_1
{

  methods:  
    "any" usebundle => file_content($(global_vars.file1));
    "any" usebundle => file_content($(global_vars.file2));
  reports:
    "*********************************";

}
exec_aout
bundle agent exec_aout
{

  classes:
    "doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");

  vars:
    doesaoutexist::
    "aout" string => execresult("$(global_vars.aoutexec)","noshell");

  reports:
    doesaoutexist::
    "*********************************";
    "$(global_vars.aoutbin) determined that $(global_vars.file2) is $(aout) than $(global_vars.file1)";
    "*********************************";
    !doesaoutexist::
    "Executable $(global_vars.aoutbin) does not exist.";

}
stat

Compares the modified time of each test file using the binary application compiled in create_aout to see if it is newer.

bundle agent stat
{

  classes:

    "doesfile1exist" expression => fileexists("$(global_vars.file1)");
    "doesfile2exist" expression => fileexists("$(global_vars.file2)");

  vars:

    doesfile1exist::

    "file1" string => "$(global_vars.file1)";
    "file2" string => "$(global_vars.file2)";

    "file1_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file1)","noshell");
    "file1_split1" slist => string_split($(file1_stat)," ",3);
    "file1_split2" string => nth("file1_split1",1);
    "file1_split3" slist => string_split($(file1_split2),"\.",3);
    "file1_split4" string => nth("file1_split3",1);

    "file2_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file2)","noshell");
    "file2_split1" slist => string_split($(file2_stat)," ",3);
    "file2_split2" string => nth("file2_split1",1);
    "file2_split3" slist => string_split($(file2_split2),"\.",3);
    "file2_split4" string => nth("file2_split3",1);

  methods:

      "any" usebundle => exec_aout();

  reports:
    doesfile1exist::
      "Parts of a second extracted extracted from stat for $(file1): $(file1_split4). Full stat output for $(file1): $(file1_stat)";
      "Parts of a second extracted extracted from stat for $(file2): $(file2_split4). Full stat output for $(file2): $(file2_stat)";
      "Using the binary Linux application stat to compare two files can help determine if the modified times between two files are different. The difference between the stat application using its additional flags and using CFEngine's built in support for getting and comparing file stats (e.g. filestat, isnewerthan) is that normally the accuracy is only to the second of the file's modified time. In order to better compare two files requires parts of a second as well, which the stat command can provide with some additional flags. Unfortunately the information must be extracted from the middle of a string, which is what the stat bundle accomplishes using the string_split and nth functions.";
      "*********************************";
    !doesfile1exist::
      "stat: $(global_vars.file1) and probably $(global_vars.file2) do not exist.";

}
outer_bundle_2

Modifies the text in the second file.

bundle agent outer_bundle_2
{
    files:

       "$(global_vars.file2)"
       create    => "false",
       edit_line => inner_bundle_2;
}
list_file_2

Uses filestat and isnewerthan to compare the two test files to see if the second one is newer. Sometimes the modifications already performed, such as copy and modifying text, happen too quickly and filestat and isnewerthan may both report that the second test file is not newer than the first, while the more accurate stat based checks in the stat bundle (see step 12) will recognize the difference.

bundle agent list_file_2
{

  methods:

      "any" usebundle => file_content($(global_vars.file1));
      "any" usebundle => file_content($(global_vars.file2));    

  classes:

      "ok" expression => isgreaterthan(filestat("$(global_vars.file2)","mtime"),filestat("$(global_vars.file1)","mtime"));
      "newer" expression => isnewerthan("$(global_vars.file2)","$(global_vars.file1)");

  reports:
    "*********************************";
      ok::
         "Using isgreaterthan+filestat determined that $(global_vars.file2) was modified later than $(global_vars.file1).";

      !ok::
         "Using isgreaterthan+filestat determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
      newer::
         "Using isnewerthan determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
      !newer::
         "Using isnewerthan determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";

}
Full Policy
body common control {

    inputs => {
       "libraries/cfengine_stdlib.cf",
    };

}

bundle agent robot
{
  reports:
"                                    77777777777";
"                                   77777777777777";
"                                    777 7777 777";
"                                  7777777777777";
"                                   777777777777";
"                                    777 7777 77";
"                                    ";
"                           ZZZZ     ZZZ ZZZZ ZZZ     ZZZZ";
"                          ZZZZZ    ZZZZZZZZZZZZZZ    ZZZZZ ";
"                        ZZZZZZZ    ZZZZZZZZZZZZZ     ZZZZZZZ";
"                         ZZZZ      -------------      ZZZZZZ";
"                       ZZZZZ        !CFENGINE!        ZZZZZ";
"                         ZZZZ      -------------      ZZZZZ";
"                       ZZZZZ       ZZZZZZZZZZZZZZ       ZZZZZ";
"                        ZZZ        ZZZZZZZZZZZZZ         ZZZ";
"                       ZZZZZ       ZZZZZZZZZZZZZ        ZZZZZ";
"                     ..?ZZZ+,,,,,  ZZZZZZZZZZZZZZ       ZZZZZ";
"                    ...ZZZZ~  ,::  ZZZZZZZZZZZZZ         ZZZZ";
"                    ..,ZZZZZ,::::::                     ZZZZZ";
"                        ZZZ                              ZZZ";
"                                  ~       ===+";
"                                    ZZZZZZZZZZZZZI??";
"                                    ZZZZZZZZZZZZZ$???";
"                                    7Z$+ ZZ  ZZZ???II";
"                                    ZZZZZ+   ZZZZZIIII";
"                                 ZZZZZ    ZZZZZ III77";
"                              +++  +$ZZ???   ZZZ";
"                              +++??ZZZZZIIIIZZZZZ";
"                               ????ZZZZZIIIIZZZZZ";
"                                ??IIZZZZ 7777ZZZ";
"                                 IIZZZZZ  77ZZZZZ";
"                                  I$ZZZZ   $ZZZZ";


}

bundle common global_vars
{
    vars:

      "gccexec" string => getenv("GCC_BIN",255);
      "rmexec" string => getenv("RM_BIN",255);

      "aoutbin" string => getenv("AOUT_BIN",255);
      "workdir" string => getenv("WORK_DIR",255);

      "aoutexec" string => "$(workdir)/$(aoutbin)";

      "file1name" string => getenv("CFE_FILE1",255);
      "file2name" string => getenv("CFE_FILE2",255);

      "file1" string => "$(workdir)/$(file1name)";
      "file2" string => "$(workdir)/$(file2name)";

    classes:
      "gclass" expression => "any";

}


bundle agent packages
{
  vars:

      "match_package" slist => {
        "gcc"
      };

  packages:
      "$(match_package)"
      package_policy => "add",
      package_method => yum;

  reports:

    gclass::
        "Package gcc installed";
        "*********************************";

}

bundle agent create_aout_source_file
{

  # This bundle creates the source file that will be compiled in bundle agent create_aout.
  # See that bunlde's comments for more information.

  vars:

    # An slist is used here instead of a straight forward string because it doesn't seem possible to create
    # line endings using \n when using a string to insert text into a file.

    "c" slist => {"#include <stdlib.h>","#include <stdio.h>","#include <sys/stat.h>","#include <string.h>","void main()","{char file1[255];strcpy(file1,\"$(global_vars.file1)\");char file2[255];strcpy(file2,\"$(global_vars.file2)\");struct stat time1;int i = lstat(file1, &time1);struct stat time2;int j = lstat(file2, &time2);if (time1.st_mtime < time2.st_mtime){printf(\"Newer\");}else{if(time1.st_mtim.tv_nsec < time2.st_mtim.tv_nsec){printf(\"Newer\");}else{printf(\"Not newer\");}}}"};

  files:
      "$(global_vars.workdir)/a.c"
      perms => system,
      create => "true",
      edit_line => Insert("@(c)");

  reports:
    "The source file $(global_vars.workdir)/a.c has been created. It will be used to compile the binary a.out, which will provide more accurate file stats to compare two files than the built in CFEngine functionality for comparing file stats, including modification time. This information will be used to determine of the second of the two files being compared is newer or not.";
    "*********************************";

}

bundle edit_line Insert(name)
{
   insert_lines:
      "$(name)";
}

bundle agent create_aout
{

    classes:

    "doesfileacexist" expression => fileexists("$(global_vars.workdir)/a.c");
    "doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");

  vars:

    # Removes any previous binary
    "rmaout" string => execresult("$(global_vars.rmexec) $(global_vars.aoutexec)","noshell");

    doesfileacexist::
    "compilestr" string => "$(global_vars.gccexec) $(global_vars.workdir)/a.c -o $(global_vars.aoutexec)";
    "gccaout" string => execresult("$(compilestr)","noshell");

  reports:
    doesfileacexist::
      "gcc output: $(gccaout)";
      "Creating aout using $(compilestr)";
    !doesfileacexist::
      "Cannot compile a.out, $(global_vars.workdir)/a.c does not exist.";   
    doesaoutexist::
      "The binary application aout has been compiled from the source in the create_aout_source_file bundle. It uses the stat library to compare two files, determine if the modified times are different, and whether the second file is newer than the first. The difference between this application and using CFEngine's built in support for getting file stats (e.g. filestat, isnewerthan), which provides file modification time accurate to a second. However, in order to better compare two files might sometimes require parts of a second as well. The stat library provides the extra support for retrieving the additional information required to get better accuracy (down to parts of a second), and is utilized by the binary application a.out that is compiled within the create_aout bundle.";
      "*********************************";

}


bundle agent test_delete
{

  files:
      "$(global_vars.file1)"
      delete => tidy;
}

bundle agent delete_file(fname)
{

  files:
      "$(fname)"
      delete => tidy;
  reports:
    "Deleted $(fname)";
}

body contain del_file
{

  useshell => "useshell";

}

bundle agent do_files_exist_1

{

  classes:

    "doesfile1exist" expression => fileexists("$(global_vars.file1)");
    "doesfile2exist" expression => fileexists("$(global_vars.file2)");

  methods:

    doesfile1exist::

    "any" usebundle => delete_file("$(global_vars.file1)"); 
    doesfile2exist::
    "any" usebundle => delete_file("$(global_vars.file2)"); 
  reports:

    !doesfile1exist::
      "$(global_vars.file1) does not exist.";
    doesfile1exist::
      "$(global_vars.file1) did exist. Call to delete it was made.";    

    !doesfile2exist::
      "$(global_vars.file2) does not exist.";
    doesfile2exist::
      "$(global_vars.file2) did exist. Call to delete it was made.";    

}


bundle agent create_file_1
{

  files:
      "$(global_vars.file1)"
      perms => system,
      create => "true";

  reports:
    "$(global_vars.file1) has been created";
}


bundle agent outer_bundle_1
{
    files:

       "$(global_vars.file1)"
       create    => "false",
       edit_line => inner_bundle_1;
}

bundle agent copy_a_file
{
  files:

      "$(global_vars.file2)"
      copy_from => local_cp("$(global_vars.file1)");

  reports:
     "$(global_vars.file1) has been copied to $(global_vars.file2)";
     "*********************************";
}

bundle agent do_files_exist_2

{

  methods:

    "any" usebundle => does_file_exist($(global_vars.file1));
    "any" usebundle => does_file_exist($(global_vars.file2));

}

bundle agent does_file_exist(filename)
{
  vars:
      "filestat" string => filestat("$(filename)","mtime");

  classes:
      "fileexists" expression => fileexists("$(filename)");

  reports:

    fileexists::

      "$(filename) exists. Last Modified Time = $(filestat).";

    !fileexists::

      "$(filename) does not exist";
}

bundle agent list_file_1
{

  methods:  
    "any" usebundle => file_content($(global_vars.file1));
    "any" usebundle => file_content($(global_vars.file2));
  reports:
    "*********************************";

}

bundle agent exec_aout
{

  classes:
    "doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");

  vars:
    doesaoutexist::
    "aout" string => execresult("$(global_vars.aoutexec)","noshell");

  reports:
    doesaoutexist::
    "*********************************";
    "$(global_vars.aoutbin) determined that $(global_vars.file2) is $(aout) than $(global_vars.file1)";
    "*********************************";
    !doesaoutexist::
    "Executable $(global_vars.aoutbin) does not exist.";

}

bundle agent stat
{

  classes:

    "doesfile1exist" expression => fileexists("$(global_vars.file1)");
    "doesfile2exist" expression => fileexists("$(global_vars.file2)");

  vars:

    doesfile1exist::

    "file1" string => "$(global_vars.file1)";
    "file2" string => "$(global_vars.file2)";

    "file1_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file1)","noshell");
    "file1_split1" slist => string_split($(file1_stat)," ",3);
    "file1_split2" string => nth("file1_split1",1);
    "file1_split3" slist => string_split($(file1_split2),"\.",3);
    "file1_split4" string => nth("file1_split3",1);

    "file2_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file2)","noshell");
    "file2_split1" slist => string_split($(file2_stat)," ",3);
    "file2_split2" string => nth("file2_split1",1);
    "file2_split3" slist => string_split($(file2_split2),"\.",3);
    "file2_split4" string => nth("file2_split3",1);

  methods:

      "any" usebundle => exec_aout();

  reports:
    doesfile1exist::
      "Parts of a second extracted extracted from stat for $(file1): $(file1_split4). Full stat output for $(file1): $(file1_stat)";
      "Parts of a second extracted extracted from stat for $(file2): $(file2_split4). Full stat output for $(file2): $(file2_stat)";
      "Using the binary Linux application stat to compare two files can help determine if the modified times between two files are different. The difference between the stat application using its additional flags and using CFEngine's built in support for getting and comparing file stats (e.g. filestat, isnewerthan) is that normally the accuracy is only to the second of the file's modified time. In order to better compare two files requires parts of a second as well, which the stat command can provide with some additional flags. Unfortunately the information must be extracted from the middle of a string, which is what the stat bundle accomplishes using the string_split and nth functions.";
      "*********************************";
    !doesfile1exist::
      "stat: $(global_vars.file1) and probably $(global_vars.file2) do not exist.";

}

bundle agent outer_bundle_2
{
    files:

       "$(global_vars.file2)"
       create    => "false",
       edit_line => inner_bundle_2;

}

bundle edit_line inner_bundle_1
{
  vars:

    "msg" string => "Helloz to World!";

  insert_lines:
    "$(msg)";

  reports:
    "inserted $(msg) into $(global_vars.file1)";

}

bundle edit_line inner_bundle_2
{
   replace_patterns:

   "Helloz to World!"
      replace_with => hello_world;

   reports:
      "Text in $(global_vars.file2) has been replaced";

}

body replace_with hello_world
{
   replace_value => "Hello World";
   occurrences => "all";
}


bundle agent list_file_2
{

  methods:

      "any" usebundle => file_content($(global_vars.file1));
      "any" usebundle => file_content($(global_vars.file2));    

  classes:

      "ok" expression => isgreaterthan(filestat("$(global_vars.file2)","mtime"),filestat("$(global_vars.file1)","mtime"));
      "newer" expression => isnewerthan("$(global_vars.file2)","$(global_vars.file1)");

  reports:
    "*********************************";
      ok::
         "Using isgreaterthan+filestat determined that $(global_vars.file2) was modified later than $(global_vars.file1).";

      !ok::
         "Using isgreaterthan+filestat determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
      newer::
         "Using isnewerthan determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
      !newer::
         "Using isnewerthan determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";

}

bundle agent file_content(filename)
{

  vars:

      "file_content" string => readfile( "$(filename)" , "0" );
      "file_stat" string => filestat("$(filename)","mtime");

  reports:
      "Contents of $(filename) = $(file_content). Last Modified Time = $(file_stat).";
      #"The report on contents will only show new content and modifications. Even if the method is called more than once, if the evaluation is exactly the same as the previous call then there will be no report (possibly because the bundle is not evaluated a second time?).";


}

body perms system
{
      mode  => "0640";
}

Enterprise API Examples

See Also: Enterprise API Reference


SQL Query Examples

Synchronous Example: Listing Hostname and IP for Ubuntu Hosts

Request:

    curl -k --user admin:admin https://test.cfengine.com/api/query -X POST -d '{ "query": "SELECT Hosts.HostName, Hosts.IPAddress FROM Hosts"}'

Response:

{
  "data": [
    {
      "header": [
        {
          "columnName": "hostname",
          "columnType": "STRING"
        },
        {
          "columnName": "ipaddress",
          "columnType": "STRING"
        }
      ],
      "query": "select hostname, ipaddress from hosts",
      "queryTimeMs": 152,
      "rowCount": 1001,
      "rows": [
        [
          "ubuntu10-2.stage.cfengine.com",
          "172.20.100.1"
        ],
        [
          "ubuntu10-3.stage.cfengine.com",
          "172.20.100.2"
        ],
        [
          "ubuntu10-4.stage.cfengine.com",
          "172.20.100.3"
        ]
      ],
    }
  ],
  "meta": {
    "count": 1,
    "page": 1,
    "timestamp": 1437051092,
    "total": 1
  }
}
Subscribed Query Example: Creating A Subscribed Query

Here we create a new query to count file changes by name and have the result sent to us by email. The schedule field is any CFEngine context expression. The backend polls subscriptions in a loop and checks whether it's time to generate a report and send it out. In the following example, user milton creates a new subscription to a report which he names file-changes-report, which will be sent out every Monday night. His boss will get an email with a link to a PDF version of the report.

Request:

    curl -k --user admin:admin https://test.cfengine.com/api/user/milton/ subscription/query/file-changes-report -X PUT -d '{"to": "boss@megaco.com", "query": "SELECT FileName, Count(*) FROM FileChangesLog GROUP BY FileName", "schedule": "Monday.Hr23.Min59", "title": "A very important file changes report""description": "Text that will be included in email""outputTypes": [ "pdf" ] }'

Response:

    204 No Content
Subscribed Query Example: Listing Report Subscriptions

Milton can list all his current subscriptions by issuing the following.

Request:

    curl -k --user admin:admin https://test.cfengine.com/api/user/milton/subscription/query

Response:

{
  "meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1351003514
  },
  "data": [
    {
      "id": "file-changes-report",
      "to": "boss@megaco.com",
      "query": "SELECT FileName, Count(*) FROM FileChangesLog GROUP BY FileName",
      "title": "A very important file changes report",
      "description": "Text that will be included in email",
      "schedule": "Monday.Hr23.Min59",
      "outputTypes": [
        "pdf"
      ]
    }
  ]
}
Subscribed Query Example: Removing A Report Subscription

Request:

    curl -k --user admin:admin https://test.cfengine.com/api/user/milton/subscription/query/file-changes-report -X DELETE

Response:

    204 No Content

Checking Status

You can get basic info about the API by issuing /api. This status information may also be useful if you contact support, as it gives some basic diagnostics.

Request

curl -k --user admin:admin https://test.cfengine.com/api/

Response

{
  "meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1351154889
  },
  "data": [
    {
      "apiName": "CFEngine Enterprise API",
      "apiVersion": "v1",
      "enterpriseVersion": "3.0.0a1.81c0d4c",
      "coreVersion": "3.5.0a1.f3649b2",
      "databaseHostname": "127.0.0.1",
      "databasePort": 27017,
      "authenticated": "internal",
      "license": {
        "expires": 1391036400,
        "installTime": 1329578143,
        "owner": "Stage Environment",
        "granted": 20,
        "licenseUsage": {
          "lastMeasured": 1351122120,
          "samples": 1905,
          "minObservedLevel": 7,
          "maxObservedLevel": 30,
          "meanUsage": 21.9689,
          "meanCumulativeUtilization": 109.8446,
          "usedToday": 7
        }
      }
    }
  ]
}

Managing Settings

Settings support two operations, GET (view settings) and POST (update settings). When settings are updated, they are sanity checked individually and as a whole. All or no settings will be updated for a request.

Example: Viewing settings

Request

curl --user admin:admin http://test.cfengine.com/api/settings

Response

{
  "meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1350992335
  },
  "data": [
    {
      "hostIdentifier": "default.sys.fqhost",
      "rbacEnabled": true,
      "ldapEnabled": true,
      "blueHostHorizon": 900,
      "sketchActivationAlertTimeout": 60
    }
  ]
}
Example: Configuring LDAP

The setting ldapEnabled turns external authentication on or off. LDAP settings are managed by the LDAP API and not this Settings API.

Request

curl --user admin:admin http://test.cfengine.com/api/settings -X PATCH -d '{ "ldapEnabled": true }'

Response

204 No Content
Example: Changing The Log Level

The API uses standard Unix syslog to log a number of events. Additionally, log events are sent to stderr, which means they may also end up in your Apache log. Log events are filtered based on the log level in settings. Suppose you wanted to have greater visibility into the processing done at the back-end. The standard log level is error. Changing it to info is done as follows.

NOTE: Change to API log level will only take effect after Apache has re-started.

Request

curl --user admin:admin http://test.cfengine.com/api/settings -X PATCH -d '{ "logLevel": "info" }'

Response

204 No Content

Managing Users and Roles

Users and Roles determine who has access to what data from the API. Roles are defined by regular expressions that determine which hosts the user can see, and what policy outcomes are restricted.

Example: Listing Users

Request

curl --user admin:admin http://test.cfengine.com/api/user

Response

{
  "meta": {
    "page": 1,
    "count": 2,
    "total": 2,
    "timestamp": 1350994249
  },
  "data": [
    {
      "id": "calvin",
      "external": true,
      "roles": [
        "Huguenots", "Marketing"
      ]
    },
    {
      "id": "quinester",
      "name": "Willard Van Orman Quine",
      "email": "noreply@@aol.com",
      "external": false,
      "roles": [
        "admin"
      ]
    }
  ]
}
Example: Creating a New User

All users will be created for the internal user table. The API will never attempt to write to an external LDAP server.

Request

curl --user admin:admin http://test.cfengine.com/api/user/snookie -X PUT -d
{
  "email": "snookie@mtv.com",
  "roles": [
    "HR"
  ]
}

Response

201 Created
}
Example: Updating an Existing User

Both internal and external users may be updated. When updating an external users, the API will essentially annotate metadata for the user, it will never write to LDAP. Consequently, passwords may only be updated for internal users. Users may only update their own records, as authenticated by their user credentials.

Request

curl --user admin:admin http://test.cfengine.com/api/user/calvin -X POST -d '{ "name": "Calvin" }'

Response

204 No Content
Example: Retrieving a User

It is possible to retrieve data on a single user instead of listing everything. The following query is similar to issuing GET /api/user?id=calvin, with the exception that the previous query accepts a regular expression for id.

Request

curl --user admin:admin http://test.cfengine.com/api/user/calvin

Response

{
  "meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1350994249
  },
  "data": [
    {
      "id": "calvin",
      "name": "Calvin",
      "external": true,
      "roles": [
        "Huguenots", "Marketing"
      ]
    },
  ]
}
Example: Adding a User to a Role

Adding a user to a role is just an update operation on the user. The full role-set is updated, so if you are only appending a role, you may want to fetch the user data first, append the role and then update. The same approach is used to remove a user from a role.

Request

curl --user admin:admin http://test.cfengine.com/api/user/snookie -X POST -d
{
  "roles": [
    "HR", "gcc-contrib"
  ]
}

Response

204 No Content
}
Example: Deleting a User

Users can only be deleted from the internal users table.

Request

curl --user admin:admin http://test.cfengine.com/api/user/snookie -X DELETE

Response

204 No Content

Tracking changes

Changes REST API allows to track the changes made by cf-agent in the infrastructure.

Example: Count changes

This examples shows how to count changes performed by cf-agent within last 24h hours.

Example is searching for changes that are performed by linux machines within generate_repairs bundle.

Request

curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy/count?include[]=linux&bundlename=generate_repairs'

Response

{
  "count": 381
}
Example: Show vacuum command executions

Show all vacuumdb executions within last 24 hours executed on policy server.

Example is searching for changes that are performed by policy_server machines that execute commands promise with command /var/cfengine/bin/vacuumdb% - there is '%' sign at the end which is a wildcard as vacuumdb is executed with different options across policy.

Request

curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy?include[]=policy_server&promisetype=commands&promiser=/var/cfengine/bin/vacuumdb%'

Response

{
  "data": [
    {
      "bundlename": "cfe_internal_postgresql_vacuum",
      "changetime": 1437642099,
      "hostkey": "SHA=6ddfd5eaa85ee681ec12ce833fd7206e4d21c76e496be5f8b403ad0ead60a6ce",
      "hostname": "hub.provisioned.1436361289.cfengine.com.com",
      "logmessages": [
        "Executing 'no timeout' ... '/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'",
        "Completed execution of '/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'"
      ],
      "policyfile": "/var/cfengine/inputs/lib/cfe_internal_hub.cf",
      "promisees": [],
      "promisehandle": "cfe_internal_postgresql_maintenance_commands_run_vacuumdb",
      "promiser": "/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb",
      "promisetype": "commands",
      "stackpath": "/default/cfe_internal_management/methods/'CFEngine_Internals'/default/cfe_internal_enterprise_main/methods/'hub'/default/cfe_internal_postgresql_vacuum/commands/'/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'[0]"
    }
  ],
  "total": 1,
  "next": null,
  "previous": null
}

Browsing Host Information

A resource /api/host is added as an alternative interface for browsing host information. For full flexibility we recommend using SQL reports via /api/query for this. however, currently vital signs (data gathered from cf-monitord) is not part of the SQL reports data model.

Example: Listing Hosts With A Given Context

Request

curl --user admin:admin http://test.cfengine.com/api/host?context-include=windows.*

Response

{
  "meta": {
    "page": 1,
    "count": 2,
    "total": 2,
    "timestamp": 1350997528
  },
  "data": [
    {
      "id": "1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8",
      "hostname": "windows2008-2.test.cfengine.com",
      "ip": "172.20.100.43"
    },
    {
      "id": "dddc95486d97e4308f164ddc1fdbbc133825f35254f9cfbd59393a671015ab99",
      "hostname": "windows2003-2.test.cfengine.com",
      "ip": "172.20.100.42"
    }
  ]
}
Example: Looking Up Hosts By Hostname

Contexts, also known as classes, are powerful. You can use them to categorize hosts according to a rich set of tags. For example, each host is automatically tagged with a canonicalized version of its hostname and IP-address. So we could lookup the host with hostname windows2003-2.test.cfengine.com as follows (lines split and indented for presentability).

Request

curl --user admin:admin http://test.cfengine.com/api/host?context-include=
   windows2003_2_stage_cfengine_com

Response

{
  "meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1350997528
  },
  "data": [
    {
      "id": "dddc95486d97e4308f164ddc1fdbbc133825f35254f9cfbd59393a671015ab99",
      "hostname": "windows2003-2.test.cfengine.com",
      "ip": "172.20.100.42"
    }
  ]
}
Example: Looking Up Hosts By IP

Similarly we can lookup the host with hostname windows2008-2.test.cfengine.com by IP as follows (lines split and indented for presentability).

Request

curl --user admin:admin http://test.cfengine.com/api/host?
   context-include=172_20_100_43

Response

{
  "meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1350997528
  },
  "data": [
    {
      "id": "1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8",
      "hostname": "windows2008-2.stage.cfengine.com",
      "ip": "172.20.100.43"
    }
  ]
}
Example: Removing Host Data

If a host has been decommissioned from a Hub, we can explicitly remove data associated with the host from the Hub, by issuing a DELETE request (lines split and indented for presentability).

Request

curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8 -X DELETE

Response

204 No Content
Example: Listing Available Vital Signs For A Host

Each host record on the Hub has a set of vital signs collected by cf-monitord on the agent. We can view the list of vitals signs from as host as follows (lines split and indented for presentability).

Request

curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a/vital

Response

{
  "meta": {
    "page": 1,
    "count": 4,
    "total": 4,
    "timestamp": 1351001799
  },
  "data": [
    {
      "id": "messages",
      "description": "New log entries (messages)",
      "units": "entries",
      "timestamp": 1351001400
    },
    {
      "id": "mem_swap",
      "description": "Total swap size",
      "units": "megabytes",
      "timestamp": 1351001400
    },
    {
      "id": "mem_freeswap",
      "description": "Free swap size",
      "units": "megabytes",
      "timestamp": 1351001400
    },
    {
      "id": "mem_free",
      "description": "Free system memory",
      "units": "megabytes",
      "timestamp": 1351001400
    },
}
Example: Retrieving Vital Sign Data

Each vital sign has a collected time series of values for up to one week. Here we retrieve the time series for the mem_free vital sign at host 4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a for October 23rd 2012 12:20pm to 12:45pm GMT (lines split and indented for presentability).

Request

curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a/vital/\
mem_free?from=1350994800&to=1350996300

Response

"meta": {
    "page": 1,
    "count": 1,
    "total": 1,
    "timestamp": 1351002265
  },
  "data": [
    {
      "id": "mem_free",
      "description": "Free system memory",
      "units": "megabytes",
      "timestamp": 1351001700,
      "values": [
        [
          1350994800,
          36.2969
        ],
        [
          1350995100,
          36.2969
        ],
        [
          1350995400,
          36.2969
        ],
        [
          1350995700,
          36.2969
        ],
        [
          1350996000,
          36.1758
        ],
        [
          1350996300,
          36.2969
        ]
      ]
    }
  ]