The Complete Examples and Tutorials
Table of Content
- Example Snippets
- General Examples
- Common Promise Patterns
- Mount NFS filesystem
- Change detection
- Create files and directories
- Copy single files
- Find the MAC address
- Aborting execution
- Set up sudo
- Updating from a central policy server
- Install packages
- Ensure a service is enabled and running
- Customize Message of the Day
- Ensure a process is not running
- Distribute ssh keys
- Set up name resolution with DNS
- Set up time management through NTP
- Check filesystem space
- Restart a Process
- CFEngine Administration Examples
- Measuring Examples
- Software Administration Examples
- Commands, Scripts, and Execution Examples
- File and Directory Examples
- File Template Examples
- Database Examples
- Network Examples
- System Security Examples
- System Information Examples
- System Administration Examples
- System File Examples
- Windows Registry Examples
- User Management and ACL Examples
- Tutorials
- Enterprise API Examples
Links to Examples
- Example Snippets: This section is divided into topical areas and includes many examples of policy and promises. Each of the snippets can be easily copied or downloaded to a policy server and used as is.
Note: CFEngine also includes a small set of examples by default, which can be
found in /var/cfengine/share/doc/examples
.
- Enterprise API Examples
- Tutorials
- Design Center examples in the CFEngine GitHub repository.
See Also:
Tutorial for Running Examples
In this tutorial, you will perform the following:
- Create a simple "Hello World!" example policy file
- Make the example a standalone policy
- Make the example an executable script
- Add the example to the main policy file (
promises.cf
)
Note if your CFEngine administrator has enabled continuous deployment of the policy from a Version Control System, your changes may be overwritten!
"Hello World" Policy Example
Policies contain bundles, which are collections of promises. A promise is a declaration of intent. Bundles allow related promises to be grouped together, as illustrated in the steps that follow.
Following these steps, you will login to your policy server via the SSH protocol, use the vi command line editor to create a policy file named hello_world.cf, and create a bundle that calls a promise to display some text.
- Log into a running server machine using ssh (PuTTY may be used if using Windows).
- Type
sudo su
for super user (enter your password if prompted). - To get to the masterfiles directory, type
cd /var/cfengine/masterfiles
. - Create the file with the command:
vi hello_world.cf
In the vi editor, enter
i
for "Insert" and enter the following content (ie. copy and paste from a text editor):bundle agent hello_world { reports: any:: "Hello World!"; }
Exit the "Insert" mode by pressing the "esc" button. This will return to the command prompt.
Save the changes to the file by typing
:w
then "Enter".Exit vi by typing
:q
then "Enter".
In the policy file above, we have defined an agent bundle named hello_world
. Agent
bundles are only evaluated by cf-agent, the agent component of CFEngine.
This bundle promises to report on any class of hosts.
Activate a Bundle Manually
Activate the bundle manually by executing the following command at prompt:
% /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf --bundlesequence hello_world
This command instructs CFEngine to ignore locks, load
the hello_world.cf
policy, and activate the hello_world
bundle. See the output below:
# /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf --bundlesequence hello_world
2013-08-20T14:03:43-0500 notice: R: Hello World!
As you get familiar with CFEngine, you'll probably start shortening this command to this equivalent:
/var/cfengine/bin/cf-agent -Kf ./hello_world.cf -b hello_world
Note the full path to the binary in the above command. CFEngine stores its binaries in /var/cfengine/bin on Linux and Unix systems. Your path might vary depending on your platform and the packages your are using. CFEngine uses /var because it is one of the Unix file systems that resides locally. Thus, CFEngine can function even if everything else fails (your other file systems, your network, and even system binaries) and possibly repair problems.
Make the Example Stand Alone
Instead of specifying the bundle sequence on the command line (as it was above), a body common
control section can be added to
the policy file. The body common control refers to those promises that are hard-coded into
all CFEngine components and therefore affect the behavior of all components. Note that only
one body common control
is allowed per agent activation.
Go back into vi by typing "vi" at the prompt. Then type i
to insert
body common control to hello_world.cf
. Place it above bundle agent hello_world, as
shown in the following example:
body common control
{
bundlesequence => { "hello_world" };
}
bundle agent hello_world
{
reports:
any::
"Hello World!";
}
Now press "esc" to exit the "Insert" mode, then type :w
to save the file changes and "Enter".
Exit vi by typing :q
then "Enter." This will return to the prompt.
Execute the following command:
/var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf
The output is shown below:
# /var/cfengine/bin/cf-agent --no-lock --file ./hello_world.cf
2013-08-20T14:25:36-0500 notice: R: Hello World!
Note: It may be necessary to add a reference to the standard library within the body common control section, and remove the bundlesequence line. Example:
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
Make the Example an Executable Script
Add the #!
marker ("shebang") to hello_world.cf
in order to invoke CFEngine policy as an executable script:
Again type "vi" then "Enter" then i
to insert the following:
#!/var/cfengine/bin/cf-agent --no-lock
Add it before body common control, as shown below:
#!/var/cfengine/bin/cf-agent --no-lock
body common control
{
bundlesequence => { "hello_world" };
}
bundle agent hello_world
{
reports:
any::
"Hello World!";
}
Now exit "Insert" mode by pressing "esc". Save file changes by typing :w
then "Enter"
then exit vi by typing :q
then "Enter". This will return to the prompt.
Make the policy file executable, and then run it, by typing the following two commands:
chmod +x ./hello_world.cf
Followed by:
./hello_world.cf
See the output below:
# chmod +x ./hello_world.cf
# ./hello_world.cf
2013-08-20T14:39:34-0500 notice: R: Hello World!
Integrating the Example into your Main Policy
Make the example policy part of your main policy by doing the following on your policy server:
Ensure the example is located in
/var/cfengine/masterfiles
.If the example contains a
body common control
section, delete it. That section will look something like this:body common control { bundlesequence => { "hello_world" }; }
You cannot have duplicate control bodies (i.e. two agent control bodies, one in the main file and one in the example) as CFEngine won't know which it should use and they may conflict.
To resolve this, copy the contents of the control body section from the
example into the identically named control body section in the main policy
file /var/cfengine/masterfiles/promises.cf
and then remove the control body
from the example.
Insert the example's bundle name in the
bundlesequence
section of the main policy file/var/cfengine/masterfiles/promises.cf
:bundlesequence => { ... "hello_world", ... };
Insert the policy file name in the
inputs
section of the main policy file/var/cfengine/masterfiles/promises.cf
:inputs => { ... "hello_world.cf", ... };
You must also remove any inputs section from the example that includes the external library:
inputs => { "libraries/cfengine_stdlib.cf" };
This is necessary, since
cfengine_stdlib.cf
is already included in the inputs section of the master policy.The example policy will now be executed every five minutes along with the rest of your main policy.
Notes: You may have to fill the example with data before it will work.
For example, the LDAP query in active_directory.cf
needs a domain name.
In the variable declaration, replace "cftesting" with your domain name:
vars:
# NOTE: Edit this to your domain, e.g. "corp"
"domain_name" string => "cftesting";
Example Snippets
- General Examples
- CFEngine Administration Examples
- Measuring Examples
- Software Administration Examples
- Commands, Scripts, and Execution Examples
- File and Directory Examples
- File Template Examples
- Database Examples
- Network Examples
- System Security Examples
- System Information Examples
- System Administration Examples
- System File Examples
- Windows Registry Examples
- User Management and ACL Examples
<!--- End include:
/home/jenkins/workspace/build-documentation-3.7/label/DOCUMENTATION_x86_64_linux_ubuntu_16/documentation/examples/example-snippets.markdown
-->
General Examples
Basic Example
To get started with CFEngine, you can imagine the following template for entering examples. This part of the code is common to all the examples.
body common control
{
bundlesequence => { "main" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent main
{
# example
}
The general pattern
The general pattern of the syntax is like this (colors in html version: red, CFEngine word; blue, user-defined word):
bundle component name(parameters)
{
what_type:
where_when::
# Traditional comment
"promiser" -> { "promisee1", "promisee2" },
comment => "The intention ...",
handle => "unique_id_label",
attribute_1 => body_or_value1,
attribute_2 => body_or_value2;
}
Hello world
body common control
{
bundlesequence => { "hello" };
}
bundle agent hello
{
reports:
linux::
"Hello world!";
}
Array example
body common control
{
bundlesequence => { "array" };
}
bundle common g
{
vars:
"array[1]" string => "one";
"array[2]" string => "two";
}
bundle agent array
{
vars:
"localarray[1]" string => "one";
"localarray[2]" string => "two";
reports:
linux::
"Global $(g.array[1]) and $(localarray[2])";
}
Common Promise Patterns
This section includes includes common promise patterns. Refer to them as you write policy for your system.
- Aborting execution
- Change detection
- Check filesystem space
- Copy single files
- Create files and directories
- Customize Message of the Day
- Distribute ssh keys
- Ensure a process is not running
- Ensure a service is enabled and running
- Find the MAC address
- Install packages
- Mount NFS filesystem
- Restart a Process
- Set up sudo
- Set up time management through NTP
- Set up name resolution with DNS
- Updating from a central policy server
Mount NFS filesystem
Mounting an NFS filesystem is straightforward using CFEngine's storage promises. The following bundle specifies the name of a remote file system server, the path of the remote file system and the mount point directory on the local machine:
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
"/mnt" mount => nfs("fileserver","/home"); # "/mnt" is the local moint point
# "fileserver" is the remote fileserver
# "/home" is the path to the remote file system
}
body mount nfs(server,source)
{
mount_type => "nfs"; # Protocol type of remote file system
mount_source => "$(source)"; # Path of remote file system
mount_server => "$(server)"; # Name or IP of remote file system server
mount_options => { "rw" }; # List of option strings to add to the file system table ("fstab")
edit_fstab => "true"; # True/false add or remove entries to the file system table ("fstab")
}
This policy can be found in /var/cfengine/share/doc/examples/example_mount_nfs.cf
Here is an example run. At start, the filesystem is not in /etc/fstab and is not mounted:
# grep mnt /etc/fstab # filesystem is not in /etc/fstab
# df |grep mnt # filesystem is not mounted
Now we run CFEngine to mount the filesystem and add it to /etc/fstab:
# cf-agent -f example_mount_nfs.cf
2013-06-08T17:48:42-0700 error: Attempting abort because mount went into a retry loop.
# grep mnt /etc/fstab
fileserver:/home /mnt nfs rw
# df |grep mnt
fileserver:/home 149912064 94414848 47882240 67% /mnt
#
Note: CFEngine errors out after it mounts the filesystem and updates /etc/fstab. There is a ticket https://cfengine.com/dev/issues/2937 open on this issue.
Change detection
This policy will look for changes recursively in a directory.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
files:
"/home/mark/tmp/web" # Directory to monitor for changes.
changes => detect_all_change,
depth_search => recurse("inf");
}
body changes detect_all_change
{
report_changes => "all";
update_hashes => "true";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
This policy can be found in
/var/cfengine/share/doc/examples/change_detect.cf
and downloaded directly from
github.
Here is an example run.
First, let's create some files for CFEngine to monitor:
# mkdir /etc/example
# date > /etc/example/example.conf
CFEngine detects new files and adds them to the file integrity database:
# cf-agent -f unit_change_detect.cf
2013-06-06T20:53:26-0700 error: /example/files/'/etc/example':
File '/etc/example/example.conf' was not in 'md5' database - new file found
# cf-agent -f unit_change_detect.cf -K
If there are no changes, CFEngine runs silently:
# cf-agent -f unit_change_detect.cf
#
Now let's update the mtime, and then the mtime and content. CFEngine will notice the changes and record the new profile:
# touch /etc/example/example.conf # update mtime
# cf-agent -f unit_change_detect.cf -K
2013-06-06T20:53:52-0700 error: Last modified time for
'/etc/example/example.conf' changed 'Thu Jun 6 20:53:18 2013'
-> 'Thu Jun 6 20:53:49 2013'
# date >> /etc/example/example.conf # update mtime and content
# cf-agent -f unit_change_detect.cf -K
2013-06-06T20:54:01-0700 error: Hash 'md5' for '/etc/example/example.conf' changed!
2013-06-06T20:54:01-0700 error: /example/files/'/etc/example': Updating hash for
'/etc/example/example.conf' to 'MD5=8576cb25c9f78bc9ab6afd2c32203ca1'
2013-06-06T20:54:01-0700 error: Last modified time for '/etc/example/example.conf'
changed 'Thu Jun 6 20:53:49 2013' -> 'Thu Jun 6 20:53:59 2013'
#
Create files and directories
The following is a standalone policy that will create the file
/home/mark/tmp/test_plain
and the directory /home/mark/tmp/test_dir/
and set permissions on both.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
files:
"/home/mark/tmp/test_plain"
The promiser specifies the path and name of the file.
perms => system,
create => "true";
The perms
attribute sets the file permissions as defined in the system
body below. The create
attribute makes sure that the files exists. If it
doesn't, CFEngine will create it.
"/home/mark/tmp/test_dir/."
perms => system,
create => "true";
The trailing /.
in the filename tells CFEngine that the promiser is a
directory.
}
body perms system
{
mode => "0640";
}
This body sets permissions to "0640"
This policy can be found in
/var/cfengine/share/doc/examples/create_filedir.cf
and downloaded directly from
github.
Example output:
# cf-agent -f unit_create_filedir.cf -I
2013-06-08T14:56:26-0700 info: /example/files/'/home/mark/tmp/test_plain': Created file '/home/mark/tmp/test_plain', mode 0640
2013-06-08T14:56:26-0700 info: /example/files/'/home/mark/tmp/test_dir/.': Created directory '/home/mark/tmp/test_dir/.'
2013-06-08T14:56:26-0700 info: /example/files/'/home/mark/tmp/test_dir/.': Object '/home/mark/tmp/test_dir' had permission 0755, changed it to 0750
#
Copy single files
This is a standalone policy example that will copy single files,
locally (local_cp
) and from a remote site (secure_cp
).
The CFEngine Standard Library (cfengine_stdlib.cf) should be
included in the /var/cfengine/inputs/libraries/
directory and
inputs list as below.
body common control
{
bundlesequence => { "mycopy" };
inputs => { "libraries/cfengine_stdlib.cf" };
}
bundle agent mycopy
{
files:
"/tmp/test_plain"
Path and name of the file we wish to copy to
comment => "/tmp/test_plain promises to be an up-to-date copy of /bin/echo to demonstrate copying a local file",
copy_from => local_cp("$(sys.workdir)/bin/file");
Copy locally from path/filename
"/tmp/test_remote_plain"
comment => "/tmp/test_plain_remote promises to be a copy of cfengine://serverhost.example.org/repo/config-files/motd",
copy_from => secure_cp("/repo/config-files/motd", "serverhost.example.org");
}
Copy remotely from path/filename and specified host ```
This policy can be found in
/var/cfengine/share/doc/examples/copy_copbl.cf
and downloaded directly from
github.
Find the MAC address
Finding the ethernet address can vary between operating systems.
We will use CFEngine's built in function execresult
to execute commands
adapted for different operating systems, assign the output to variables,
and filter for the MAC address. We then report on the result.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
linux::
"interface" string => execresult("/sbin/ifconfig eth0", "noshell");
solaris::
"interface" string => execresult("/usr/sbin/ifconfig bge0", "noshell");
freebsd::
"interface" string => execresult("/sbin/ifconfig le0", "noshell");
darwin::
"interface" string => execresult("/sbin/ifconfig en0", "noshell");
# Use the CFEngine function 'regextract' to match the MAC address,
# assign it to an array called mac and set a class to indicate positive match
classes:
linux::
"ok" expression => regextract(
".*HWaddr ([^\s]+).*(\n.*)*", # pattern to match
"$(interface)", # string to scan for pattern
"mac" # put the text that matches the pattern into this array
);
solaris|freebsd::
"ok" expression => regextract(
".*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
darwin::
"ok" expression => regextract(
"(?s).*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
# Report on the result
reports:
ok::
"MAC address is $(mac[1])"; # return first element in array "mac"
}
This policy can be found in /var/cfengine/masterfiles/example_find_mac_addr.cf
Example run:
# cf-agent -f example_find_mac_addr.cf
2013-06-08T16:59:19-0700 notice: R: MAC address is a4:ba:db:d7:59:32
#
While the above illustrates the flexiblity of CFEngine in running external commands and parsing their output, as of CFEngine 3.3.0, Nova 2.2.0 (2011), you can get the MAC address natively:
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
linux:: "interface" string => "eth0";
solaris:: "interface" string => "bge0";
freebsd:: "interface" string => "le0";
darwin:: "interface" string => "en0";
reports:
"MAC address of $(interface) is: $(sys.hardware_mac[$(interface)])";
}
Aborting execution
Sometimes it is useful to abort a bundle execution if certain conditions are not met,
for example when validating input to a bundle. The following policy uses a list of
regular expressions for classes, or class expressions that cf-agent
will watch out for.
If any of these classes becomes defined, it will cause the current bundle to be aborted.
body common control
{
bundlesequence => { "example" };
}
body agent control
{
abortbundleclasses => { "invalid" };
}
bundle agent example
{
vars:
#"userlist" slist => { "mark", "john" }; # contains all valid entries
"userlist" slist => { "mark", "john", "thomas" }; # contains one invalid entry
classes:
"invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(userlist)"); # The class 'invalid' is set if the user name does not
# contain exactly four un-capitalized letters (bundle
# execution will be aborted if set)
reports:
!invalid::
"User name $(userlist) is valid at 4 letters";
}
This policy can be found in
/var/cfengine/share/doc/examples/abort.cf
and downloaded directly from
github.
This is how the policy runs when the userlist is valid:
# cf-agent -f unit_abort.cf
R: User name mark is valid at 4 letters
R: User name john is valid at 4 letters
#
This is how the policy runs when the userlist contains an invalid entry:
# cf-agent -f unit_abort.cf
Bundle example aborted on defined class "invalid"
#
To run this example file as part of your main policy you need to make an additional change:
There cannot be two body agent control
in the main policy. Delete the
body agent control
section from /var/cfengine/masterfiles/unit_abort.cf
.
Copy and paste abortbundleclasses => { "invalid" };
into
/var/cfengine/masterfiles/controls/cf_agent.cf
. If you add it to
the end of the file it should look something like this:
...
# dryrun => "true";
abortbundleclasses => { "invalid" };
}
Set up sudo
Setting up sudo is straightforward, we recommend managing it by copying trusted files from a repository. The following bundle will copy a master sudoers file to /etc/sudoers
(/tmp/sudoers
in this example - change it to /etc/sudoers
to use in production).
body common control
{
bundlesequence => { "sudoers" };
inputs => { "libraries/cfengine_stdlib.cf" };
}
bundle agent sudoers
{
# Define the master location of the sudoers file
vars:
"master_location" string => "/var/cfengine/masterfiles";
# Copy the master sudoers file to /etc/sudoers
files:
"/tmp/sudoers" # change to /etc/sudoers to use in production
comment => "Make sure the sudo configuration is secure and up to date",
perms => mog("440","root","root"),
copy_from => secure_cp("$(master_location)/sudoers","$(sys.policy_hub)");
}
We recommend editing the master sudoers file using visudo
or a similar tool. It is possible to use CFEngine's file editing capabilities to edit sudoers directly, but this does not guarantee syntax correctness and you might end up locked out.
Example run:
# cf-agent -f temp.cf -KI
2013-06-08T19:13:21-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T19:13:22-0700 info: Running full policy integrity checks
2013-06-08T19:13:23-0700 info: Copying from '192.168.183.208:/var/cfengine/masterfiles/sudoers'
2013-06-08T19:13:23-0700 info: /sudoers/files/'/tmp/sudoers': Object '/tmp/sudoers' had permission 0600, changed it to 0440
#
For reference we include an example of a simple sudoers file:
# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
Defaults env_reset
# User privilege specification
root ALL=(ALL) ALL
# Allow members of group sudo to execute any command after they have
# provided their password
%sudo ALL=(ALL) ALL
# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL
john ALL=(ALL) ALL
Updating from a central policy server
This is a conceptual example without any test policy associated with it.
The default policy shipped with CFEngine contains a centralized updating of policy that covers more subtleties than this example, and handles fault tolerance. Here is the main idea behind it. For simplicity, we assume that all hosts are on network 10.20.30.* and that the central policy server is 10.20.30.123.
bundle agent update
{
vars:
"master_location" string => "/var/cfengine/masterfiles";
"policy_server" string => "10.20.30.123";
comment => "IP address to locate your policy host.";
files:
"$(sys.workdir)/inputs"
perms => system("600"),
copy_from => remote_cp("$(master_location)",$(policy_server)),
depth_search => recurse("inf"); # This ensures recursive copying of all subdirectories
"$(sys.workdir)/bin"
perms => system("700"),
copy_from => remote_cp("/usr/local/sbin","localhost"),
depth_search => recurse("inf"); # This ensures recursive copying of all subdirectories
}
In addition the server needs to grant access to the clients, this is done in the body server control
:
body server control
{
allowconnects => { "127.0.0.1" , "10.20.30" };
allowallconnects => { "127.0.0.1" , "10.20.30" };
trustkeysfrom => { "127.0.0.1" , "10.20.30" };
}
Since we assume that all hosts are on network 10.20.30.* they will be granted access. In the default policy this is set to $(sys.policy_hub)/16
, i.e. all hosts in the same class B network as the hub will gain access. You will need to modify the access control list in body server control
if you have clients outside of the policy server's class B network.
Granting access to files and folders needs to be done in bundle server access_rules()
:
bundle server access_rules()
{
access:
10_20_30_123::
"/var/cfengine/masterfiles"
admit => { "127.0.0.1", "10.20.30" };
}
Install packages
Install desired packages.
body common control
{
bundlesequence => { "install_packages" };
inputs => { "libraries/cfengine_stdlib.cf" };
}
bundle agent install_packages
{
vars:
"desired_packages"
slist => { # list of packages we want
"ntp",
"lynx"
};
packages:
"$(desired_packages)" # operate on listed packages
package_policy => "add", # What to do with packages: install them.
package_method => generic; # Infer package manager (e.g. apt, yum) from the OS.
}
Caution: package management is a dirty business. If things don't go smoothly
using the generic method, you may have to use a method specific to your package
manager and get to your elbows in the details. But try generic
first. You
may get lucky.
Mind package names can differ OS to OS. For example, Apache httpd is "httpd" on Red Hat, and "apache2" on Debian.
Version comparison can be tricky when involving multipart version identifiers with numbers and letters.
CFEngine downloads the necessary packages from the default repositories if they are not present on the local machine, then installs them if they are not already installed.
Example run:
# dpkg -r lynx ntp # remove packages so CFEngine has something to repair
(Reading database ... 234887 files and directories currently installed.)
Removing lynx ...
Removing ntp ...
* Stopping NTP server ntpd [ OK ]
Processing triggers for ureadahead ...
Processing triggers for man-db ...
# cf-agent -f install_packages.cf # install packages
# dpkg -l lynx ntp # show installed packages
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-====================-====================-====================================================================
ii lynx 2.8.8dev.12-2ubuntu0 all Text-mode WWW Browser (transitional package)
ii ntp 1:4.2.6.p3+dfsg-1ubu amd64 Network Time Protocol daemon and utility programs
#
There are examples in /var/cfengine/share/doc/examples/
of installing packages using specific package managers:
- Red Hat (unit_package_yum.cf)
- Debian (unit_package_apt.cf)
- MSI for Windows (unit_package_msi_file.cf)
- Solaris (unit_package_solaris.cf)
- SuSE Linux (unit_package_zypper.cf)
Ensure a service is enabled and running
This example shows how to ensure services are started or stopped appropriately.
body file control
{
inputs => { "$(sys.libdir)/services.cf", "$(sys.libdir)/commands.cf" };
}
bundle agent main
{
vars:
linux::
"enable[ssh]"
string => ifelse( "debian|ubuntu", "ssh", "sshd"),
comment => "The name of the ssh service varies on different platforms.
Here we set the name of the service based on existing
classes and defaulting to `sshd`";
"disable[apache]"
string => ifelse( "debian|ubuntu", "apache2", "httpd" ),
comment => "The name of the apache web service varies on different
platforms. Here we set the name of the service based on
existing classes and defaulting to `httpd`";
"enable[cron]"
string => ifelse( "debian|ubuntu", "cron", "crond" ),
comment => "The name of the cron service varies on different
platforms. Here we set the name of the service based on
existing classes and defaulting to `crond`";
"disable[cups]"
string => "cups",
comment => "Printing services are not needed on most hosts.";
"enabled" slist => getvalues( enable );
"disabled" slist => getvalues( disable );
services:
linux::
"$(enabled)" -> { "SysOps" }
service_policy => "start",
comment => "These services should be running because x, y or z.";
"$(disabled)" -> { "SysOps" }
service_policy => "stop",
comment => "These services should not be running because x, y or z.";
systemd::
"sysstat"
service_policy => "stop",
comment => "Standard service handling for sysstat only works with
systemd. Other inits need cron entries managed.";
}
This policy can be found in
/var/cfengine/share/doc/examples/services.cf
and downloaded directly from
github.
Note: Not all services behave in the standard way. Some services may require custom handling. For example it is not uncommon for some services to not provide correct return codes for status checks.
See Also:
Example usage on systemd
We can see that before the policy run sysstat
is inactive, apache2
is
active, cups
is active, ssh
is active and cron
is inactive.
root@ubuntu:# systemctl is-active sysstat apache2 cups ssh cron
inactive
active
active
active
inactive
Now we run the policy to converge the system to the desired state.
root@ubuntu:# cf-agent --no-lock --inform --file ./services.cf
info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q stop apache2'
info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q stop apache2'
info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q stop cups'
info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q stop cups'
info: Executing 'no timeout' ... '/bin/systemctl --no-ask-password --global --system -q start cron'
info: Completed execution of '/bin/systemctl --no-ask-password --global --system -q start cron'
After the policy run we can see that systat
, apache2
, and cups
are
inactive. ssh
and cron
are active as specified in the policy.
root@ubuntu:/home/nickanderson/CFEngine/core/examples# systemctl is-active sysstat apache2 cups ssh cron
inactive
inactive
inactive
active
active
Example usage with System V
We can see that before the policy run sysstat
is not reporting status
correctly , httpd
is running, cups
is running, sshd
is running and
crond
is not running.
[root@localhost examples]# service sysstat status; echo $?
3
[root@localhost examples]# service httpd status; echo $?
httpd (pid 3740) is running...
0
[root@localhost examples]# service cups status; echo $?
cupsd (pid 3762) is running...
0
[root@localhost examples]# service sshd status; echo $?
openssh-daemon (pid 3794) is running...
0
[root@localhost examples]# service crond status; echo $?
crond is stopped
3
Now we run the policy to converge the system to the desired state.
[root@localhost examples]# cf-agent -KIf ./services.cf
info: Executing 'no timeout' ... '/etc/init.d/crond start'
info: Completed execution of '/etc/init.d/crond start'
info: Executing 'no timeout' ... '/etc/init.d/httpd stop'
info: Completed execution of '/etc/init.d/httpd stop'
info: Executing 'no timeout' ... '/etc/init.d/cups stop'
info: Completed execution of '/etc/init.d/cups stop'
After the policy run we can see that systat
is still not reporting status correctly (some services do not respond to standard checks), apache2
, and cups
are
inactive. ssh
and cron
are active as specified in the policy.
[root@localhost examples]# service sysstat status; echo $?
3
[root@localhost examples]# service httpd status; echo $?
httpd is stopped
3
[root@localhost examples]# service cups status; echo $?
cupsd is stopped
3
[root@localhost examples]# service sshd status; echo $?
openssh-daemon (pid 3794) is running...
0
[root@localhost examples]# service crond status; echo $?
crond (pid 3929) is running...
0
Customize Message of the Day
The Message of the Day is displayed when you log in or connect to a server. It typically shows information about the operating system, license information, last login, etc.
It is often useful to customize the Message of the Day to inform your users about some specifics of the system they are connecting to. In this example we will look at a bundle which adds three lines to the /etc/motd
file to inform about some system characteristics and that the system is managed by CFEngine.
The bundle is defined like this:
body common control
{
bundlesequence => { "edit_motd" };
}
bundle agent edit_motd
{
files:
"/etc/motd"
edit_line => my_motd,
edit_defaults => empty,
create => "true";
}
The bundle my_motd
and body empty
detail what content we want
in the file using CFEngine's built-in line-editor.
bundle edit_line my_motd
{
vars:
"interfaces_str" string => join(", ","sys.interfaces");
"ipaddresses_str" string => join(", ","sys.ip_addresses");
insert_lines:
"Welcome to $(sys.fqhost)!
This system is managed by CFEngine.
The policy was last updated on $(sys.last_policy_update).
The system has $(sys.cpus) cpus.
Network interfaces on this system are $(interfaces_str),
and the ip-addresses assigned are $(ipaddresses_str).";
}
The my_motd
bundle describes the content we want.
body edit_defaults empty
{
empty_file_before_editing => "true";
}
Baseline memory model of file to zero/empty before populating the model using edit_line my_motd ```
This policy can be found in
/var/cfengine/share/doc/examples/motd.cf
and downloaded directly from
github.
Example run:
##### ls /tmp/motd
ls: cannot access /tmp/motd: No such file or directory
##### cf-agent -f motd.cf
##### cat /tmp/motd
Welcome to tashkent!
This system is managed by CFEngine.
The policy was last updated on Sat Jun 8 15:16:00 2013.
The system has 4 cpus.
Network interfaces on this system are eth0, eth1,
and the ip-addresses assigned are 127.0.0.1, 10.10.23.68, 192.168.183.208.
#####
Ensure a process is not running
This is a standalone policy that will kill the sleep
process. You can adapt
it to make sure that any undesired process is not running.
body common control
{
bundlesequence => { "process_kill" };
}
bundle agent process_kill
{
processes:
"sleep"
signals => { "term", "kill" }; #Signals are presented as an ordered list to the process.
#On Windows, only the kill signal is supported, which terminates the process.
}
This policy can be found in /var/cfengine/share/doc/examples/unit_process_kill.cf
.
Example run:
# /bin/sleep 1000 &
[1] 5370
# cf-agent -f unit_process_kill.cf
[1]+ Terminated /bin/sleep 1000
#
Now let's do it again with inform mode turned on, and CFEngine will show the process table entry that matched the pattern we specified ("sleep"):
# /bin/sleep 1000 &
[1] 5377
# cf-agent -f unit_process_kill.cf -IK
2013-06-08T16:30:06-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T16:30:06-0700 info: Running full policy integrity checks
2013-06-08T16:30:06-0700 info: /process_kill/processes/'sleep': Signalled 'term' (15) to process 5377 (root 5377 3854 5377 0.0 0.0 11352 0 612 1 16:30 00:00:00 /bin/sleep 1000)
[1]+ Terminated /bin/sleep 1000
#
If we add the -v switch to turn on verbose mode, we see the /bin/ps command CFEngine used to dump the process table:
# cf-agent -f unit_process_kill.cf -Kv
...
2013-06-08T16:38:20-0700 verbose: Observe process table with /bin/ps -eo user,pid,ppid,pgid,pcpu,pmem,vsz,ni,rss,nlwp,stime,time,args
2013-06-08T16:38:20-0700 verbose: Matched 'root 5474 3854 5474 0.0 0.0 11352 0 612 1 16:38 00:00:00 /bin/sleep 1000'
...
Distribute ssh keys
Let's say we have a list of users that are trusted to login to a set of servers managed by CFEngine.
We are going to implement this trust relationship by ensuring the users' accounts on managed servers have a .ssh/authorized_keys file with each user's public key.
Let's assume we collected all users' public keys into a single directory on the server and that users exist on the clients (and have corresponding home directory). The following CFEngine policy distributes the keys from /var/cfengine/masterfiles/ssh_keys on the policy server to /var/cfengine/inputs/ssh_keys on the managed servers and from there each user's key will go to each user's .ssh/authorized_keys file.
Note: special variable $(sys.policy_hub) contains the hostname of the policy server.
You have to adapt this policy in the mentioned places for it to work in your environment.
body common control {
bundlesequence => { "distribute_ssh_keys" };
inputs => { "libraries/cfengine_stdlib.cf" };
}
bundle agent distribute_ssh_keys
{
vars:
"users" slist => { "user1", "user2" }; # List of users to be included in key distribution.
# Modify to include actual users.
"source_server" string => "$(sys.policy_hub)"; # Server where keys are stored
"source_directory" string => "/var/cfengine/masterfiles/ssh_keys"; # Source directory of key files
"local_cache" string => "/var/cfengine/inputs/ssh_keys"; # Local cache of key files
files:
"$(local_cache)/$(users).pub"
comment => "Copy public keys from an authorized source into a cache on localhost",
perms => mo("600","root"),
copy_from => remote_cp("$(source_directory)/$(users).pub","$(source_server)"),
action => if_elapsed("60"); # wait 60 min before checking this promise again
# Ensure that authorized_keys file exists and has permissions 600 and call a file editing promise
"/home/$(users)/.ssh/authorized_keys"
comment => "Edit the authorized keys into the user's personal keyring",
create => "true",
perms => m("600"),
edit_line => insert_file_if_no_line_matching("$(users)","$(local_cache)/$(users).pub"),
action => if_elapsed("60");
}
#####
bundle edit_line insert_file_if_no_line_matching(user,file)
{
# Check if user exists in the authorized_keys file
classes:
"have_user"
expression => regline("$(user).*","$(this.promiser)");
# Insert the content of the key file into authorized_keys if the user's key is not already there
insert_lines:
!have_user::
"$(file)"
insert_type => "file";
}
Example run:
First, let's setup for the run. Put users' SSH keys into the key distribution point on the policy hub:
policy_hub# ls /var/cfengine/masterfiles/ssh_keys/*pub
/var/cfengine/masterfiles/ssh_keys/user1.pub /var/cfengine/masterfiles/ssh_keys/user2.pub
policy_hub#
There are no authorized_keys files on the managed servers, but the home (and .ssh) directories exist:
# ls -d /home/user*/.ssh
/home/user1/.ssh /home/user2/.ssh
# ls /home/user?/.ssh/authorized_keys
ls: cannot access /home/user?/.ssh/authorized_keys: No such file or directory
#
Run CFEngine on one of the managed servers to create and populate /var/cfengine/inputs/ssh_keys from source (policy_hub:/var/cfengine/masterfiles/ssh_keys) and then install each user's key into that user's authorized_keys file:
# cf-agent -f ssh.cf
2013-06-08T15:49:29-0700 error: Failed to chdir into '/var/cfengine/inputs/ssh_keys'
#
Note: the above error only happens on the first run. Then /var/cfengine/inputs/ssh_keys is created and this error does not recur.
The local cache now contains the users' public keys:
ls /var/cfengine/inputs/ssh_keys/
user1.pub
user2.pub
#
CFEngine created authorized_keys files:
# ls /home/user?/.ssh/auth*keys
/home/user1/.ssh/authorized_keys
/home/user2/.ssh/authorized_keys
#
CFEngine installed the user's keys:
# more /home/user?/.ssh/auth*keys
::::::::::::::
/home/user1/.ssh/authorized_keys
::::::::::::::
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCjtmJf9QfME2KIV19C96EyRg1dizxKMTjLRsPtwsmC2fRyA3fRFvpUVKApigDTNxF5nDqfgGtY9
0KhnuqjhOgYWnpm4dmiTdFXJ5XHuNPCc4JpsXBeyMy2f8e1aobb/dN5UhSSZmYb84FkYwbI/EkxJ46CmmOpOi6C5AjYfqwzshIGNgJS39hbtsUimc
qBAOYTHzVpm5+KfHbNryZ9ORWEVcPvnchKtEfNu8iuDdecOxmWWUPhEyhUz7/SfZ4cPs7692JcIX2XQCsvsGWS5JPiVXGDPCcLz7WNI2A7rohoC9f
vpE11CBigl7zTlB0M7nQYzpjaf7qS3AvOXw5CLUPD user1@examplehost
::::::::::::::
/home/user2/.ssh/authorized_keys
::::::::::::::
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDhaBkZg7t63kNXqduU1LzLH+8DEkTGAhOjharGf6TMWL9fkWXS+Xjj2iD7KZgT2VBC9Hf8o+HhL
al5kyHYH8qRxtPXMm5UVhIHnq8hxDQQPo/jW62wwxB0N2pF8oU4sMzMzCANJYE3C6H0rjIzgloiCIkBwL21WoFhxZ145z7VoKTEf0ICRk2+xmCc2W
hX1pQVJzs5GlKlWEsJUp8Skqt+OuJTtIS4R3nJALvo7zindvum12DcbWfsrV5oW3gl89GkyDAdi1mWaqBmGX5qF5b19KaP4qdth61foUTR7NyHuCs
C/hNB84Loy+2nMU8QpKJ7Ha6UyBtU2YrzDxL3YPgJ user2@examplehost
#
Set up name resolution with DNS
There are many ways to configure name resolution. A simple and straightforward approach is to implement this as a simple editing promise for the /etc/resolv.conf
file.
body common control
{
bundlesequence => { "edit_name_resolution" };
}
bundle agent edit_name_resolution
{
files:
"/tmp/resolv.conf" # This is for testing, change to "$(sys.resolv)" to put in production
comment => "Add lines to the resolver configuration",
create => "true", # Make sure the file exists, create it if not
edit_line => resolver, # Call the resolver bundle defined below to do the editing
edit_defaults => empty; # Baseline memory model of file to empty before processing
# bundle edit_line resolver
}
bundle edit_line resolver
{
insert_lines:
any:: # Class/context where you use the below nameservers. Change to appropriate class
# for your system (if not any::, for example server_group::, ubuntu::, etc.)
# insert the search domain or name servers we want
"search mydomain.tld" location => start; # Replace mydomain.tld with your domain name
# The search line will always be at the start of the file
"nameserver 128.39.89.8";
"nameserver 128.39.74.66";
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
body location start
{
before_after => "before";
}
Example run:
# cf-agent -f unit_edit_name_resolution.cf # set up resolv.conf
# cat /tmp/resolv.conf # show resolv.conf
search mydomain.tld
nameserver 128.39.89.8
nameserver 128.39.74.66
# echo 'nameserver 0.0.0.0' >> /tmp/resolv.conf # mess up resolv.conf
# cf-agent -f ./unit_edit_name_resolution.cf -KI # heal resolv.conf
2013-06-08T18:38:12-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T18:38:12-0700 info: Running full policy integrity checks
2013-06-08T18:38:12-0700 info: /edit_name_resolution/files/'/tmp/resolv.conf': Edit file '/tmp/resolv.conf'
# cat /tmp/resolv.conf # show healed resolv.conf
search mydomain.tld
nameserver 128.39.89.8
nameserver 128.39.74.66
#
Set up time management through NTP
The following sets up a local NTP server that synchronizes with pool.ntp.org and clients that synchronize with your local NTP server. See bottom of this example if you don't want to build a server, but use a "brute force" method (repeated ntpdate syncs).
This example demonstrates you can have a lot of low-level detailed control if you want it.
bundle agent system_time_ntp
{
vars:
linux::
"cache_dir" string => "$(sys.workdir)/cache"; # Cache directory for NTP config files
"ntp_conf" string => "/etc/ntp.conf"; # Target file for NTP configuration
"ntp_server" string => "172.16.12.161"; #
"ntp_network" string => "172.16.12.0"; # IP address and netmask of your local NTP server
"ntp_mask" string => "255.255.255.0"; #
"ntp_pkgs" slist => { "ntp" }; # NTP packages to be installed to ensure service
# Define a class for the NTP server
classes:
any::
"ntp_hosts" or => { classmatch(canonify("ipv4_$(ntp_server)")) };
# Ensure that the NTP packages are installed
packages:
ubuntu::
"$(ntp_pkgs)"
comment => "setup NTP",
package_policy => "add",
package_method => generic;
# Ensure existence of file and directory for NTP drift learning statistics
files:
linux::
"/var/lib/ntp/ntp.drift"
comment => "Enable ntp service",
create => "true";
"/var/log/ntpstats/."
comment => "Create a statistic directory",
perms => mog("644","ntp","ntp"),
create => "true";
ntp_hosts::
# Build the cache configuration file for the NTP server
"/var/cfengine/cache/ntp.conf"
comment => "Build $(this.promiser) cache file for NTP server",
create => "true",
edit_defaults => empty,
edit_line => restore_ntp_master("$(ntp_network)","$(ntp_mask)");
centos.ntp_hosts::
# Copy the cached configuration file to its target destination
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_centos");
ubuntu.ntp_hosts::
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_ubuntu");
!ntp_hosts::
# Build the cache configuration file for the NTP client
"$(cache_dir)/ntp.conf"
comment => "Build $(this.promiser) cache file for NTP client",
create => "true",
edit_defaults => empty,
edit_line => restore_ntp_client("$(ntp_server)");
centos.!ntp_hosts::
# Copy the cached configuration file to its target destination
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_centos");
ubuntu.!ntp_hosts::
"$(ntp_conf)"
comment => "Ensure $(this.promiser) in a perfect condition",
copy_from => local_cp("$(cache_dir)/ntp.conf"),
classes => if_repaired("refresh_ntpd_ubuntu");
# Set classes (conditions) for to restart the NTP daemon if there have been any changes to configuration
processes:
centos::
"ntpd.*"
restart_class => "refresh_ntpd_centos";
ubuntu::
"ntpd.*"
restart_class => "refresh_ntpd_ubuntu";
# Restart the NTP daemon if the configuration has changed
commands:
refresh_ntpd_centos::
"/etc/init.d/ntpd restart";
refresh_ntpd_ubuntu::
"/etc/init.d/ntp restart";
}
#######################################################
bundle edit_line restore_ntp_master(network,mask)
{
vars:
"list" string =>
"######################################
# ntp.conf-master
driftfile /var/lib/ntp/ntp.drift
statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
# Use public servers from the pool.ntp.org project.
# Please consider joining the pool (http://www.pool.ntp.org/join.html).
# Consider changing the below servers to a location near you for better time
# e.g. server 0.europe.pool.ntp.org, or server 0.no.pool.ntp.org etc.
server 0.centos.pool.ntp.org
server 1.centos.pool.ntp.org
server 2.centos.pool.ntp.org
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict -4 default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1
# Hosts on local network are less restricted.
restrict $(network) mask $(mask) nomodify notrap";
insert_lines:
"$(list)";
}
#######################################################
bundle edit_line restore_ntp_client(serverip)
{
vars:
"list" string =>
"######################################
# This file is protected by cfengine #
######################################
# ntp.conf-client
driftfile /var/lib/ntp/ntp.drift
statsdir /var/log/ntpstats/
statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable
# Permit time synchronization with our time source, but do not
# permit the source to query or modify the service on this system.
restrict -4 default kod nomodify notrap nopeer noquery
restrict -6 default kod nomodify notrap nopeer noquery
# Permit all access over the loopback interface. This could
# be tightened as well, but to do so would effect some of
# the administrative functions.
restrict 127.0.0.1
restrict ::1
server $(serverip)
restrict $(serverip) nomodify";
insert_lines:
"$(list)";
}
This policy can be found in /var/cfengine/share/doc/examples/example_ntp.cf
If you don't want to build a server, you might do like this:
bundle agent time_management
{
vars:
any::
"ntp_server" string => "no.pool.ntp.org";
commands:
any::
"/usr/sbin/ntpdate $(ntp_server)"
contain => silent;
}
This is a hard reset of the time, it corrects it immediately. This may cause problems if there are large deviations in time and you are using time sensitive software on your system. An NTP daemon setup as shown above, on the other hand, slowly adapts the time to avoid causing disruption. In addition, the NTP daemon can be configured to learn your system's time drift and automatically adjust for it without having to be in touch with the server at all times.
Check filesystem space
Check how much space (in KB) is available on a directory's current partition.
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
classes:
"has_space" expression => isgreaterthan($(free), 0);
vars:
"free" int => diskfree("/tmp");
reports:
has_space::
"The filesystem has free space";
!has_space::
"The filesystem has NO free space";
}
R: The filesystem has free space
This policy can be found in
/var/cfengine/share/doc/examples/diskfree.cf
and downloaded directly from
github.
Example output:
# cf-agent -f unit_diskfree.cf
R: Freedisk 48694692
# df -k /tmp
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 149911836 93602068 48694692 66% /
#
Restart a Process
This is a standalone policy that will restart three CFEngine processes if they are not running.
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
vars:
"component" slist => { # List of processes to monitor
"cf-monitord",
"cf-serverd",
"cf-execd"
};
processes:
"$(component)"
restart_class => canonify("start_$(component)"); # Set the class "start_<component>" if it is not running
commands:
"/var/cfengine/bin/$(component)"
ifvarclass => canonify("start_$(component)"); # Evaluate the class "start_<component>", CFEngine will run
# the command if "start_<component> is set.
}
Notes: The canonify
function translates illegal characters to underscores, e.g. start_cf-monitord
becomes start_cf_monitord
. Only alphanumerics and underscores are allowed in CFEngine identifiers (names of variables, classes, bundles, etc.)
This policy can be found in /var/cfengine/share/doc/examples/unit_process_restart.cf
.
Example run:
# ps -ef |grep cf-
root 4305 1 0 15:14 ? 00:00:02 /var/cfengine/bin/cf-execd
root 4311 1 0 15:14 ? 00:00:05 /var/cfengine/bin/cf-serverd
root 4397 1 0 15:15 ? 00:00:06 /var/cfengine/bin/cf-monitord
# kill 4311
# ps -ef |grep cf-
root 4305 1 0 15:14 ? 00:00:02 /var/cfengine/bin/cf-execd
root 4397 1 0 15:15 ? 00:00:06 /var/cfengine/bin/cf-monitord
# cf-agent -f unit_process_restart.cf
# ps -ef |grep cf-
root 4305 1 0 15:14 ? 00:00:02 /var/cfengine/bin/cf-execd
root 4397 1 0 15:15 ? 00:00:06 /var/cfengine/bin/cf-monitord
root 8008 1 0 18:18 ? 00:00:00 /var/cfengine/bin/cf-serverd
#
And again, in Inform mode:
# kill 8008
# cf-agent -f unit_process_restart.cf -I
2013-06-08T18:19:51-0700 info: This agent is bootstrapped to '192.168.183.208'
2013-06-08T18:19:51-0700 info: Running full policy integrity checks
2013-06-08T18:19:51-0700 info: /process_restart/processes/'$(component)': Making a one-time restart promise for 'cf-serverd'
2013-06-08T18:19:51-0700 info: Executing 'no timeout' ... '/var/cfengine/bin/cf-serverd'
2013-06-08T18:19:52-0700 info: Completed execution of '/var/cfengine/bin/cf-serverd'
#
CFEngine Administration Examples
Ordering promises
This counts to five by default. If we change ‘/bin/echo one’ to ‘/bin/echox one’, then the command will fail, causing us to skip five and go to six instead.
This shows how dependencies can be chained in spite of the order of promises in the bundle.
Normally the order of promises in a bundle is followed, within each promise type, and the types are ordered according to normal ordering.
body common control
{
bundlesequence => { "order" };
}
bundle agent order
{
vars:
"list" slist => { "three", "four" };
commands:
ok_later::
"/bin/echo five";
otherthing::
"/bin/echo six";
any::
"/bin/echo one" classes => d("ok_later","otherthing");
"/bin/echo two";
"/bin/echo $(list)";
preserved_class::
"/bin/echo seven";
}
body classes d(if,else)
{
promise_repaired => { "$(if)" };
repair_failed => { "$(else)" };
persist_time => "0";
}
Aborting execution
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
body agent control
{
abortbundleclasses => { "invalid.Hr16" };
}
bundle agent testbundle
{
vars:
"userlist" slist => { "xyz", "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
bundle agent subtest(user)
{
classes:
"invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(user)");
reports:
!invalid::
"User name $(user) is valid at 4 letters";
invalid::
"User name $(user) is invalid";
}
Measuring Examples
Measurements
body common control
{
bundlesequence => { "report" };
}
body monitor control
{
forgetrate => "0.7";
histograms => "true";
}
bundle agent report
{
reports:
"
Free memory read at $(mon.av_free_memory_watch)
cf_monitord read $(mon.value_monitor_self_watch)
";
}
bundle monitor watch
{
measurements:
# Test 1 - extract string matching
"/home/mark/tmp/testmeasure"
handle => "blonk_watch",
stream_type => "file",
data_type => "string",
history_type => "weekly",
units => "blonks",
match_value => find_blonks,
action => sample_min("10");
# Test 2 - follow a special process over time
# using cfengine's process cache to avoid resampling
"/var/cfengine/state/cf_rootprocs"
handle => "monitor_self_watch",
stream_type => "file",
data_type => "int",
history_type => "static",
units => "kB",
match_value => proc_value(".*cf-monitord.*",
"root\s+[0-9.]+\s+[0-9.]+\s+[0-9.]+\s+[0-9.]+\s+([0-9]+).*");
# Test 3, discover disk device information
"/bin/df"
handle => "free_disk_watch",
stream_type => "pipe",
data_type => "slist",
history_type => "static",
units => "device",
match_value => file_system;
# Update this as often as possible
# Test 4
"/tmp/file"
handle => "line_counter",
stream_type => "file",
data_type => "counter",
match_value => scanlines("MYLINE.*"),
history_type => "log";
}
body match_value scanlines(x)
{
select_line_matching => "^$(x)$";
}
body action sample_min(x)
{
ifelapsed => "$(x)";
expireafter => "$(x)";
}
body match_value find_blonks
{
select_line_number => "2";
extraction_regex => "Blonk blonk ([blonk]+).*";
}
body match_value free_memory # not willy!
{
select_line_matching => "MemFree:.*";
extraction_regex => "MemFree:\s+([0-9]+).*";
}
body match_value proc_value(x,y)
{
select_line_matching => "$(x)";
extraction_regex => "$(y)";
}
body match_value file_system
{
select_line_matching => "/.*";
extraction_regex => "(.*)";
}
Software Administration Examples
- Software and patch installation
- Postfix mail configuration
- Set up a web server
- Add software packages to the system
- Application baseline
- Service management (windows)
- Software distribution
- Web server modules
- Ensure a service is enabled and running
- Managing Software
- Install packages
Software and patch installation
Example for Debian:
body common control
{
bundlesequence => { "packages" };
}
body agent control
{
environment => { "DEBIAN_FRONTEND=noninteractive" };
}
bundle agent packages
{
vars:
# Test the simplest case -- leave everything to the yum smart manager
"match_package" slist => {
"apache2"
# "apache2-mod_php5",
# "apache2-prefork",
# "php5"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => apt;
}
body package_method apt
{
any::
# ii acpi 0.09-3ubuntu1
package_changes => "bulk";
package_list_command => "/usr/bin/dpkg -l";
package_list_name_regex => "ii\s+([^\s]+).*";
package_list_version_regex => "ii\s+[^\s]+\s+([^\s]+).*";
# package_list_arch_regex => "none";
package_installed_regex => ".*"; # all reported are installed
#package_name_convention => "$(name)_$(version)_$(arch)";
package_name_convention => "$(name)";
# Use these only if not using a separate version/arch string
# package_version_regex => "";
# package_name_regex => "";
# package_arch_regex => "";
package_add_command => "/usr/bin/apt-get --yes install";
package_delete_command => "/usr/bin/apt-get --yes remove";
package_update_command => "/usr/bin/apt-get --yes dist-upgrade";
#package_verify_command => "/bin/rpm -V";
}
Examples MSI for Windows, by name:
body common control
{
bundlesequence => { "packages" };
}
bundle agent packages
{
vars:
"match_package" slist => {
"7zip"
};
packages:
"$(match_package)"
package_policy => "update",
package_select => ">=",
package_architectures => { "x86_64" },
package_version => "3.00",
package_method => msi_vmatch;
}
body package_method msi_vmatch
{
package_changes => "individual";
package_file_repositories => { "$(sys.workdir)\software_updates\windows", "s:\su" };
package_installed_regex => ".*";
package_name_convention => "$(name)-$(version)-$(arch).msi";
package_add_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_update_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_delete_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /x";
}
Windows MSI by version:
body common control
{
bundlesequence => { "packages" };
}
bundle agent packages
{
vars:
"match_package" slist => {
"7zip"
};
packages:
"$(match_package)"
package_policy => "update",
package_select => ">=",
package_architectures => { "x86_64" },
package_version => "3.00",
package_method => msi_vmatch;
}
body package_method msi_vmatch
{
package_changes => "individual";
package_file_repositories => { "$(sys.workdir)\software_updates\windows", "s:\su" };
package_installed_regex => ".*";
package_name_convention => "$(name)-$(version)-$(arch).msi";
package_add_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_update_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /i";
package_delete_command => "\"$(sys.winsysdir)\msiexec.exe\" /qn /x";
}
Examples for solaris:
bundle agent example_using_ips_package_method
{
packages:
solaris::
"shell/zsh"
package_policy => "add",
package_method => ips;
}
bundle agent example_using_solaris_package_method
{
files:
solaris::
"/tmp/$(admin_file)"
create => "true",
edit_defaults => empty_file, # defined in stdlib
edit_line => create_solaris_admin_file; # defined in stdlib
packages:
solaris::
"SMCzlib"
package_policy => "add",
package_method => solaris( "SMCzlib",
"zlib-1.2.3-sol10-sparc-local",
"$(admin_file)");
}
bundle agent example_using_solaris_install_package_method
{
packages:
solaris::
"SMCzlib"
package_method => solaris_install("/tmp/SMCzlib.adminfile")
}
bundle agent example_using_pkgsrc_module
{
packages:
solaris::
"vim"
policy => "present",
package_module => pkgsrc;
}
Examples for yum based systems:
body common control
{
bundlesequence => { "packages" };
inputs => { "cfengine_stdlib.cf" }
}
bundle agent packages
{
vars:
# Test the simplest case -- leave everything to the yum smart manager
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => yum;
}
SuSE Linux's package manager zypper is the most powerful alternative:
body common control
{
bundlesequence => { "packages" };
inputs => { "cfengine_stdlib.cf" }
}
bundle agent packages
{
vars:
# Test the simplest case -- leave everything to the zypper smart manager
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => zypper;
}
Postfix mail configuration
body common control
{
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { postfix };
}
bundle agent postfix
{
vars:
"prefix" string => "/etc";
"smtpserver" string => "localhost";
"mailrelay" string => "mailx.example.org";
files:
"$(prefix)/main.cf"
edit_line => prefix_postfix;
"$(prefix)/sasl-passwd"
create => "true",
perms => mo("0600","root"),
edit_line => append_if_no_line("$(smtpserver) _$(sys.fqhost):chmsxrcynz4etfrejizhs22");
}
bundle edit_line prefix_postfix
{
#
# Value have the form NAME = "quoted space separated list"
#
vars:
"ps[relayhost]" string => "[$(postfix.mailrelay)]:587";
"ps[mydomain]" string => "iu.hio.no";
"ps[smtp_sasl_auth_enable]" string => "yes";
"ps[smtp_sasl_password_maps]" string => "hash:/etc/postfix/sasl-passwd";
"ps[smtp_sasl_security_options]" string => "";
"ps[smtp_use_tls]" string => "yes";
"ps[default_privs]" string => "mailman";
"ps[inet_protocols]" string => "all";
"ps[inet_interfaces]" string => "127.0.0.1";
"parameter_name" slist => getindices("ps");
delete_lines:
"$(parameter_name).*";
insert_lines:
"$(parameter_name) = $(ps[$(parameter_name)])";
}
bundle edit_line AppendIfNSL(parameter)
{
insert_lines:
"$(parameter)"; # This is default
}
Set up a web server
Adapt this template to your operating system by adding multiple classes. Each web server runs something like the present module, which is entered into the bundlesequence like this:
bundle agent web_server(state)
{
vars:
"document_root" string => "/";
####################################################
# Site specific configuration - put it in this file
####################################################
"site_http_conf" string => "/home/mark/CFEngine-inputs/httpd.conf";
####################################################
# Software base
####################################################
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
#########################################################
processes:
web_ok.on::
"apache2"
restart_class => "start_apache";
off::
"apache2"
process_stop => "/etc/init.d/apache2 stop";
#########################################################
commands:
start_apache::
"/etc/init.d/apache2 start"; # or startssl
#########################################################
packages:
"$(match_package)"
package_policy => "add",
package_method => zypper,
classes => if_ok("software_ok");
#########################################################
files:
software_ok::
"/etc/sysconfig/apache2"
edit_line => fixapache,
classes => if_ok("web_ok");
#########################################################
reports:
!software_ok.on::
"The web server software could not be installed";
#########################################################
classes:
"on" expression => strcmp("$(state)","on");
"off" expression => strcmp("$(state)","off");
}
bundle edit_line fixapache
{
vars:
"add_modules" slist => {
"ssl",
"php5"
};
"del_modules" slist => {
"php3",
"php4",
"jk"
};
insert_lines:
"APACHE_CONF_INCLUDE_FILES=\"$(web_server.site_http_conf)\"";
field_edits:
#####################################################################
# APACHE_MODULES="actions alias ssl php5 dav_svn authz_default jk" etc..
#####################################################################
"APACHE_MODULES=.*"
# Insert module "columns" between the quoted RHS
# using space separators
edit_field => quotedvar("$(add_modules)","append");
"APACHE_MODULES=.*"
# Delete module "columns" between the quoted RHS
# using space separators
edit_field => quotedvar("$(del_modules)","delete");
# if this line already exists, edit it
}
Add software packages to the system
body common control
{
inputs => { "$(sys.libdir)/packages.cf" }
bundlesequence => { "packages" };
}
bundle agent packages
{
vars:
"match_package" slist => {
"apache2",
"apache2-mod_php5",
"apache2-prefork",
"php5"
};
packages:
solaris::
"$(match_package)"
package_policy => "add",
package_method => solaris;
redhat|SuSE::
"$(match_package)"
package_policy => "add",
package_method => yum_rpm;
methods:
# equivalent in 3.6, no OS choices
"" usebundle => ensure_present($(match_package));
}
Note you can also arrange to hide all the differences between package managers on an OS basis, but since some OSs have multiple managers, this might not be 100 percent correct.
Application baseline
bundle agent app_baseline
{
methods:
windows::
"any" usebundle => detect_adobereader;
}
bundle agent detect_adobereader
{
vars:
windows::
"value1" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "ENU_GUID");
"value2" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "VersionMax");
"value3" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Adobe\Acrobat Reader\9.0\Installer", "VersionMin");
classes:
windows::
"is_correct" and => {
strcmp($(value1), "{AC76BA86-7AD7-1033-7B44-A93000000001}"),
strcmp($(value2), "90003"),
islessthan($(value3), "10001" )
};
reports:
windows.!is_correct::
'Adobe Reader is not correctly deployed - got "$(value1)", "$(value2)", "$(value3)"';
}
Service management (windows)
body common control
{
bundlesequence => { "winservice" };
}
bundle agent winservice
{
vars:
"bad_services" slist => { "Alerter", "ClipSrv" };
services:
windows::
"$(bad_services)"
service_policy => "disable",
comment => "Disable services that create security issues";
}
Software distribution
bundle agent check_software
{
vars:
# software to install if not installed
"include_software" slist => {
"7-zip-4.50-$(sys.arch).msi"
};
# this software gets updated if it is installed
"autoupdate_software" slist => {
"7-zip"
};
# software to uninstall if it is installed
"exclude_software" slist => {
"7-zip-4.65-$(sys.arch).msi"
};
methods:
# "any" usebundle => add_software( "@(check_software.include_software)", "$(sys.policy_hub)" );
# "any" usebundle => update_software( "@(check_software.autoupdate_software)", "$(sys.policy_hub)" );
# "any" usebundle => remove_software( "@(check_software.exclude_software)", "$(sys.policy_hub)" );
}
bundle agent add_software(pkg_name)
{
vars:
# dir to install from locally - can also check multiple directories
"local_software_dir" string => "C:\Program Files\Cfengine\software\add";
files:
"$(local_software_dir)"
copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/add", "$(srv)"),
depth_search => recurse("1"),
classes => if_repaired("got_newpkg"),
comment => "Copy software from remote repository";
packages:
# When to check if the package is installed ?
got_newpkg|any::
"$(pkg_name)"
package_policy => "add",
package_method => msi_implicit( "$(local_software_dir)" ),
classes => if_else("add_success", "add_fail" ),
comment => "Install new software, if not already present";
reports::
add_fail::
"Failed to install one or more packages";
}
#########################################################################
bundle agent update_software(sw_names)
{
vars:
# dir to install from locally - can also check multiple directories
"local_software_dir" string => "C:\Program Files\Cfengine\software\update";
files:
"$(local_software_dir)"
copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/update", "$(srv)"),
depth_search => recurse("1"),
classes => if_repaired("got_newpkg"),
comment => "Copy software updates from remote repository";
packages:
# When to check if the package is updated ?
got_newpkg|any::
"$(sw_names)"
package_policy => "update",
package_select => ">=", # picks the newest update available
package_architectures => { "$(sys.arch)" }, # install 32 or 64 bit package ?
package_version => "1.0", # at least version 1.0
package_method => msi_explicit( "$(local_software_dir)" ),
classes => if_else("update_success", "update_fail");
reports:
update_fail::
"Failed to update one or more packages";
}
#########################################################################
bundle agent remove_software(pkg_name)
{
vars:
# dir to install from locally - can also check multiple directories
"local_software_dir" string => "C:\Program Files\Cfengine\software\remove";
files:
"$(local_software_dir)"
copy_from => remote_cp("/var/cfengine/master_software_updates/$(sys.flavour)_$(sys.arch)/remove", "$(srv)"),
depth_search => recurse("1"),
classes => if_repaired("got_newpkg"),
comment => "Copy removable software from remote repository";
packages:
got_newpkg::
"$(pkg_name)"
package_policy => "delete",
package_method => msi_implicit( "$(local_software_dir)" ),
classes => if_else("remove_success", "remove_fail" ),
comment => "Remove software, if present";
reports::
remove_fail::
"Failed to remove one or more packages";
}
Web server modules
The problem of editing the correct modules into the list of standard modules for the Apache web server. This example is based on the standard configuration deployment of SuSE Linux. Simply provide the list of modules you want and another list that you don't want.
body common control
{
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => {
apache
};
}
bundle agent apache
{
files:
SuSE::
"/etc/sysconfig/apache2"
edit_line => fixapache;
}
bundle edit_line fixapache
{
vars:
"add_modules" slist => {
"dav",
"dav_fs",
"ssl",
"php5",
"dav_svn",
"xyz",
"superduper"
};
"del_modules" slist => {
"php3",
"jk",
"userdir",
"imagemap",
"alias"
};
insert_lines:
"APACHE_CONF_INCLUDE_FILES=\"/site/masterfiles/local-http.conf\"";
field_edits:
#####################################################################
# APACHE_MODULES="authz_host actions alias ..."
#####################################################################
# Values have the form NAME = "quoted space separated list"
"APACHE_MODULES=.*"
# Insert module "columns" between the quoted RHS
# using space separators
edit_field => quoted_var($(add_modules), "append");
"APACHE_MODULES=.*"
# Delete module "columns" between the quoted RHS
# using space separators
edit_field => quoted_var($(del_modules), "delete");
# if this line already exists, edit it
}
Commands, Scripts, and Execution Examples
- Command or script execution
- Change directory for command
- Commands example
- Execresult example
- Methods
- Method validation
- Trigger classes
Command or script execution
Execute a command, for instance to start a MySQL service. Note that simple shell commands like rm or mkdir cannot be managed by CFEngine, so none of the protections that CFEngine offers can be applied to the process. Moreover, this starts a new process, adding to the burden on the system.
body common control
{
bundlesequence => { "my_commands" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_commands
{
commands:
Sunday.Hr04.Min05_10.myhost::
"/usr/bin/update_db";
any::
"/etc/mysql/start"
contain => setuid("mysql");
}
Change directory for command
body common control
{
bundlesequence => { "example" };
}
body contain cd(dir)
{
chdir => "${dir}";
useshell => "true";
}
bundle agent example
{
commands:
"/bin/pwd"
contain => cd("/tmp");
}
Commands example
body common control
{
bundlesequence => { "my_commands" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_commands
{
commands:
Sunday.Hr04.Min05_10.myhost::
"/usr/bin/update_db";
any::
"/etc/mysql/start"
contain => setuid("mysql");
}
Execresult example
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"my_result" string => execresult("/bin/ls /tmp","noshell");
reports:
"Variable is $(my_result)";
}
Methods
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
bundle agent testbundle
{
vars:
"userlist" slist => { "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
bundle agent subtest(user)
{
commands:
"/bin/echo Fix $(user)";
reports:
"Finished doing stuff for $(user)";
}
Method validation
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
body agent control
{
abortbundleclasses => { "invalid" };
}
bundle agent testbundle
{
vars:
"userlist" slist => { "xyz", "mark", "jeang", "jonhenrik", "thomas", "eben" };
methods:
"any" usebundle => subtest("$(userlist)");
}
bundle agent subtest(user)
{
classes:
"invalid" not => regcmp("[a-z][a-z][a-z][a-z]","$(user)");
reports:
!invalid::
"User name $(user) is valid at 4 letters";
invalid::
"User name $(user) is invalid";
}
Trigger classes
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" string => "
One potato
Two potato
Three potahto
Four
";
files:
"/tmp/test_insert"
edit_line => Insert("$(insert.v)"),
edit_defaults => empty,
classes => trigger("edited");
commands:
edited::
"/bin/echo make bananas";
reports:
edited::
"The potatoes are bananas";
}
bundle edit_line Insert(name)
{
insert_lines:
"Begin$(const.n) $(name)$(const.n)End";
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
body classes trigger(x)
{
promise_repaired => { $(x) };
}
File and Directory Examples
- Create files and directories
- Copy single files
- Copy directory trees
- Disabling and rotating files
- Add lines to a file
- Check file or directory permissions
- Commenting lines in a file
- Copy files
- Copy and flatten directory
- Copy then edit a file convergently
- Deleting lines from a file
- Deleting lines exception
- Editing files
- Editing tabular files
- Inserting lines in a file
- Back references in filenames
- Add variable definitions to a file
- Linking files
- Listing files-pattern in a directory
- Locate and transform files
- BSD flags
- Search and replace text
- Selecting a region in a file
- Warn if matching line in file
Create files and directories
Create files and directories and set permissions.
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/test_plain"
perms => system,
create => "true";
"/home/mark/tmp/test_dir/."
perms => system,
create => "true";
}
body perms system
{
mode => "0640";
}
Copy single files
Copy single files, locally (local_cp) or from a remote site (secure_cp). The Community Open Promise-Body Library (COPBL; cfengine_stdlib.cf) should be included in the /var/cfengine/inputs/ directory and input as below.
body common control
{
bundlesequence => { "mycopy" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent mycopy
{
files:
"/home/mark/tmp/test_plain"
copy_from => local_cp("$(sys.workdir)/bin/file");
"/home/mark/tmp/test_remote_plain"
copy_from => secure_cp("$(sys.workdir)/bin/file","serverhost");
}
Copy directory trees
Copy directory trees, locally (local_cp) or from a remote site (secure_cp). (depth_search => recurse("")) defines the number of sublevels to include, ("inf") gets entire tree.
body common control
{
bundlesequence => { "my_recursive_copy" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_recursive_copy
{
files:
"/home/mark/tmp/test_dir"
copy_from => local_cp("$(sys.workdir)/bin/."),
depth_search => recurse("inf");
"/home/mark/tmp/test_dir"
copy_from => secure_cp("$(sys.workdir)/bin","serverhost"),
depth_search => recurse("inf");
}
Disabling and rotating files
Use the following simple steps to disable and rotate files. See the Community Open Promise-Body Library if you wish more details on what disable and rotate does.
body common control
{
bundlesequence => { "my_disable" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent my_disable
{
files:
"/home/mark/tmp/test_create"
rename => disable;
"/home/mark/tmp/rotate_my_log"
rename => rotate("4");
}
Add lines to a file
There are numerous approaches to adding lines to a file. Often the order of a configuration file is unimportant, we just need to ensure settings within it. A simple way of adding lines is show below.
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"lines" string =>
"
One potato
Two potato
Three potatoe
Four
";
files:
"/tmp/test_insert"
create => "true",
edit_line => append_if_no_line("$(insert.lines)");
}
Also you could write this using a list variable:
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"lines" slist => { "One potato", "Two potato",
"Three potatoe", "Four" };
files:
"/tmp/test_insert"
create => "true",
edit_line => append_if_no_line("@(insert.lines)");
}
Check file or directory permissions
bundle agent check_perms
{
vars:
"ns_files" slist => {
"/local/iu/logs/admin",
"/local/iu/logs/security",
"/local/iu/logs/updates",
"/local/iu/logs/xfer"
};
files:
NameServers::
"/local/dns/pz"
perms => mo("644","dns"),
depth_search => recurse("1"),
file_select => exclude("secret_file");
"/local/iu/dns/pz/FixSerial"
perms => m("755"),
file_select => plain;
"$(ns_files)"
perms => mo("644","dns"),
file_select => plain;
"$(ftp)/pub"
perms => mog("644","root","other");
"$(ftp)/pub"
perms => m("644"),
depth_search => recurse("inf");
"$(ftp)/etc" perms => mog("111","root","other");
"$(ftp)/usr/bin/ls" perms => mog("111","root","other");
"$(ftp)/dev" perms => mog("555","root","other");
"$(ftp)/usr" perms => mog("555","root","other");
}
Commenting lines in a file
body common control
{
version => "1.2.3";
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/cf3_test"
create => "true",
edit_line => myedit("second");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
replace_patterns:
# replace shell comments with C comments
"#(.*)"
replace_with => C_comment,
select_region => MySection("New section");
}
body replace_with C_comment
{
replace_value => "/* $(match.1) */"; # backreference 0
occurrences => "all"; # first, last all
}
body select_region MySection(x)
{
select_start => "\[$(x)\]";
select_end => "\[.*\]";
}
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/comment_test"
create => "true",
edit_line => comment_lines_matching;
}
bundle edit_line comment_lines_matching
{
vars:
"regexes" slist => { "one.*", "two.*", "four.*" };
replace_patterns:
"^($(regexes))$"
replace_with => comment("# ");
}
body replace_with comment(c)
{
replace_value => "$(c) $(match.1)";
occurrences => "all";
}
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/comment_test"
create => "true",
edit_line => uncomment_lines_matching("\s*mark.*","#");
}
bundle edit_line uncomment_lines_matching(regex,comment)
{
replace_patterns:
"#($(regex))$" replace_with => uncomment;
}
body replace_with uncomment
{
replace_value => "$(match.1)";
occurrences => "all";
}
Copy files
files:
"/var/cfengine/inputs"
handle => "update_policy",
perms => m("600"),
copy_from => u_scp("$(master_location)",@(policy_server)),
depth_search => recurse("inf"),
file_select => input_files,
action => immediate;
"/var/cfengine/bin"
perms => m("700"),
copy_from => u_scp("/usr/local/sbin","localhost"),
depth_search => recurse("inf"),
file_select => cf3_files,
action => immediate,
classes => on_change("reload");
Copy and flatten directory
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
}
bundle agent testbundle
{
files:
"/home/mark/tmp/testflatcopy"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/words","127.0.0.1"),
perms => system,
depth_search => recurse("inf"),
classes => satisfied("copy_ok");
"/home/mark/tmp/testcopy/single_file"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/Cfengine3/trunk/README","127.0.0.1"),
perms => system;
reports:
copy_ok::
"Files were copied..";
}
body perms system
{
mode => "0644";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
body copy_from mycopy(from,server)
{
source => "$(from)";
servers => { "$(server)" };
compare => "digest";
verify => "true";
copy_backup => "true"; #/false/timestamp
purge => "false";
type_check => "true";
force_ipv4 => "true";
trustkey => "true";
collapse_destination_dir => "true";
}
body classes satisfied(x)
{
promise_repaired => { "$(x)" };
persist_time => "0";
}
body server control
{
allowconnects => { "127.0.0.1" , "::1" };
allowallconnects => { "127.0.0.1" , "::1" };
trustkeysfrom => { "127.0.0.1" , "::1" };
}
bundle server access_rules()
{
access:
"/home/mark/LapTop"
admit => { "127.0.0.1" };
}
Copy then edit a file convergently
To convergently chain a copy followed by edit, you need a staging file. First you copy to the staging file. Then you edit the final file and insert the staging file into it as part of the editing. This is convergent with respect to both stages of the process.
bundle agent master
{
files:
"$(final_destination)"
create => "true",
edit_line => fix_file("$(staging_file)"),
edit_defaults => empty,
perms => mo("644","root"),
action => ifelapsed("60");
}
bundle edit_line fix_file(f)
{
insert_lines:
"$(f)"
# insert this into an empty file to reconstruct
insert_type => "file";
replace_patterns:
"searchstring"
replace_with => With("replacestring");
}
Deleting lines from a file
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
files:
"/tmp/resolv.conf" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => resolver,
edit_defaults => def;
}
bundle edit_line resolver
{
vars:
"search" slist => { "search iu.hio.no cfengine.com", "nameserver 128.39.89.10" };
delete_lines:
"search.*";
insert_lines:
"$(search)" location => end;
}
body edit_defaults def
{
empty_file_before_editing => "false";
edit_backup => "false";
max_file_size => "100000";
}
body location start
{
# If not line to match, applies to whole text body
before_after => "before";
}
body location end
{
# If not line to match, applies to whole text body
before_after => "after";
}
Deleting lines exception
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/passwd_excerpt"
create => "true",
edit_line => MarkNRoot;
}
bundle edit_line MarkNRoot
{
delete_lines:
"mark.*|root.*" not_matching => "true";
}
Editing files
This is a huge topic. See also See Add lines to a file, See Editing tabular files, etc. Editing a file can be complex or simple, depending on needs.
Here is an example of how to comment out lines matching a number of patterns:
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent testbundle
{
vars:
"patterns" slist => { "finger.*", "echo.*", "exec.*", "rstat.*",
"uucp.*", "talk.*" };
files:
"/etc/inetd.conf"
edit_line => comment_lines_matching("@(testbundle.patterns)","#");
}
Editing tabular files
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
vars:
"userset" slist => { "one-x", "two-x", "three-x" };
files:
# Make a copy of the password file
"/home/mark/tmp/passwd"
create => "true",
edit_line => SetUserParam("mark","6","/set/this/shell");
"/home/mark/tmp/group"
create => "true",
edit_line => AppendUserParam("root","4","@(userset)");
commands:
"/bin/echo" args => $(userset);
}
bundle edit_line SetUserParam(user,field,val)
{
field_edits:
"$(user):.*"
# Set field of the file to parameter
edit_field => col(":","$(field)","$(val)","set");
}
bundle edit_line AppendUserParam(user,field,allusers)
{
vars:
"val" slist => { @(allusers) };
field_edits:
"$(user):.*"
# Set field of the file to parameter
edit_field => col(":","$(field)","$(val)","alphanum");
}
body edit_field col(split,col,newval,method)
{
field_separator => $(split);
select_field => $(col);
value_separator => ",";
field_value => $(newval);
field_operation => $(method);
extend_fields => "true";
}
Inserting lines in a file
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" string => " One potato";
files:
"/tmp/test_insert"
create => "true",
edit_line => Insert("$(insert.v)");
}
bundle edit_line Insert(name)
{
insert_lines:
" $(name)"
whitespace_policy => { "ignore_leading", "ignore_embedded" };
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" string => "
One potato
Two potato
Three potatoe
Four
";
files:
"/tmp/test_insert"
create => "true",
edit_line => Insert("$(insert.v)"),
edit_defaults => empty;
}
bundle edit_line Insert(name)
{
insert_lines:
"Begin$(const.n)$(name)$(const.n)End";
}
body edit_defaults empty
{
empty_file_before_editing => "false";
}
body common control
{
any::
bundlesequence => { "insert" };
}
bundle agent insert
{
vars:
"v" slist => {
"One potato",
"Two potato",
"Three potatoe",
"Four"
};
files:
"/tmp/test_insert"
create => "true",
edit_line => Insert("@(insert.v)");
# edit_defaults => empty;
}
bundle edit_line Insert(name)
{
insert_lines:
"$(name)";
}
body edit_defaults empty
{
empty_file_before_editing => "true";
}
Back references in filenames
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
# The back reference in a path only applies to the last link
# of the pathname, so the (tmp) gets ignored
"/tmp/(cf3)_(.*)"
edit_line => myedit("second $(match.2)");
# but ...
# "/tmp/cf3_test"
# create => "true",
# edit_line => myedit("second $(match.1)");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
insert_lines:
"$(edit_variable)";
}
Add variable definitions to a file
body common control
{
bundlesequence => { "setvars" };
inputs => { "cf_std_library.cf" };
}
bundle agent setvars
{
vars:
# want to set these values by the names of their array keys
"rhs[lhs1]" string => " Mary had a little pig";
"rhs[lhs2]" string => "Whose Fleece was white as snow";
"rhs[lhs3]" string => "And everywhere that Mary went";
# oops, now change pig -> lamb
files:
"/tmp/system"
create => "true",
edit_line => set_variable_values("setvars.rhs");
}
Results in:
- lhs1= Mary had a little pig
- lhs2=Whose Fleece was white as snow
- lhs3=And everywhere that Mary went
An example of this would be to add variables to /etc/sysctl.conf on Linux:
body common control
{
bundlesequence => { "setvars" };
inputs => { "cf_std_library.cf" };
}
bundle agent setvars
{
vars:
# want to set these values by the names of their array keys
"rhs[net/ipv4/tcp_syncookies]" string => "1";
"rhs[net/ipv4/icmp_echo_ignore_broadcasts]" string => "1";
"rhs[net/ipv4/ip_forward]" string => "1";
# oops, now change pig -> lamb
files:
"/etc/sysctl"
create => "true",
edit_line => set_variable_values("setvars.rhs");
}
Linking files
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
# Make a copy of the password file
"/home/mark/tmp/passwd"
link_from => linkdetails("/etc/passwd"),
move_obstructions => "true";
"/home/mark/tmp/linktest"
link_from => linkchildren("/usr/local/sbin");
#child links
}
body link_from linkdetails(tofile)
{
source => "$(tofile)";
link_type => "symlink";
when_no_source => "force"; # kill
}
body link_from linkchildren(tofile)
{
source => "$(tofile)";
link_type => "symlink";
when_no_source => "force"; # kill
link_children => "true";
when_linking_children => "if_no_such_file"; # "override_file";
}
body common control
{
any::
bundlesequence => {
"testbundle"
};
}
bundle agent testbundle
{
files:
"/home/mark/tmp/test_to" -> "someone"
depth_search => recurse("inf"),
perms => modestuff,
action => tell_me;
}
body depth_search recurse(d)
{
rmdeadlinks => "true";
depth => "$(d)";
}
body perms modestuff
{
mode => "o-w";
}
body action tell_me
{
report_level => "inform";
}
Listing files-pattern in a directory
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"ls" slist => lsdir("/etc","p.*","true");
reports:
"ls: $(ls)";
}
Locate and transform files
body common control
{
any::
bundlesequence => {
"testbundle"
};
version => "1.2.3";
}
bundle agent testbundle
{
files:
"/home/mark/tmp/testcopy"
file_select => pdf_files,
transformer => "/usr/bin/gzip $(this.promiser)",
depth_search => recurse("inf");
}
body file_select pdf_files
{
leaf_name => { ".*.pdf" , ".*.fdf" };
file_result => "leaf_name";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
BSD flags
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
files:
freebsd::
"/tmp/newfile"
create => "true",
perms => setbsd;
}
body perms setbsd
{
bsdflags => { "+uappnd","+uchg", "+uunlnk", "-nodump" };
}
Search and replace text
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/replacestring"
create => "true",
edit_line => myedit("second");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
replace_patterns:
# replace shell comments with C comments
"puppet"
replace_with => With("cfengine 3");
}
body replace_with With(x)
{
replace_value => $(x);
occurrences => "first";
}
body select_region MySection(x)
{
select_start => "\[$(x)\]";
select_end => "\[.*\]";
}
Selecting a region in a file
body common control
{
version => "1.2.3";
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/testfile"
create => "true",
edit_line => myedit("second");
}
bundle edit_line myedit(parameter)
{
vars:
"edit_variable" string => "private edit variable is $(parameter)";
replace_patterns:
# comment out lines after start
"([^#].*)"
replace_with => comment,
select_region => ToEnd("Start.*");
}
body replace_with comment
{
replace_value => "# $(match.1)"; # backreference 0
occurrences => "all"; # first, last all
}
body select_region ToEnd(x)
{
select_start => $(x);
}
Warn if matching line in file
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/var/cfengine/inputs/.*"
edit_line => DeleteLinesMatching(".*cfenvd.*"),
action => WarnOnly;
}
bundle edit_line DeleteLinesMatching(regex)
{
delete_lines:
"$(regex)" action => WarnOnly;
}
body action WarnOnly
{
action_policy => "warn";
}
File Template Examples
Templating
With CFEngine you have a choice between editing `deltas' into files or distributing more-or-less finished templates. Which method you should choose depends should be made by whatever is easiest.
If you are managing only part of the file, and something else (e.g. a package manager) is managing most of it, then it makes sense to use CFEngine file editing.
If you are managing everything in the file, then it makes sense to make the edits by hand and install them using CFEngine. You can use variables within source text files and let CFEngine expand them locally in situ, so that you can make generic templates that apply netwide.
Example template:
MYVARIABLE = something or other
HOSTNAME = $(sys.host) # CFEngine fills this in
To copy and expand this template, you can use a pattern like this:
bundle agent get_template(final_destination,mode)
{
vars:
# This needs to ne preconfigured to your site
"masterfiles" string => "/home/mark/tmp";
"this_template" string => lastnode("$(final_destination)","/");
files:
"$(final_destination).staging"
comment => "Get template and expand variables for this host",
perms => mo("400","root"),
copy_from => remote_cp("$(masterfiles)/templates/$(this_template)","$(policy_server)"),
action => if_elapsed("60");
"$(final_destination)"
comment => "Expand the template",
create => "true",
edit_line => expand_template("$(final_destination).staging"),
edit_defaults => empty,
perms => mo("$(mode)","root"),
action => if_elapsed("60");
}
The the following driving code (based on `copy then edit') can be placed in a library, after configuring to your environmental locations:
bundle agent get_template(final_destination,mode)
{
vars:
# This needs to ne preconfigured to your site
"masterfiles" string => "/home/mark/tmp";
"this_template" string => lastnode("$(final_destination)","/");
files:
"$(final_destination).staging"
comment => "Get template and expand variables for this host",
perms => mo("400","root"),
copy_from => remote_cp("$(masterfiles)/templates/$(this_template)","$(policy_server)"),
action => if_elapsed("60");
"$(final_destination)"
comment => "Expand the template",
create => "true",
edit_line => expand_template("$(final_destination).staging"),
edit_defaults => empty,
perms => mo("$(mode)","root"),
action => if_elapsed("60");
}
Database Examples
Database creation
body common control
{
bundlesequence => { "dummy" };
}
body knowledge control
{
#sql_database => "postgres";
sql_owner => "postgres";
sql_passwd => ""; # No passwd
sql_type => "postgres";
}
bundle knowledge dummy
{
topics:
}
body common control
{
bundlesequence => { "databases" };
}
bundle agent databases
{
#commands:
# "/usr/bin/createdb cf_topic_maps",
# contain => as_user("mysql");
databases:
"knowledge_bank/topics"
database_operation => "create",
database_type => "sql",
database_columns => {
"topic_name,varchar,256",
"topic_comment,varchar,1024",
"topic_id,varchar,256",
"topic_type,varchar,256",
"topic_extra,varchar,26"
},
database_server => myserver;
}
body database_server myserver
{
none::
db_server_owner => "postgres";
db_server_password => "";
db_server_host => "localhost";
db_server_type => "postgres";
db_server_connection_db => "postgres";
any::
db_server_owner => "root";
db_server_password => "";
db_server_host => "localhost";
db_server_type => "mysql";
db_server_connection_db => "mysql";
}
body contain as_user(x)
{
exec_owner => "$(x)";
}
Network Examples
- Find MAC address
- Client-server example
- Read from a TCP socket
- Set up a PXE boot server
- Resolver management
- Mount NFS filesystem
- Unmount NFS filesystem
- Find the MAC address
- Mount NFS filesystem
Find MAC address
Finding the ethernet address can be hard, but on Linux it is straightforward.
bundle agent test
{
vars:
linux::
"interface" string => execresult("/sbin/ifconfig eth0","noshell");
solaris::
"interface" string => execresult("/usr/sbin/ifconfig bge0","noshell");
freebsd::
"interface" string => execresult("/sbin/ifconfig le0","noshell");
darwin::
"interface" string => execresult("/sbin/ifconfig en0","noshell");
classes:
linux::
"ok" expression => regextract(
".*HWaddr ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
solaris::
"ok" expression => regextract(
".*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
freebsd::
"ok" expression => regextract(
".*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
darwin::
"ok" expression => regextract(
"(?s).*ether ([^\s]+).*(\n.*)*",
"$(interface)",
"mac"
);
reports:
ok::
"MAC address is $(mac[1])";
}
Client-server example
body common control
{
bundlesequence => { "testbundle" };
version => "1.2.3";
#fips_mode => "true";
}
bundle agent testbundle
{
files:
"/home/mark/tmp/testcopy"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/words","127.0.0.1"),
perms => system,
depth_search => recurse("inf"),
classes => satisfied("copy_ok");
"/home/mark/tmp/testcopy/single_file"
comment => "test copy promise",
copy_from => mycopy("/home/mark/LapTop/Cfengine3/trunk/README","127.0.0.1"),
perms => system;
reports:
copy_ok::
"Files were copied..";
}
body perms system
{
mode => "0644";
}
body depth_search recurse(d)
{
depth => "$(d)";
}
body copy_from mycopy(from,server)
{
source => "$(from)";
servers => { "$(server)" };
compare => "digest";
encrypt => "true";
verify => "true";
copy_backup => "true"; #/false/timestamp
purge => "false";
type_check => "true";
force_ipv4 => "true";
trustkey => "true";
}
body classes satisfied(x)
{
promise_repaired => { "$(x)" };
persist_time => "0";
}
body server control
{
allowconnects => { "127.0.0.1" , "::1" };
allowallconnects => { "127.0.0.1" , "::1" };
trustkeysfrom => { "127.0.0.1" , "::1" };
# allowusers
}
bundle server access_rules()
{
access:
"/home/mark/LapTop"
admit => { "127.0.0.1" };
}
Read from a TCP socket
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"my80" string => readtcp("research.iu.hio.no","80","GET /index.php HTTP/1.1$(const.r)$(const.n)Host: research.iu.hio.no$(const.r)$(const.n)$(const.r)$(const.n)",20);
classes:
"server_ok" expression => regcmp(".*200 OK.*\n.*","$(my80)");
reports:
server_ok::
"Server is alive";
!server_ok::
"Server is not responding - got $(my80)";
}
Set up a PXE boot server
Use CFEngine to set up a PXE boot server.
body common control
{
bundlesequence => { "pxe" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent pxe
{
vars:
"software" slist => {
"atftp",
"dhcp-server",
"syslinux",
"apache2"
};
"dirs" slist => {
"/tftpboot",
"/tftpboot/CFEngine/rpm",
"/tftpboot/CFEngine/inputs",
"/tftpboot/pxelinux.cfg",
"/tftpboot/kickstart",
"/srv/www/repos"
};
"tmp_location" string => "/tftpboot/CFEngine/inputs";
# Distros that we can install
"rh_distros" slist => { "4.7", "5.2" };
"centos_distros" slist => { "5.2" };
# File contents of atftp configuration
"atftpd_conf" string =>
"
ATFTPD_OPTIONS=\"--daemon \"
ATFTPD_USE_INETD=\"no\"
ATFTPD_DIRECTORY=\"/tftpboot\"
ATFTPD_BIND_ADDRESSES=\"\"
";
# File contents of DHCP configuration
"dhcpd" string =>
"
DHCPD_INTERFACE=\"eth0\"
DHCPD_RUN_CHROOTED=\"yes\"
DHCPD_CONF_INCLUDE_FILES=\"\"
DHCPD_RUN_AS=\"dhcpd\"
DHCPD_OTHER_ARGS=\"\"
DHCPD_BINARY=\"\"
";
"dhcpd_conf" string =>
"
allow booting;
allow bootp;
ddns-update-style none; ddns-updates off;
subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.20 192.168.0.254;
default-lease-time 3600;
max-lease-time 4800;
option routers 192.168.0.1;
option domain-name \"test.CFEngine.com\";
option domain-name-servers 192.168.0.1;
next-server 192.168.0.1;
filename \"pxelinux.0\";
}
group {
host node1 {
# Dummy machine
hardware ethernet 00:0F:1F:94:FE:07;
fixed-address 192.168.0.11;
option host-name \"node1\";
}
host node2 {
# Dell Inspiron 1150
hardware ethernet 00:0F:1F:0E:70:E7;
fixed-address 192.168.0.12;
option host-name \"node2\";
}
}
";
# File contains of Apache2 HTTP configuration
"httpd_conf" string =>
"
<Directory /srv/www/repos>
Options Indexes
AllowOverride None
</Directory>
Alias /repos /srv/www/repos
<Directory /tftpboot/distro/RHEL/5.2>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/rhel/5.2 /tftpboot/distro/RHEL/5.2
<Directory /tftpboot/distro/RHEL/4.7>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/rhel/4.7 /tftpboot/distro/RHEL/4.7
<Directory /tftpboot/distro/CentOS/5.2>
Options Indexes
AllowOverride None
</Directory>
Alias /distro/centos/5.2 /tftpboot/distro/CentOS/5.2
<Directory /tftpboot/kickstart>
Options Indexes
AllowOverride None
</Directory>
Alias /kickstart /tftpboot/kickstart
<Directory /tftpboot/CFEngine>
Options Indexes
AllowOverride None
</Directory>
Alias /CFEngine /tftpboot/CFEngine
";
# File contains of Kickstart for RHEL5 configuration
"kickstart_rhel5_conf" string =>
"
auth --useshadow --enablemd5
bootloader --location=mbr
clearpart --all --initlabel
graphical
firewall --disabled
firstboot --disable
key 77244a6377a8044a
keyboard no
lang en_US
logging --level=info
url --url=http://192.168.0.1/distro/rhel/5.2
network --bootproto=dhcp --device=eth0 --onboot=on
reboot
rootpw --iscrypted $1$eOnXdDPF$279sQ//zry6rnQktkATeM0
selinux --disabled
timezone --isUtc Europe/Oslo
install
part swap --bytes-per-inode=4096 --fstype=\"swap\" --recommended
part / --bytes-per-inode=4096 --fstype=\"ext3\" --grow --size=1
%packages
@core
@base
db4-devel
openssl-devel
gcc
flex
bison
libacl-devel
libselinux-devel
pcre-devel
device-mapper-multipath
-sysreport
%post
cd /root
rpm -i http://192.168.0.1/CFEngine/rpm/CFEngine-3.0.1b1-1.el5.i386.rpm
cd /etc/yum.repos.d
wget http://192.168.0.1/repos/RHEL5.Base.repo
rpm --import /etc/pki/rpm-gpg/*
yum clean all
yum update
mkdir -p /root/CFEngine_init
cd /root/CFEngine_init
wget -nd -r http://192.168.0.1/CFEngine/inputs/
/usr/local/sbin/cf-agent -B
/usr/local/sbin/cf-agent
";
# File contains of PXElinux boot menu
"pxelinux_boot_menu" string =>
"
boot options:
rhel5 - install 32 bit i386 RHEL 5.2 (MANUAL)
rhel5w - install 32 bit i386 RHEL 5.2 (AUTO)
rhel4 - install 32 bit i386 RHEL 4.7 AS (MANUAL)
centos5 - install 32 bit i386 CentOS 5.2 (Desktop) (MANUAL)
";
# File contains of PXElinux default configuration
"pxelinux_default" string =>
"
default rhel5
timeout 300
prompt 1
display pxelinux.cfg/boot.msg
F1 pxelinux.cfg/boot.msg
label rhel5
kernel vmlinuz-RHEL5U2
append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/rhel/5.2
label rhel5w
kernel vmlinuz-RHEL5U2
append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 ks=http://192.168.0.1/kickstart/kickstart-RHEL5U2.cfg
label rhel4
kernel vmlinuz-RHEL4U7
append initrd=initrd-RHEL4U7 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/rhel/4.7
label centos5
kernel vmlinuz-CentOS5.2
append initrd=initrd-CentOS5.2 load_ramdisk=1 ramdisk_size=16384 install=http://192.168.0.1/distro/centos/5.2
";
# File contains of specified PXElinux default to be a RHEL5 webserver
"pxelinux_rhel5_webserver" string =>
"
default rhel5w
label rhel5w
kernel vmlinuz-RHEL5U2
append initrd=initrd-RHEL5U2 load_ramdisk=1 ramdisk_size=16384 ks=http://192.168.0.1/kickstart/kickstart-RHEL5U2.cfg
";
# File contains of a local repository for RHEL5
"rhel5_base_repo" string =>
"
[Server]
name=Server
baseurl=http://192.168.0.1/repos/rhel5/Server/
enable=1
[VT]
name=VT
baseurl=http://192.168.0.1/repos/rhel5/VT/
enable=1
[Cluster]
name=Cluster
baseurl=http://192.168.0.1/repos/rhel5/Cluster/
enable=1
[ClusterStorage]
name=Cluster Storage
baseurl=http://192.168.0.1/repos/rhel5/ClusterStorage/
enable=1
";
#####################################################
files:
packages_ok::
# Create files/dirs and edit the new files
"/tftpboot/distro/RHEL/$(rh_distros)/."
create => "true";
"/tftpboot/distro/CentOS/$(centos_distros)/."
create => "true";
"$(dirs)/."
create => "true";
"/tftpboot/pxelinux.cfg/boot.msg"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(pxelinux_boot_menu)"),
edit_defaults => empty;
"/tftpboot/pxelinux.cfg/default"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(pxelinux_default)"),
edit_defaults => empty;
"/tftpboot/pxelinux.cfg/default.RHEL5.webserver"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(pxelinux_rhel5_webserver)"),
edit_defaults => empty;
"/tftpboot/kickstart/kickstart-RHEL5U2.cfg"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(kickstart_rhel5_conf)"),
edit_defaults => empty;
"/srv/www/repos/RHEL5.Base.repo"
create => "true",
perms => mo("644","root"),
edit_line => append_if_no_line("$(rhel5_base_repo)"),
edit_defaults => empty;
# Copy files
"/tftpboot"
copy_from => local_cp("/usr/share/syslinux"),
depth_search => recurse("inf"),
file_select => pxelinux_files,
action => immediate;
"$(tmp_location)"
perms => m("644"),
copy_from => local_cp("/var/cfengine/inputs"),
depth_search => recurse("inf"),
file_select => input_files,
action => immediate;
# Edit atftp, dhcp and apache2 configurations
"/etc/sysconfig/atftpd"
edit_line => append_if_no_line("$(atftpd_conf)"),
edit_defaults => empty,
classes => satisfied("atftpd_ready");
"/etc/sysconfig/dhcpd"
edit_line => append_if_no_line("$(dhcpd)"),
edit_defaults => empty;
"/etc/dhcpd.conf"
edit_line => append_if_no_line("$(dhcpd_conf)"),
edit_defaults => empty,
classes => satisfied("dhcpd_ready");
"/etc/apache2/httpd.conf"
edit_line => append_if_no_line("$(httpd_conf)"),
edit_defaults => std_defs,
classes => satisfied("apache2_ok");
# Make a static link
"/tftpboot/pxelinux.cfg/C0A8000C"
link_from => mylink("/tftpboot/pxelinux.cfg/default.RHEL5.webserver");
# Hash comment some lines for apaches
apache2_ok::
"/etc/apache2/httpd.conf"
edit_line => comment_lines_matching_apache2("#"),
classes => satisfied("apache2_ready");
commands:
# Restart services
atftpd_ready::
"/etc/init.d/atftpd restart";
dhcpd_ready::
"/etc/init.d/dhcpd restart";
apache2_ready::
"/etc/init.d/apache2 restart";
#####################################################
packages:
ipv4_192_168_0_1::
# Only the PXE boot server
"$(software)"
package_policy => "add",
package_method => zypper,
classes => satisfied("packages_ok");
}
body file_select pxelinux_files
{
leaf_name => { "pxelinux.0" };
file_result => "leaf_name";
}
body copy_from mycopy_local(from,server)
{
source => "$(from)";
compare => "digest";
}
body link_from mylink(x)
{
source => "$(x)";
link_type => "symlink";
}
body classes satisfied(new_class)
{
promise_kept => { "$(new_class)"};
promise_repaired => { "$(new_class)"};
}
bundle edit_line comment_lines_matching_apache2(comment)
{
vars:
"regex" slist => { "\s.*Options\sNone", "\s.*AllowOverride\sNone", "\s.*Deny\sfrom\sall" };
replace_patterns:
"^($(regex))$"
replace_with => comment("$(comment)");
}
body file_select input_files
{
leaf_name => { ".*.cf",".*.dat",".*.txt" };
file_result => "leaf_name";
}
Resolver management
bundle common g # globals
{
vars:
"searchlist" slist => {
"search iu.hio.no",
"search cfengine.com"
};
"nameservers" slist => {
"128.39.89.10",
"128.39.74.16",
"192.168.1.103"
};
classes:
"am_name_server" expression => reglist("@(nameservers)","$(sys.ipv4[eth1])");
}
body common control
{
any::
bundlesequence => {
"g",
resolver(@(g.searchlist),@(g.nameservers))
};
domain => "iu.hio.no";
}
bundle agent resolver(s,n)
{
files:
# When passing parameters down, we have to refer to
# a source context
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => doresolv("@(this.s)","@(this.n)"),
edit_defaults => reconstruct;
# or edit_defaults => modify
}
bundle edit_line doresolv(s,n)
{
vars:
"line" slist => { @(s), @(n) };
insert_lines:
"$(line)";
}
body edit_defaults reconstruct
{
empty_file_before_editing => "true";
edit_backup => "false";
max_file_size => "100000";
}
body edit_defaults modify
{
empty_file_before_editing => "false";
edit_backup => "false";
max_file_size => "100000";
}
Mount NFS filesystem
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
"/mnt" mount => nfs("slogans.iu.hio.no","/home");
}
body mount nfs(server,source)
{
mount_type => "nfs";
mount_source => "$(source)";
mount_server => "$(server)";
#mount_options => { "rw" };
edit_fstab => "true";
unmount => "true";
}
Unmount NFS filesystem
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
# Assumes the filesystem has been exported
"/mnt" mount => nfs("server.example.org","/home");
}
body mount nfs(server,source)
{
mount_type => "nfs";
mount_source => "$(source)";
mount_server => "$(server)";
edit_fstab => "true";
unmount => "true";
}
System Security Examples
- Distribute root passwords
- Distribute ssh keys
- Distribute ssh keys
Distribute root passwords
body common control
{
version => "1.2.3";
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { "SetRootPassword" };
}
bundle common g
{
vars:
"secret_keys_dir" string => "/tmp";
}
bundle agent SetRootPassword
{
vars:
# Or get variables directly from server with Enterprise
"remote-passwd" string => remotescalar("rem_password","127.0.0.1","yes");
# Test this on a copy
files:
"/var/cfengine/ppkeys/rootpw.txt"
copy_from => secure_cp("$(sys.fqhost)-root.txt","master_host.example.org");
# or $(pw_class)-root.txt
"/tmp/shadow"
edit_line => SetRootPw;
}
bundle edit_line SetRootPw
{
vars:
# Assume this file contains a single string of the form root:passwdhash:
# with : delimiters to avoid end of line/file problems
"pw" int => readstringarray("rpw","$(sys.workdir)/ppkeys/rootpw.txt",
"#[^\n]*",":","1","200");
field_edits:
"root:.*"
# Set field of the file to parameter
edit_field => col(":","2","$(rpw[root][1])","set");
}
bundle server passwords
{
vars:
# Read a file of format
#
# classname: host1,host2,host4,IP-address,regex.*,etc
#
"pw_classes" int => readstringarray("acl","$(g.secret_keys_dir)/classes.txt",
"#[^\n]*",":","100","4000");
"each_pw_class" slist => getindices("acl");
access:
"/secret/keys/$(each_pw_class)-root.txt"
admit => splitstring("$(acl[$(each_pw_class)][1])" , ":" , "100"),
ifencrypted => "true";
}
Distribute ssh keys
bundle agent allow_ssh_rootlogin_from_authorized_keys(user,sourcehost)
{
vars:
"local_cache" string => "/var/cfengine/ssh_cache";
"authorized_source" string => "/master/CFEngine/ssh_keys";
files:
"$(local_cache)/$(user).pub"
comment => "Copy public keys from a an authorized cache into a cache on localhost",
perms => mo("600","root"),
copy_from => remote_cp("$(authorized_source)/$(user).pub","$(sourcehost)"),
action => if_elapsed("60");
"/root/.ssh/authorized_keys"
comment => "Edit the authorized keys into the user's personal keyring",
edit_line => insert_file_if_no_line_matching("$(user)","$(local_cache)/$(user).pub"),
action => if_elapsed("60");
}
bundle agent allow_ssh_login_from_authorized_keys(user,sourcehost)
{
vars:
"local_cache" string => "/var/cfengine/ssh_cache";
"authorized_source" string => "/master/CFEngine/ssh_keys";
files:
"$(local_cache)/$(user).pub"
comment => "Copy public keys from a an authorized cache into a cache on localhost",
perms => mo("600","root"),
copy_from => remote_cp("$(authorized_source)/$(user).pub","$(sourcehost)"),
action => if_elapsed("60");
"/home/$(user)/.ssh/authorized_keys"
comment => "Edit the authorized keys into the user's personal keyring",
edit_line => insert_file_if_no_line_matching("$(user)","$(local_cache)/$(user).pub"),
action => if_elapsed("60");
}
bundle edit_line insert_file_if_no_line_matching(user,file)
{
classes:
"have_user" expression => regline("$(user).*","$(this.promiser)");
insert_lines:
!have_user::
"$(file)"
insert_type => "file";
}
System Information Examples
- Change detection
- Hashing for change detection (tripwire)
- Check filesystem space
- Class match example
- Global classes
- Logging
- Check filesystem space
Change detection
body common control
{
bundlesequence => { "testbundle" };
inputs => { "cfengine_stdlib.cf" };
}
bundle agent testbundle
{
files:
"/usr"
changes => detect_all_change,
depth_search => recurse("inf"),
action => background;
}
Hashing for change detection (tripwire)
Change detection is a powerful and easy way to monitor your environment, increase awareness and harden your system against security breaches.
body common control
{
bundlesequence => { "testbundle" };
inputs => { "$(sys.libdir)/stdlib.cf" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/web" -> "me"
changes => detect_all_change,
depth_search => recurse("inf");
}
Check filesystem space
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
vars:
"free" int => diskfree("/tmp");
reports:
"Freedisk $(free)";
}
Class match example
body common control
{
bundlesequence => { "example" };
}
bundle agent example
{
classes:
"do_it" and => { classmatch(".*_3"), "linux" };
reports:
do_it::
"Host matches pattern";
}
Global classes
body common control
{
bundlesequence => { "g","tryclasses_1", "tryclasses_2" };
}
bundle common g
{
classes:
"one" expression => "any";
"client_network" expression => iprange("128.39.89.0/24");
}
bundle agent tryclasses_1
{
classes:
"two" expression => "any";
}
bundle agent tryclasses_2
{
classes:
"three" expression => "any";
reports:
one.three.!two::
"Success";
}
body common control
{
bundlesequence => { "g","tryclasses_1", "tryclasses_2" };
}
bundle common g
{
classes:
"one" expression => "any";
"client_network" expression => iprange("128.39.89.0/24");
}
bundle agent tryclasses_1
{
classes:
"two" expression => "any";
}
bundle agent tryclasses_2
{
classes:
"three" expression => "any";
reports:
one.three.!two::
"Success";
}
Logging
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
vars:
"software" slist => { "/root/xyz", "/tmp/xyz" };
files:
"$(software)"
create => "true",
action => logme("$(software)");
}
body action logme(x)
{
log_kept => "/tmp/private_keptlog.log";
log_failed => "/tmp/private_faillog.log";
log_repaired => "/tmp/private_replog.log";
log_string => "$(sys.date) $(x) promise status";
}
body common control
{
bundlesequence => { "one" };
}
bundle agent one
{
files:
"/tmp/xyz"
create => "true",
action => log;
}
body action log
{
log_level => "inform";
}
System Administration Examples
- Centralized Management
- Laptop support configuration
- Process management
- Kill process
- Restart process
- Mount a filesystem
- Manage a system process
- Set up HPC clusters
- Set up name resolution
- Set up sudo
- Environments (virtual)
- Environment variables
- Tidying garbage files
Centralized Management
These examples show a simple setup for starting with a central approach to management of servers. Centralization of management is a simple approach suitable for small environments with few requirements. It is useful for clusters where systems are all alike.
All hosts the same
Variation in hosts
Updating from a central hub
All hosts the same
This shows the simplest approach in which all hosts are the same. It is too simple for most environments, but it serves as a starting point. Compare it to the next section that includes variation.
body common control
{
bundlesequence => { "central" };
}
bundle agent central
{
vars:
"policy_server" string => "myhost.domain.tld";
"mypackages" slist => {
"nagios"
"gcc",
"apache2",
"php5"
};
files:
# Password management can be very simple if all hosts are identical
"/etc/passwd"
comment => "Distribute a password file",
perms => mog("644","root","root"),
copy_from => secure_cp("/home/mark/LapTop/words/RoadAhead","$(policy_server)");
packages:
"$(mypackages)"
package_policy => "add",
package_method => generic;
# Add more promises below ...
}
body server control
{
allowconnects => { "127.0.0.1" , "::1", "10.20.30" };
allowallconnects => { "127.0.0.1" , "::1", "10.20.30" };
trustkeysfrom => { "127.0.0.1" , "::1", "10.20.30" };
# allowusers
}
bundle server access_rules()
{
access:
# myhost.domain.tld makes this file available to 10.20.30*
myhost_domain_tld::
"/etc/passwd"
admit => { "127.0.0.1", "10.20.30" };
}
Variation in hosts
body common control
{
bundlesequence => { "central" };
}
bundle agent central
{
classes:
"mygroup_1" or => { "myhost", "host1", "host2", "host3" };
"mygroup_2" or => { "host4", "host5", "host6" };
vars:
"policy_server" string => "myhost.domain.tld";
mygroup_1::
"mypackages" slist => {
"nagios"
"gcc",
"apache2",
"php5"
};
mygroup_2::
"mypackages" slist => {
"apache"
"mysql",
"php5"
};
files:
# Password management can be very simple if all hosts are identical
"/etc/passwd"
comment => "Distribute a password file",
perms => mog("644","root","root"),
copy_from => secure_cp("/etc/passwd","$(policy_server)");
packages:
"$(mypackages)"
package_policy => "add",
package_method => generic;
# Add more promises below ...
}
body server control
{
allowconnects => { "127.0.0.1" , "::1", "10.20.30" };
allowallconnects => { "127.0.0.1" , "::1", "10.20.30" };
trustkeysfrom => { "127.0.0.1" , "::1", "10.20.30" };
# allowusers
}
bundle server access_rules()
{
access:
# myhost.domain.tld makes this file available to 10.20.30*
myhost_domain_tld::
"/etc/passwd"
admit => { "127.0.0.1", "10.20.30" };
}
Updating from a central hub
The configuration bundled with the CFEngine source code contains an example of centralized updating of policy that covers more subtleties than this example, and handles fault tolerance. Here is the main idea behind it. For simplicity, we assume that all hosts are on network 10.20.30.* and that the central policy server/hub is 10.20.30.123.
bundle agent update
{
vars:
"master_location" string => "/var/cfengine/masterfiles";
"policy_server" string => "10.20.30.123",
comment => "IP address to locate your policy host.";
files:
"$(sys.workdir)/inputs"
perms => system("600"),
copy_from => remote_cp("$(master_location)",$(policy_server)),
depth_search => recurse("inf");
"$(sys.workdir)/bin"
perms => system("700"),
copy_from => remote_cp("/usr/local/sbin","localhost"),
depth_search => recurse("inf");
}
body server control
{
allowconnects => { "127.0.0.1" , "10.20.30" };
allowallconnects => { "127.0.0.1" , "10.20.30" };
trustkeysfrom => { "127.0.0.1" , "10.20.30" };
}
bundle server access_rules()
{
access:
10_20_30_123::
"/var/cfengine/masterfiles"
admit => { "127.0.0.1", "10.20.30" };
}
Laptop support configuration
Laptops do not need a lot of confguration support. IP addresses are set by DHCP and conditions are changeable. But you want to set your DNS search domains to familiar settings in spite of local DHCP configuration, and another useful trick is to keep a regular backup of disk changes on the local disk. This won't help against disk destruction, but it is a huge advantage when your user accidentally deletes files while travelling or offline.
body common control
{
bundlesequence => {
"update",
"garbage_collection",
"main",
"backup",
};
inputs => {
"update.cf",
"site.cf",
"library.cf"
};
}
body agent control
{
# if default runtime is 5 mins we need this for long jobs
ifelapsed => "15";
}
body monitor control
{
forgetrate => "0.7";
}
body executor control
{
splaytime => "1";
mailto => "mark@iu.hio.no";
smtpserver => "localhost";
mailmaxlines => "30";
# Instead of a separate update script, now do this
exec_command => "$(sys.workdir)/bin/cf-agent -f failsafe.cf && $(sys.workdir)/bin/cf-agent";
}
bundle agent main
{
vars:
"component" slist => { "cf-monitord", "cf-serverd" };
# - - - - - - - - - - - - - - - - - - - - - - - -
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => resolver,
edit_defaults => def;
processes:
"$(component)" restart_class => canonify("start_$(component)");
# - - - - - - - - - - - - - - - - - - - - - - - -
commands:
"$(sys.workdir)/bin/$(component)"
ifvarclass => canonify("start_$(component)");
}
bundle agent backup
{
files:
"/home/backup"
copy_from => cp("/home/mark"),
depth_search => recurse("inf"),
file_select => exclude_files,
action => longjob;
}
bundle agent garbage_collection
{
files:
"$(sys.workdir)/outputs"
delete => tidy,
file_select => days_old("3"),
depth_search => recurse("inf");
}
Process management
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
processes:
"sleep"
signals => { "term", "kill" };
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
"sleep"
process_count => up("sleep");
reports:
sleep_out_of_control::
"Out of control";
}
body process_count up(s)
{
match_range => "5,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
".*"
process_select => proc_finder("a.*"),
process_count => up("cfservd");
}
body process_count up(s)
{
match_range => "1,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
body process_select proc_finder(p)
{
stime_range => irange(ago("0","0","0","2","0","0"),now);
process_result => "stime";
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
".*"
process_select => proc_finder("a.*"),
process_count => up("cfservd");
}
body process_count up(s)
{
match_range => "1,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
body process_select proc_finder(p)
{
process_owner => { "avahi", "bin" };
command => "$(p)";
pid => "100,199";
vsize => "0,1000";
process_result => "command.(process_owner|vsize)";
}
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
processes:
"/usr/bin/daemon"
restart_class => "launch";
commands:
launch::
"/usr/bin/daemon";
}
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
vars:
"component" slist => {
"cf-monitord",
"cf-serverd",
"cf-execd"
};
processes:
"$(component)"
restart_class => canonify("start_$(component)");
commands:
"/var/cfengine/bin/$(component)"
ifvarclass => canonify("start_$(component)");
}
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
"cfservd"
process_count => up("cfservd");
cfservd_out_of_control::
"cfservd"
signals => { "stop" , "term" },
restart_class => "start_cfserv";
commands:
start_cfserv::
"/usr/local/sbin/cfservd";
}
body process_count up(s)
{
match_range => "1,10"; # or irange("1","10");
out_of_range_define => { "$(s)_out_of_control" };
}
Kill process
body common control
{
bundlesequence => { "test" };
}
bundle agent test
{
processes:
"sleep"
signals => { "term", "kill" };
}
Restart process
A basic pattern for restarting processes:
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
processes:
"/usr/bin/daemon"
restart_class => "launch";
commands:
launch::
"/usr/bin/daemon";
}
This can be made more sophisticated to handle generic lists:
body common control
{
bundlesequence => { "process_restart" };
}
bundle agent process_restart
{
vars:
"component" slist => {
"cf-monitord",
"cf-serverd",
"cf-execd"
};
processes:
"$(component)"
restart_class => canonify("start_$(component)");
commands:
"/var/cfengine/bin/$(component)"
ifvarclass => canonify("start_$(component)");
}
Why? Separating this into two parts gives a high level of control and conistency to CFEngine. There are many options for command execution, like the ability to run commands in a sandbox or as `setuid'. These should not be reproduced in processes.
Mount a filesystem
body common control
{
bundlesequence => { "mounts" };
}
bundle agent mounts
{
storage:
"/mnt" mount => nfs("slogans.iu.hio.no","/home");
}
body mount nfs(server,source)
{
mount_type => "nfs";
mount_source => "$(source)";
mount_server => "$(server)";
#mount_options => { "rw" };
edit_fstab => "true";
unmount => "true";
}
Manage a system process
Ensure running
Ensure not running
Prune processes
Ensure running
The simplest example might look like this:
bundle agent restart_process
{
processes:
"httpd"
comment => "Make sure apache web server is running",
restart_class => "restart_httpd";
commands:
restart_httpd::
"/etc/init.d/apache2 restart";
}
This example shows how the CFEngine components could be started using a pattern.
bundle agent CFEngine_processes
{
vars:
"components" slist => { "cf-execd", "cf-monitord", "cf-serverd", "cf-hub" };
processes:
"$(components)"
comment => "Make sure server parts of CFEngine are running",
restart_class => canonify("start_$(component)");
commands:
"$(sys.workdir)/bin/$(component)"
comment => "Make sure server parts of CFEngine are running",
ifvarclass => canonify("start_$(components)");
}
Ensure not running
bundle agent restart_process
{
vars:
"killprocs" slist => { "snmpd", "gameserverd", "irc", "crack" };
processes:
"$(killprocs)"
comment => "Ensure processes are not running",
signals => { "term", "kill" };
}
Prune processes
This example kills processes owned by a particular user that have exceeded 100000 bytes of resident memory.
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
processes:
".*"
process_select => big_processes("mark"),
signals => { "term" };
}
body process_select big_processes(o)
{
process_owner => { $(o) };
rsize => irange("100000","900000");
process_result => "rsize.process_owner";
}
Set up HPC clusters
HPC cluster machines are usually all identical, so the CFEngine configuration is very simple. HPC clients value CPU and memory resources, so we can shut down unnecessary services to save CPU. We can also change the scheduling rate of CFEngine to run less frequently, and save a little:
body executor control
{
splaytime => "1";
mailto => "cfengine@example.com";
smtpserver => "localhost";
mailmaxlines => "30";
# Once per hour, on the hour
schedule => { "Min00_05" };
}
bundle agent services_disable
{
vars:
# list all of xinetd services (case sensitive)
"xinetd_services" slist => {
"imap",
"imaps",
"ipop2",
"ipop3",
"krb5-telnet",
"klogin",
"kshell",
"ktalk",
"ntalk",
"pop3s",
};
methods:
# perform the actual disable all xinetd services according to the list above
"any" usebundle => disable_xinetd("$(xinetd_services)");
processes:
"$(xinetd_services)"
signals => { "kill" };
}
bundle agent disable_xinetd(name)
{
vars:
"status" string => execresult("/sbin/chkconfig --list $(name)", "useshell");
classes:
"on" expression => regcmp(".*on.*","$(status)");
commands:
on::
"/sbin/chkconfig $(name) off",
comment => "disable $(name) service";
reports:
on::
"disable $(name) service.";
}
Set up name resolution
There are many ways to do name resolution setup1 We write a reusable bundle using the editing features.
A simple and straightforward approach is to maintain a separate modular bundle for this task. This avoids too many levels of abstraction and keeps all the information in one place. We implement this as a simple editing promise for the /etc/resolv.conf file.
bundle agent system_files
{
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
comment => "Add lines to the resolver configuration",
create => "true",
edit_line => resolver,
edit_defaults => std_edits;
# ...other system files ...
}
bundle edit_line resolver
{
delete_lines:
# delete any old name servers or junk we no longer need
"search.*";
"nameserver 80.65.58.31";
"nameserver 80.65.58.32";
"nameserver 82.103.128.146";
"nameserver 78.24.145.4";
"nameserver 78.24.145.5";
"nameserver 128.39.89.10";
insert_lines:
"search mydomain.tld" location => start;
special_net::
"nameserver 128.39.89.8";
"nameserver 128.39.74.66";
!special_net::
"nameserver 128.38.34.12";
any::
"nameserver 212.112.166.18";
"nameserver 212.112.166.22";
}
A second approach is to try to conceal the operational details behind a veil of abstraction.
bundle agent system_files
{
vars:
"searchlist" string => "iu.hio.no CFEngine.com";
"nameservers" slist => {
"128.39.89.10",
"128.39.74.16",
"192.168.1.103"
};
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => doresolv("$(s)","@(this.n)"),
edit_defaults => empty;
# ....
}
bundle edit_line doresolv(search,names)
{
insert_lines:
"search $(search)";
"nameserver $(names)";
}
bundle agent system_files { # ...
files: "/etc/hosts" comment => "Add hosts to the /etc/hosts file", edit_line => fix_etc_hosts; }
bundle edit_line fix_etc_hosts { vars: "names[127.0.0.1]" string => "localhost localhost.CFEngine.com"; "names[128.39.89.12]" string => "myhost myhost.CFEngine.com"; "names[128.39.89.13]" string => "otherhost otherhost.CFEngine.com"; # etc
"i" slist => getindices("names");
insert_lines: "$(i) $(names[$(i)])"; } ```
DNS is not the only name service, of course. Unix has its older /etc/hosts file which can also be managed using file editing. We simply append this to the system_files bundle.
bundle agent system_files
{
vars:
"searchlist" string => "iu.hio.no CFEngine.com";
"nameservers" slist => {
"128.39.89.10",
"128.39.74.16",
"192.168.1.103"
};
files:
"$(sys.resolv)" # test on "/tmp/resolv.conf" #
create => "true",
edit_line => doresolv("$(s)","@(this.n)"),
edit_defaults => empty;
# ....
}
bundle edit_line doresolv(search,names)
{
insert_lines:
"search $(search)";
"nameserver $(names)";
}
bundle agent system_files { # ...
files: "/etc/hosts" comment => "Add hosts to the /etc/hosts file", edit_line => fix_etc_hosts; }
bundle edit_line fix_etc_hosts { vars: "names[127.0.0.1]" string => "localhost localhost.CFEngine.com"; "names[128.39.89.12]" string => "myhost myhost.CFEngine.com"; "names[128.39.89.13]" string => "otherhost otherhost.CFEngine.com"; # etc
"i" slist => getindices("names");
insert_lines: "$(i) $(names[$(i)])"; } ```
Set up sudo
Setting up sudo is straightforward, and is best managed by copying trusted files from a repository.
bundle agent system_files
{
vars:
"masterfiles" string => "/subversion_projects/masterfiles";
# ...
files:
"/etc/sudoers"
comment => "Make sure the sudo configuration is secure and up to date",
perms => mog("440","root","root"),
copy_from => secure_cp("$(masterfiles)/sudoers","$(policy_server)");
}
Environments (virtual)
body common control
{
bundlesequence => { "my_vm_cloud" };
}
bundle agent my_vm_cloud
{
vars:
"vms[atlas]" slist => { "guest1", "guest2", "guest3" };
environments:
scope||any:: # These should probably be in class "any" to ensure uniqueness
"$(vms[$(sys.host)])"
environment_resources => virt_xml("$(xmlfile[$(this.promiser)])"),
environment_interface => vnet("eth0,192.168.1.100/24"),
environment_type => "test",
environment_host => "atlas";
# default environment_state => "create" on host, and "suspended elsewhere"
}
body environment_resources virt_xml(specfile)
{
env_spec_file => "$(specfile)";
}
body environment_interface vnet(primary)
{
env_name => "$(this.promiser)";
env_addresses => { "$(primary)" };
host1::
env_network => "default_vnet1";
host2::
env_network => "default_vnet2";
}
Environment variables
body common control
{
bundlesequence => { "my_vm_cloud" };
}
bundle agent my_vm_cloud
{
environments:
"centos5"
environment_resources => virt_xml,
environment_type => "xen",
environment_host => "ursa-minor";
# default environment_state => "create" on host, and "suspended elsewhere"
}
body environment_resources virt_xml
{
env_spec_file => "/srv/xen/centos5-libvirt-create.xml";
}
Tidying garbage files
Emulating the `tidy' feature of CFEngine 2.
body common control
{
any::
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/tmp/test"
delete => tidy,
file_select => zero_age,
depth_search => recurse("inf");
}
body depth_search recurse(d)
{
#include_basedir => "true";
depth => "$(d)";
}
body delete tidy
{
dirlinks => "delete";
rmdirs => "false";
}
body file_select zero_age
{
mtime => irange(ago(1,0,0,0,0,0),now);
file_result => "mtime";
}
System File Examples
- Editing password or group files
- Editing password or group files custom
- Log rotation
- Garbage collection
- Manage a system file
Editing password or group files
To change the password of a system, we need to edit a file. A file is a complex object – once open there is a new world of possible promises to make about its contents. CFEngine has bundles of promises that are specially for editing.
body common control
{
inputs => { "$(sys.libdir)/stdlib.cf" };
bundlesequence => { "edit_passwd" };
}
bundle agent edit_passwd
{
vars:
"userset" slist => { "user1", "user2", "user3" };
files:
"/etc/passwd"
edit_line => set_user_field("mark","7","/set/this/shell");
"/etc/group"
edit_line => append_user_field("root","4","@(main.userset)");
}
Editing password or group files custom
In this example the bundles from the Community Open Promise-Body Library are included directly in the policy instead of being input as a separate file.
body common control
{
bundlesequence => { "addpasswd" };
}
bundle agent addpasswd
{
vars:
# want to set these values by the names of their array keys
"pwd[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
"pwd[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pwd[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
files:
"/tmp/passwd"
create => "true",
edit_line => append_users_starting("addpasswd.pwd");
}
bundle edit_line append_users_starting(v)
{
vars:
"index" slist => getindices("$(v)");
classes:
"add_$(index)" not => userexists("$(index)");
insert_lines:
"$($(v)[$(index)])",
ifvarclass => "add_$(index)";
}
bundle edit_line append_groups_starting(v)
{
vars:
"index" slist => getindices("$(v)");
classes:
"add_$(index)" not => groupexists("$(index)");
insert_lines:
"$($(v)[$(index)])",
ifvarclass => "add_$(index)";
}
Log rotation
body common control
{
bundlesequence => { "testbundle" };
}
bundle agent testbundle
{
files:
"/home/mark/tmp/rotateme"
rename => rotate("4");
}
body rename rotate(level)
{
rotate => "$(level)";
}
Garbage collection
body common control
{
bundlesequence => { "garbage_collection" };
inputs => { "cfengine_stdlib.cf" };
}
bundle agent garbage_collection
{
files:
Sunday::
"$(sys.workdir)/nova_repair.log"
comment => "Rotate the promises repaired logs each week",
rename => rotate("7"),
action => if_elapsed("10000");
"$(sys.workdir)/nova_notkept.log"
comment => "Rotate the promises not kept logs each week",
rename => rotate("7"),
action => if_elapsed("10000");
"$(sys.workdir)/promise.log"
comment => "Rotate the promises not kept logs each week",
rename => rotate("7"),
action => if_elapsed("10000");
any::
"$(sys.workdir)/outputs"
comment => "Garbage collection of any output files",
delete => tidy,
file_select => days_old("3"),
depth_search => recurse("inf");
"$(sys.workdir)/"
comment => "Garbage collection of any output files",
delete => tidy,
file_select => days_old("14"),
depth_search => recurse("inf");
# Other resources
"/tmp"
comment => "Garbage collection of any temporary files",
delete => tidy,
file_select => days_old("3"),
depth_search => recurse("inf");
"/var/log/apache2/.*bz"
comment => "Garbage collection of rotated log files",
delete => tidy,
file_select => days_old("30"),
depth_search => recurse("inf");
"/var/log/apache2/.*gz"
comment => "Garbage collection of rotated log files",
delete => tidy,
file_select => days_old("30"),
depth_search => recurse("inf");
"/var/log/zypper.log"
comment => "Prevent the zypper log from choking the disk",
rename => rotate("0"),
action => if_elapsed("10000");
}
Manage a system file
Simple template
Simple versioned template
Macro template
Custom editing
Simple template
bundle agent hand_edited_config_file
{
vars:
"file_template" string =>
"
127.0.0.1 localhost
::1 localhost ipv6-localhost ipv6-loopback
fe00::0 ipv6-localnet
ff00::0 ipv6-mcastprefix
ff02::1 ipv6-allnodes
ff02::2 ipv6-allrouters
ff02::3 ipv6-allhosts
10.0.0.100 host1.domain.tld host1
10.0.0.101 host2.domain.tld host2
10.0.0.20 host3.domain.tld host3
10.0.0.21 host4.domain.tld host4
";
##############################################################
files:
"/etc/hosts"
comment => "Define the content of all host files from this master source",
create => "true",
edit_line => append_if_no_lines("$(file_template)"),
edit_defaults => empty,
perms => mo("$(mode)","root"),
action => if_elapsed("60");
}
Simple versioned template
The simplest approach to managing a file is to maintain a master copy by hand, keeping it in a version controlled repository (e.g. svn), and installing this version on the end machine.
We'll assume that you have a version control repository that is located on some independent server, and has been checked out manually once (with authentication) in /mysite/masterfiles.
bundle agent hand_edited_config_file
{
vars:
"masterfiles" string => "/mysite/masterfiles";
"policy_server" string => "policy_host.domain.tld";
files:
"/etc/hosts"
comment => "Synchronize hosts with a hand-edited template in svn",
perms => m("644"),
copy_from => remote_cp("$(masterfiles)/trunk/hosts_master","$(policy_server)");
commands:
"/usr/bin/svn update"
comment => "Update the company document repository including manuals to a local copy",
contain => silent_in_dir("$(masterfiles)/trunk"),
ifvarclass => canonify("$(policy_server)");
}
Macro template
The next simplest approach to file management is to add variables to the template that will be expanded into local values at the end system, e.g. using variables like ‘$(sys.host)’ for the name of the host within the body of the versioned template.
bundle agent hand_edited_template
{
vars:
"masterfiles" string => "/mysite/masterfiles";
"policy_server" string => "policy_host.domain.tld";
files:
"/etc/hosts"
comment => "Synchronize hosts with a hand-edited template in svn",
perms => m("644"),
create => "true",
edit_line => expand_template("$(masterfiles)/trunk/hosts_master"),
edit_defaults => empty,
action => if_elapsed("60");
commands:
"/usr/bin/svn update"
comment => "Update the company document repository including manuals to a local copy",
contain => silent_in_dir("$(masterfiles)/trunk"),
ifvarclass => canonify("$(policy_server)");
}
The macro template file may contain variables, as below, that get expanded by CFEngine.
bundle agent hand_edited_template
{
vars:
"masterfiles" string => "/mysite/masterfiles";
"policy_server" string => "policy_host.domain.tld";
files:
"/etc/hosts"
comment => "Synchronize hosts with a hand-edited template in svn",
perms => m("644"),
create => "true",
edit_line => expand_template("$(masterfiles)/trunk/hosts_master"),
edit_defaults => empty,
action => if_elapsed("60");
commands:
"/usr/bin/svn update"
comment => "Update the company document repository including manuals to a local copy",
contain => silent_in_dir("$(masterfiles)/trunk"),
ifvarclass => canonify("$(policy_server)");
}
127.0.0.1 localhost $(sys.host) ::1 localhost ipv6-localhost ipv6-loopback fe00::0 ipv6-localnet ff00::0 ipv6-mcastprefix ff02::1 ipv6-allnodes ff02::2 ipv6-allrouters ff02::3 ipv6-allhosts 10.0.0.100 host1.domain.tld host1 10.0.0.101 host2.domain.tld host2 10.0.0.20 host3.domain.tld host3 10.0.0.21 host4.domain.tld host4
$(definitions.more_hosts) ```
Custom editing
If you do not control the starting state of the file, because it is distributed by an operating system vendor for instance, then editing the final state is the best approach. That way, you will get changes that are made by the vendor, and will ensure your own modifications are kept even when updates arrive.
bundle agent modifying_managed_file
{
vars:
"data" slist => { "10.1.2.3 sirius", "10.1.2.4 ursa-minor", "10.1.2.5 orion"};
files:
"/etc/hosts"
comment => "Append a list of lines to the end of a file if they don't exist",
perms => m("644"),
create => "true",
edit_line => append_if_no_lines("modifying_managed_file.data"),
action => if_elapsed("60");
}
Another example shows how to set the values of variables using a data-driven approach and methods from the standard library.
body common control
{
bundlesequence => { "testsetvar" };
}
bundle agent testsetvar
{
vars:
"v[variable_1]" string => "value_1";
"v[variable_2]" string => "value_2";
files:
"/tmp/test_setvar"
edit_line => set_variable_values("testsetvar.v");
}
Windows Registry Examples
Windows registry
body common control
{
bundlesequence => { "reg" };
}
bundle agent reg
{
vars:
"value" string => registryvalue("HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine","value3");
reports:
windows::
"Value extracted: $(value)";
}
unit_registry_cache.cf
body common control
{
bundlesequence => {
# "registry_cache"
# "registry_restore"
};
}
bundle agent registry_cache
{
databases:
windows::
"HKEY_LOCAL_MACHINE\SOFTWARE\Adobe"
database_operation => "cache",
database_type => "ms_registry",
comment => "Save correct registry settings for Adobe products";
}
bundle agent registry_restore
{
databases:
windows::
"HKEY_LOCAL_MACHINE\SOFTWARE\Adobe"
database_operation => "restore",
database_type => "ms_registry",
comment => "Make sure Adobe products have correct registry settings";
}
unit_registry.cf
body common control
{
bundlesequence => { "databases" };
}
bundle agent databases
{
databases:
windows::
# Registry has (value,data) pairs in "keys" which are directories
# "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS"
# database_operation => "create",
# database_type => "ms_registry";
# "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
# database_operation => "create",
# database_rows => { "value1,REG_SZ,new value 1", "value2,REG_SZ,new val 2"} ,
# database_type => "ms_registry";
"HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
database_operation => "delete",
database_columns => { "value1", "value2" } ,
database_type => "ms_registry";
# "HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS\Cfengine"
# database_operation => "cache", # cache,restore
# registry_exclude => { ".*Windows.*CurrentVersion.*", ".*Touchpad.*", ".*Capabilities.FileAssociations.*", ".*Rfc1766.*" , ".*Synaptics.SynTP.*", ".*SupportedDevices.*8086", ".*Microsoft.*ErrorThresholds" },
# database_type => "ms_registry";
"HKEY_LOCAL_MACHINE\SOFTWARE\Cfengine AS"
database_operation => "restore",
database_type => "ms_registry";
}
User Management and ACL Examples
- Manage users
- Add users
- Add users to passwd and group
- ACL file example
- ACL generic example
- ACL secret example
- Active directory example
- Active list users directory example
- Active directory show users example
- Get a list of users
- LDAP interactions
Manage users
There are many approaches to managing users. You can edit system files like /etc/passwd directly, or you can use commands on some systems like ‘useradd’ or ‘adduser’. In all cases it is desirable to make this a data-driven process.
Add users
Remove users
Add users
A simple approach which adds new users to the password file, and to a group called ‘users’ in the group file. Is shown below. This example does not edit the shadow file. A simple pattern that can be modified for use is shown below.
Note that, although this is a simple minded approach, it is the most efficient of the approaches shown here as all operations can be carried out in a single operation for each file.
bundle agent addusers
{
vars:
# Add some users
"pw[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
"pw[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pw[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
"users" slist => getindices("pw");
files:
"/etc/passwd"
edit_line => append_users_starting("addusers.pw");
# "/etc/shadow"
# edit_line => append_users_starting("$(users):defaultpasswd:::::::");
"/etc/group"
edit_line => append_user_field("users","4","@(addusers.users)");
"/home/$(users)/."
create => "true",
perms => mog("755","$(users)","users");
# equivalent to ALL of the above in 3.6 (see documentation)
users:
"$(users)" policy => "present", shell => "/bin/bash";
}
A second approach is to use the shell commands supplied by some operating systems; this assumes that suitable defaults have been set up manually. Also the result is not repairable in a simple convergent manner. The command needs to edit multiple files for each user, and is quite inefficient.
bundle agent addusers
{
vars:
# Add some users
"pw[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
"pw[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pw[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
"users" slist => getindices("pw");
methods:
"any" usebundle => user_add("$(users)","$(pw[$(users)])");
}
bundle agent user_add(x,pw)
{
files:
"/etc/passwd"
edit_line => append_users_starting("addusers.pw");
# "/etc/shadow"
# edit_line => append_users_starting("$(users):defaultpasswd:::::::");
"/etc/group"
edit_line => append_user_field("users","4","@(addusers.users)");
"/home/$(users)/."
create => "true",
perms => mog("755","$(users)","users");
}
An alternative approach is to use a method to wrap around the handling of a user. Although this looks nice, it is less efficient than the first method because it must edit the files multiple times.
bundle agent addusers
{
vars:
# Add some users
"pw[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
"pw[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pw[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
"users" slist => getindices("pw");
methods:
"any" usebundle => user_add("$(users)","$(pw[$(users)])");
}
bundle agent user_add(x,pw)
{
files:
"/etc/passwd"
edit_line => append_users_starting("addusers.pw");
# "/etc/shadow"
# edit_line => append_users_starting("$(users):defaultpasswd:::::::");
"/etc/group"
edit_line => append_user_field("users","4","@(addusers.users)");
"/home/$(users)/."
create => "true",
perms => mog("755","$(users)","users");
}
Add users to passwd and group
Add lines to the password file, and users to group if they are not already there.
body common control
{
bundlesequence => { "addpasswd" };
inputs => { "cf_std_library.cf" };
}
bundle agent addpasswd
{
vars:
# want to set these values by the names of their array keys
"pwd[mark]" string => "mark:x:1000:100:Mark Burgess:/home/mark:/bin/bash";
"pwd[fred]" string => "fred:x:1001:100:Right Said:/home/fred:/bin/bash";
"pwd[jane]" string => "jane:x:1002:100:Jane Doe:/home/jane:/bin/bash";
"users" slist => getindices("pwd");
files:
"/etc/passwd"
create => "true",
edit_line => append_users_starting("addpasswd.pwd");
"/etc/group"
edit_line => append_user_field("users","4","@(addpasswd.users)");
}
ACL file example
body common control
{
bundlesequence => { "acls" };
}
bundle agent acls
{
files:
"/media/flash/acl/test_dir"
depth_search => include_base,
acl => template;
}
body acl template
{
acl_method => "overwrite";
acl_type => "posix";
acl_directory_inherit => "parent";
aces => { "user:*:r(wwx),-r:allow", "group:*:+rw:allow", "mask:x:allow", "all:r"};
}
body acl win
{
acl_method => "overwrite";
acl_type => "ntfs";
acl_directory_inherit => "nochange";
aces => { "user:Administrator:rw", "group:Bad:rwx(Dpo):deny" };
}
body depth_search include_base
{
include_basedir => "true";
}
ACL generic example
body common control
{
bundlesequence => { "acls" };
}
bundle agent acls
{
files:
"/media/flash/acl/test_dir"
depth_search => include_base,
acl => test;
}
body acl test
{
acl_type => "generic";
aces => {"user:bob:rwx", "group:staff:rx", "all:r"};
}
body depth_search include_base
{
include_basedir => "true";
}
ACL secret example
body common control
{
bundlesequence => { "acls" };
}
bundle agent acls
{
files:
windows::
"c:\Secret"
acl => win,
depth_search => include_base,
comment => "Secure the secret directory from unauthorized access";
}
body acl win
{
acl_method => "overwrite";
aces => { "user:Administrator:rwx" };
}
body depth_search include_base
{
include_basedir => "true";
}
Active directory example
bundle agent active_directory
{
vars:
# NOTE: Edit this to your domain, e.g. "corp", may also need more DC's after it
"domain_name" string => "cftesting";
"user_name" string => "Guest";
# NOTE: We can also extract data from remote Domain Controllers
dummy.DomainController::
"domain_controller" string => "localhost";
"userlist" slist => ldaplist(
"ldap://$(domain_controller)",
"CN=Users,DC=$(domain_name),DC=com",
"(objectClass=user)",
"sAMAccountName",
"subtree",
"none");
classes:
dummy.DomainController::
"gotuser" expression => ldaparray(
"userinfo",
"ldap://$(domain_controller)",
"CN=$(user_name),CN=Users,DC=$(domain_name),DC=com",
"(name=*)",
"subtree",
"none");
reports:
dummy.DomainController::
'Username is "$(userlist)"';
dummy.gotuser::
"Got user data; $(userinfo[name]) has logged on $(userinfo[logonCount]) times";
}
Active list users directory example
bundle agent ldap
{
vars:
"userlist" slist => ldaplist(
"ldap://cf-win2003",
"CN=Users,DC=domain,DC=cf-win2003",
"(objectClass=user)",
"sAMAccountName",
"subtree",
"none");
reports:
'Username: "$(userlist)"';
}
Active directory show users example
bundle agent ldap
{
classes:
"gotdata" expression => ldaparray(
"myarray",
"ldap://cf-win2003",
"CN=Test Pilot,CN=Users,DC=domain,DC=cf-win2003",
"(name=*)",
"subtree",
"none");
reports:
gotdata::
"Got user data";
!gotdata::
"Did not get user data";
}
Get a list of users
body common control
{
bundlesequence => { test };
}
bundle agent test
{
vars:
"allusers" slist => getusers("zenoss,mysql,at","12,0");
reports:
linux::
"Found user $(allusers)";
}
LDAP interactions
body common control
{
bundlesequence => { "ldap" , "followup"};
}
bundle agent ldap
{
vars:
# Get the first matching value for "uid"
"value" string => ldapvalue("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","none");
# Get all matching values for "uid" - should be a single record match
"list" slist => ldaplist("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","none");
classes:
"gotdata" expression => ldaparray("myarray","ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(uid=mark)","subtree","none");
"found" expression => regldap("ldap://eternity.iu.hio.no","dc=cfengine,dc=com","(sn=User)","uid","subtree","jon.*","none");
reports:
linux::
"LDAP VALUE $(value) found";
"LDAP LIST VALUE $(list)";
gotdata::
"Found specific entry data ...$(ldap.myarray[uid]),$(ldap.myarray[gecos]), etc";
found::
"Matched regex";
}
bundle agent followup
{
reports:
linux::
"Different bundle ...$(ldap.myarray[uid]),$(ldap.myarray[gecos]),...";
}
Tutorials
Familarize yourself with CFEngine by following these step by step tutorials.
Additional tutorials to help you get started including screen casts can be found in the CFEngine Learning Center.
JSON and YAML Support in CFEngine
Introduction
JSON is a well-known data language. It even has a specification (See http://json.org).
YAML is another well-known data language. It has a longer, much more complex specification (See http://yaml.org).
CFEngine has core support for JSON and YAML. Let's see what it can do.
Problem statement
We'd like to read, access, and merge JSON-sourced data structures: they should be weakly typed, arbitrarily nested, with consistent quoting and syntax.
We'd like to read, access, and merge YAML-sourced data structures just like JSON-sourced, to keep policy and internals simple.
In addition, we must not break backward compatibility with CFEngine
3.5 and older, so we'd like to use the standard CFEngine array a[b]
syntax.
Data containers
A new data type, the data container, was introduced in 3.6.
It's simply called data
. The documentation with some examples is at https://cfengine.com/docs/master/reference-promise-types-vars.html#data-container-variables
Reading JSON
There are many ways to read JSON data; here are a few:
readjson()
: read from a JSON file, e.g."mydata" data => readjson("/my/file", 100k);
parsejson()
: read from a JSON string, e.g."mydata" data => parsejson('{ "x": "y" }');
data_readstringarray()
anddata_readstringarrayidx()
: read text data from a file, split it on a delimiter, and make them into structured data.mergedata()
: merge data containers, slists, and classic CFEngine arrays, e.g."mydata" data => mergedata(container1, slist2, array3);
mergedata
in particular is very powerful. It can convert a slist or a classic CFEngine array to a data container easily: "mydata" data => mergedata(myslist);
Reading YAML
There are two ways to read YAML data:
readyaml()
: read from a YAML file, e.g."mydata" data => readyaml("/my/file.yaml", 100k);
parseyaml()
: read from a YAML string, e.g."mydata" data => parseyaml('- arrayentry1');
Since these functions return data containers, everything about JSON-sourced data structures applies to YAML-sourced data structures as well.
Accessing JSON
To access JSON data, you can use:
- the
nth()
function to access an array element, e.g."myx" string => nth(container1, 0);
- the
nth
function to access a map element, e.g."myx" string => nth(container1, "x");
- the
a[b]
notation, e.g."myx" string => "$(container1[x])";
. You can nest, e.g.a[b][c][0][d]
. This only works if the element is something that can be expanded in a string. So a number or a string work. A list of strings or numbers works. A key-value map underx
won't work. - the
getindices()
andgetvalues()
functions, just like classic CFEngine arrays
A full example
This example can be saved and run. It will load a key-value map where the keys are class names and the values are hostname regular expressions or class names.
- if your host name is
c
orb
or the classesc
orb
are defined, thedev
class will be defined - if your host name is
flea
or the classflea
is defined, theprod
class will be defined - if your host name is
a
or the classa
is defined, theqa
class will be defined - if your host name is
linux
or the classlinux
is defined, theprivate
class will be defined
Easy, right?
body common control
{
bundlesequence => { "run" };
}
bundle agent run
{
vars:
"bykey" data => parsejson('{ "dev": ["c", "b"], "prod": ["flea"], "qa": ["a"], "private": ["linux"] }');
"keys" slist => getindices("bykey");
classes:
# define the class from the key name if any of the items under the key match the host name
"$(keys)" expression => regcmp("$(bykey[$(keys)])", $(sys.host));
# define the class from the key name if any of the items under the key are a defined class
"$(keys)" expression => classmatch("$(bykey[$(keys)])");
reports:
"keys = $(keys)";
"I am in class $(keys)" ifvarclass => $(keys);
}
So, where's the magic? Well, if you're familiar with classic CFEngine
arrays, you will be happy to hear that the exact same syntax works
with them. In other words, data containers don't change how you use
CFEngine. You still use getindices
to get the keys, then iterate
through them and look up values.
Well, you can change
"bykey" data => parsejson('{ "dev": ["c", "b"], "prod": ["flea"], "qa": ["a"], "private": ["linux"] }');
with
"bykey" data => data_readstringarray(...);
and read the same container from a text file. The file should be formatted like this to produce the same data as above:
dev c b
prod flea
qa a
private linux
You can also use
"bykey" data => readjson(...);
and read the same container from a JSON file.
Summary
Using JSON and YAML from CFEngine is easy and does not change how you use CFEngine. Try it out and see for yourself!
Reporting and Remediation of Security Vulnerabilities
Prerequisites
- CFEngine 3.6 Enterprise Hub
- At least one client vulnerable to CVE-2014-6271
Overview
Remediating security vulnerabilities is a common issue. Sometimes you want to know the extent to which your estate is affected by a threat. Identification of affected systems can help you prioritize and plan remediation efforts. In this tutorial you will learn how to inventory your estate and build alerts to find hosts that are affected by the #shellshock exploit. After identifying the affected hosts you will patch a subset of the hosts and then be able to see the impact on your estate. The same methodology can be applied to other issues.
Note: The included policy does not require CFEngine Enterprise. Only the reporting functionality (Mission Portal) requires the Enterprise version.
Inventory CVE-2013-6271
Writing inventory policy with CFEngine is just like any other CFEngine policy,
except for the addition of special meta
attributes used to augment the
inventory interface. First you must know how to collect the information you
want. In this case we know that a vulnerable system will have the word
vulnerable listed in the output of the command
env x='() { :;}; echo vulnerable' $(bash) -c 'echo testing CVE-2014-6271'
.
This bundle will check if the host is vulnerable to the CVE, define a class CVE_2014_6217 if it is vulnerable and augment Mission Portals Inventory interface in CFEngine Enterprise.
bundle agent inventory_CVE_2014_6271
{
meta:
"description" string => "Remote exploit vulnerability in bash http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-6271";
"tags" slist => { "autorun" };
vars:
"env" string => "$(paths.env)";
"bash" string => "/bin/bash";
"echo" string => "$(paths.echo)";
"test_result" string => execresult("$(env) x='() { :;}; $(echo) vulnerable' $(bash) -c 'echo testing CVE-2014-6271'", "useshell");
CVE_2014_6271::
"vulnerable"
string => "CVE-2014-6271",
meta => { "inventory", "attribute_name=Vulnerable CVE(s)" };
classes:
"CVE_2014_6271"
expression => regcmp( "vulnerable.*", "$(test_result)" ),
scope => "namespace",
persistence => "10",
comment => "We persist the class for 2 agent runs so that bundles
activated before this bundle can use the class on the next
agent execution to coordinate things like package updates.";
reports:
DEBUG|DEBUG_cve_2014_6217::
"Test Result: $(test_result)";
CVE_2014_6271.(inform_mode|verbose_mode)::
"Tested Vulnerable for CVE-2014-6271: $($(this.bundle)_meta.description)";
}
What does this inventory policy do?
Meta type promises are used to attach additional information to bundles. We have set 'description' so that future readers of the policy will know what the policy is for and how to get more information on the vulnerability. For the sake of simplicity in this example set 'autorun' as a tag to the bundle. This makes the bundle available for automatic activation when using the autorun feature in the Masterfiles Policy Framework.
Next we set the paths to the binaries that we will use to exeucte our test command. As of this writing the paths for 'env' and 'echo' are both in the standard libraries paths bundle, but 'bash' is not. Note that you may need to adjust the path to bash for your platforms. Then we run our test command and place the command output into the 'test_result' variable. Since we have no CVE_2014_6271 class defined yet, the next promise to set the variable 'vulnerable' to 'CVE-2014-6271' will be skipped on the first pass. Then the classes type promise is evaluated and defines the class CVE_2014_6271 if the output matches the regular expression 'vulnerable.*'. Finally the reports are evaluated before starting the second pass. If the class 'DEBUG' or 'DEBUG_inventory_CVE_2014_6271' is set the test command output will be shown, and if the vulnerability is present agent is running in inform or verbose mode message indicating the host is vulnerable along with the description will be output.
On the second pass only that variable 'vulnerable' will be set with the value 'CVE-2014-6271' if the host is vulnerable. Note how this variable tagged with 'inventory' and 'attribute_name='. These are special meta tags that CFEngine Enterprise uses in order to display information.
Deploy the policy
As noted previously, in this example we will use autorun for simplicity. Please
ensure that the class "services_autorun" is defined. The easiest way to do this
is to change "services_autorun" expression => "!any";
to "services_autorun"
expression => "any";
in def.cf
.
Once you have autorun enabled you need only save the policy into
services/autorun/inventory_CVE_2014_6271.cf
.
Report on affected system inventory
Within 20 minutes of deploying the policy you should be able to see results in the Inventory Reporting interface.
A new Inventory attribute 'Vulnerable CVE(s)' is available.
Report showing CVEs that each host is vulnerable to.
Chart the Vulnerable CVE(s) and get a visual breakdown.
Build Dashboard Widget with Alerts
Let's add alerts for CVE(s) to the dashboard.
Give the dashboard widget a name.
Configure an general CVE alert for the dashboard.
Add an additional alert for this specific CVE.
See the dashboard alert in action.
Remediate Vulnerabilities
Now that we know the extent of exposure lets ensure bash gets updated on some
of the affected systems. Save the following policy into
services/autorun/remediate_CVE_2014_6271.cf
bundle agent remediate_CVE_2014_6271
{
meta:
"tags" slist => { "autorun" };
classes:
"allow_update" or => { "hub", "host001" };
methods:
allow_update.CVE_2014_6271::
"Upgrade_Bash"
usebundle => package_latest("bash");
}
What does this remediation policy do?
For simplicity of the example this policy defines the class allow_update on hub and host001, but you could use any class that makes sense to you. If the allow_update class is set, and the class CVE_2014_6271 is defined (indicating the host is vulnerable) then the policy ensures that bash is updated to the latest version available.
Report on affected systems inventory after remediation
Within 20 minutes or so of the policy being deployed you will be able to report on the state of remediation.
See the remediation efforts relfected in the dashboard.
Drill down into the dashboard and alert details.
Run an Inventory report to see hosts and their CVE status.
Chart the Vulnerable CVE(s) and get a visual breakdown.
Summary
In this tutorial you have learned how to use the reporting and inventory features of CFEngine Enterprise to discover and report on affected systems before and after remediation efforts.
Create, Modify, and Delete Files
Prerequisites
- Read the tutorial Tutorial for Running Examples
- Ensure you have read and understand the section on how to make an example stand alone
- Ensure you have read the note at the end of that section regarding modification of the body common control to the following:
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
Note: This change is not necessary for supporting each of the examples in this tutorial. It will be included only in those examples that require it.
List Files
Note: The following workflow assumes the directory /home/user already exists. If it does not either create the directory or adjust the example to a path of your choosing.
Create a file /var/cfengine/masterfiles/file_test.cf that includes the following text:
bundle agent list_file { vars: "ls" slist => lsdir("/home/user","test_plain.txt","true"); reports: "ls: $(ls)"; }
Run the following command to remove any existing test file at the location we wish to use for testing this example:
rm /home/user/test_plain.txt
Test to ensure there is no file /home/user/test_plain.txt, using the following command (the expected result is that there should be no file listed at the location /home/user/test_plain.txt):
ls /home/user/test_plain.txt
Run the following command to instruct CFEngine to see if the file exists (the expected result is that no report will be generated (because the file does not exist):
/var/cfengine/bin/cf-agent --no-lock --file /var/cfengine/masterfiles/file_test.cf --bundlesequence list_file
Create a file for testing the example, using the following command:
touch /home/user/test_plain.txt
Run the following command to instruct CFEngine to search for the file (the expected result is that a report will be generated, because the file exists):
/var/cfengine/bin/cf-agent --no-lock --file /var/cfengine/masterfiles/file_test.cf --bundlesequence list_file
Double check the file exists, using the following command (the expected result is that there will be a file listed at the location /home/user/test_plain.txt):
ls /home/user/test_plain.txt
Run the following command to remove the file:
rm /home/user/test_plain.txt
Create a File
bundle agent testbundle
{
files:
"/home/user/test_plain.txt"
perms => system,
create => "true";
}
bundle agent list_file
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
bundle agent list_file_2
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
body perms system
{
mode => "0640";
}
ls /home/user/test_plain.txt
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2
ls /home/user/test_plain.txt
rm /home/user/test_plain.txt
Delete a File
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent testbundle
{
files:
"/home/user/test_plain.txt"
perms => system,
create => "true";
}
bundle agent test_delete
{
files:
"/home/user/test_plain.txt"
delete => tidy;
}
bundle agent list_file
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
bundle agent list_file_2
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
body perms system
{
mode => "0640";
}
rm /home/user/test_plain.txt
ls /home/user/test_plain.txt
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,test_delete,list_file_2
ls /home/user/test_plain.txt
rm /home/user/test_plain.txt
(last command will throw an error because the file doesn't exist!)
Modify a File
rm /home/user/test_plain.txt
ls /home/user/test_plain.txt
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,testbundle,list_file_2
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,list_file_2
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent testbundle
{
files:
"/home/user/test_plain.txt"
perms => system,
create => "true";
}
bundle agent test_delete
{
files:
"/home/user/test_plain.txt"
delete => tidy;
}
bundle agent list_file
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
bundle agent list_file_2
{
vars:
"ls" slist => lsdir("/home/user","test_plain.txt","true");
reports:
"ls: $(ls)";
}
# Finds the file, if exists calls bundle to edit line
bundle agent outer_bundle_1
{
files:
"/home/user/test_plain.txt"
create => "false",
edit_line => inner_bundle_1;
}
# Finds the file, if exists calls bundle to edit line
bundle agent outer_bundle_2
{
files:
"/home/user/test_plain.txt"
create => "false",
edit_line => inner_bundle_2;
}
# Inserts lines
bundle edit_line inner_bundle_1
{
vars:
"msg" string => "Helloz to World!";
insert_lines:
"$(msg)";
}
# Replaces lines
bundle edit_line inner_bundle_2
{
replace_patterns:
"Helloz to World!"
replace_with => hello_world;
}
body replace_with hello_world
{
replace_value => "Hello World";
occurrences => "all";
}
body perms system
{
mode => "0640";
}
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence list_file,test_delete,list_file_2
ls /home/user/test_plain.txt
rm /home/user/test_plain.txt
Copy a File and Edit its Text
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent testbundle
{
files:
"/home/ichien/test_plain.txt"
perms => system,
create => "true";
reports:
"test_plain.txt has been created";
}
bundle agent test_delete
{
files:
"/home/ichien/test_plain.txt"
delete => tidy;
}
bundle agent do_files_exist
{
vars:
"mylist" slist => { "/home/ichien/test_plain.txt", "/home/ichien/test_plain_2.txt" };
classes:
"exists" expression => filesexist("@(mylist)");
reports:
exists::
"test_plain.txt and test_plain_2.txt files exist";
!exists::
"test_plain.txt and test_plain_2.txt files do not exist";
}
bundle agent do_files_exist_2
{
vars:
"mylist" slist => { "/home/ichien/test_plain.txt", "/home/ichien/test_plain_2.txt" };
classes:
"exists" expression => filesexist("@(mylist)");
reports:
exists::
"test_plain.txt and test_plain_2.txt files both exist";
!exists::
"test_plain.txt and test_plain_2.txt files do not exist";
}
bundle agent list_file_1
{
vars:
"ls1" slist => lsdir("/home/ichien","test_plain.txt","true");
"ls2" slist => lsdir("/home/ichien","test_plain_2.txt","true");
"file_content_1" string => readfile( "/home/ichien/test_plain.txt" , "33" );
"file_content_2" string => readfile( "/home/ichien/test_plain_2.txt" , "33" );
reports:
#"ls1: $(ls1)";
#"ls2: $(ls2)";
"Contents of /home/ichien/test_plain.txt = $(file_content_1)";
"Contents of /home/ichien/test_plain_2.txt = $(file_content_2)";
}
bundle agent list_file_2
{
vars:
"ls1" slist => lsdir("/home/ichien","test_plain.txt","true");
"ls2" slist => lsdir("/home/ichien","test_plain_2.txt","true");
"file_content_1" string => readfile( "/home/ichien/test_plain.txt" , "33" );
"file_content_2" string => readfile( "/home/ichien/test_plain_2.txt" , "33" );
reports:
#"ls1: $(ls1)";
#"ls2: $(ls2)";
"Contents of /home/ichien/test_plain.txt = $(file_content_1)";
"Contents of /home/ichien/test_plain_2.txt = $(file_content_2)";
}
bundle agent outer_bundle_1
{
files:
"/home/ichien/test_plain.txt"
create => "false",
edit_line => inner_bundle_1;
}
# Copies file
bundle agent copy_a_file
{
files:
"/home/ichien/test_plain_2.txt"
copy_from => local_cp("/home/ichien/test_plain.txt");
reports:
"test_plain.txt has been copied to test_plain_2.txt";
}
bundle agent outer_bundle_2
{
files:
"/home/ichien/test_plain_2.txt"
create => "false",
edit_line => inner_bundle_2;
}
bundle edit_line inner_bundle_1
{
vars:
"msg" string => "Helloz to World!";
insert_lines:
"$(msg)";
reports:
"inserted $(msg) into test_plain.txt";
}
bundle edit_line inner_bundle_2
{
replace_patterns:
"Helloz to World!"
replace_with => hello_world;
reports:
"Text in test_plain_2.txt has been replaced";
}
body replace_with hello_world
{
replace_value => "Hello World";
occurrences => "all";
}
body perms system
{
mode => "0640";
}
/var/cfengine/bin/cf-agent --no-lock --file ./file_test.cf --bundlesequence test_delete,do_files_exist,testbundle,outer_bundle_1,copy_a_file,do_files_exist_2,list_file_1,outer_bundle_2,list_file_2
Distribute files from a central location
CFEngine can manage many machines simply by distributing policies to all its hosts. This tutorial describes how to distribute files to hosts from a central policy server location. For this example, we will distribute software patches.
Files are centrally stored on the policy server (hub). In our example, they are stored in /storage/patches
.
These patch files must also exist on the agent host (client) in /storage/deploy/patches
. To do this,
perform the following instructions:
Check out masterfiles from your central repository
CFEngine stores the master copy of all policy in the /var/cfengine/masterfiles
directory.
Ensure that you are working with the latest version of your masterfiles
.
git clone url
or
git pull origin master
Make policy changes
Define locations
Before files can be copied we must know where files should be copied from and where files should be copied to. If these locations are used by multiple components, then defining them in a common bundle can reduce repetition. Variables and classes that are defined in common bundles are accessible by all CFEngine components. This is especially useful in the case of file copies because the same variable definition can be used both by the policy server when granting access and by the agent host when performing the copy.
The policy framework includes a common bundle called def
. In this example, we
will add two variables--dir_patch_store
and dir_patch_deploy
--to this existing bundle.
These variables provide path definitions for storing and deploying patches.
Add the following variable information to the masterfiles/def.cf
file:
"dir_patch_store"
string => "/storage/patches",
comment => "Define patch files source location",
handle => "common_def_vars_dir_patch_store";
"dir_patch_deploy"
string => "/storage/deploy/patches",
comment => "Define patch files deploy location",
handle => "common_def_vars_dir_patch_deploy";
}
These common variables can be referenced from the rest of the policy by using their fully
qualified names,
$(def.dir_patch_store)
and $(def.dir_patch_deploy)
Grant file access
Access must be granted before files can be copied. The right to access a file
is provided by cf-serverd
, the server component of CFEngine. Enter access information using the access
promise type in the bundle server access_rules
section. This section is located in
controls/cf_serverd.cf
in the policy framework.
For our example, add the following information to controls/cf_serverd.cf
:
"$(def.dir_patch_store)"
handle => "server_access_grant_locations_files_patch_store_for_hosts",
admit => { ".*$(def.domain)", @(def.acl) },
comment => "Hosts need to download patch files from the central location";
Create a custom library for reusable synchronization policy
You might need to frequently synchronize or copy a directory structure from the policy server to an agent host. Thus, identifying reusable parts of policy and abstracting them for later use is a good idea. This information is stored in a custom library.
Create a custom library called lib/custom/files.cf
. Add the following content:
bundle agent sync_from_policyserver(source_path, dest_path)
# @brief Sync files from the policy server to the agent
#
# @param source_path Location on policy server to copy files from
# @param dest_path Location on agent host to copy files to
{
files:
"$(dest_path)/."
handle => "sync_from_policy_server_files_dest_path_copy_from_source_path_sys_policy_hub",
copy_from => sync_cp("$(source_path)", "$(sys.policy_hub)"),
depth_search => recurse("inf"),
comment => "Ensure files from $(sys.policy_hub):$(source_path) exist in $(dest_path)";
}
This reusable policy will be used to synchronize a directory on the policy server to a directory on the agent host.
Create a patch policy
Organize in a way that makes the most sense to you and your team. We recommend organizing policy by services.
Create services/patching.cf
with the following content:
# Patching Policy
bundle agent patching
# @brief Ensure various aspects of patching are handeled
# We can break down the various parts of patching into separate bundles. This
# allows us to become less overwhelmed by details if numerous specifics
# exist in one or more aspect for different host classifications.
{
methods:
"Patch Distribution"
handle => "patching_methods_patch_distribution",
usebundle => "patch_distribution",
comment => "Ensure patches are properly distributed";
}
bundle agent patch_distribution
# @brief Ensures that our patches are distributed to the proper locations
{
files:
"$(def.dir_patch_deploy)/."
handle => "patch_distribution_files_def_dir_patch_deploy_exists",
create => "true",
comment => "If the destination directory does not exist, we have no place
to which to copy the patches.";
methods:
"Patches"
handle => "patch_distribution_methods_patches_from_policyserver_def_dir_patch_store_to_def_dir_patch_deploy",
usebundle => sync_from_policyserver("$(def.dir_patch_store)", "$(def.dir_patch_deploy)"),
comment => "Patches need to be present on host systems so that we can use
them. By convention we use the policy server as the central
distribution point.";
}
The above policy contains two bundles. We have separated a top-level patching
bundle from a more specific patch_distribution
bundle. This is an
illustration of how to use bundles in order to abstract details. You
might, for example, have some hosts that you don’t want to fully
synchronize so you might use a different method or copy from a
different path. Creating numerous bundles allows you to move those details away from the top
level of what is involved in patching. If people are interested in what
is involved in patch distribution, they can view that bundle for specifics.
Integrate the policy
Now that all the pieces of the policy are in place, they must be integrated
into the policy so they can be activated. Add each policy file to the inputs
section which is found under body common control
. Once the policy file is included in
inputs, the bundle can be activated. Bundles can be activated by adding them to either the
bundlesequence
or they can be called as a methods
type promise.
Add the following entries to promises.cf
under body common control
-> inputs
:
"lib/custom/files.cf",
"services/patching.cf",
and the following to promises.cf
under body common control
-> bundlesequence
:
"patching",
Now that all of the policy has been edited and is in place, check for syntax errors by
running cf-promises -f ./promises.cf
. This promise is activated from the service_catalogue
bundle.
Commit Changes
Set up trackers in the Mission Portal (Enterprise Users Only)
Before committing the changes to your repository, log in to the Mission Portal and set up a Tracker so that you can see the policy as it goes out. To do this, perform the following:
Navigate to the Hosts section. Select All hosts. Select the Events tab, located in the right-hand panel. Click Add new tracker.
Name it Patch Failure. Set the Report Type to Promise not Kept. Under Watch, enter .patch. Set the Start Time to Now and then click Done to close the Start Time window. Click Start to save the new tracker. This tracker watches for any promise handle that includes the string patch where a promise is not kept.
Add another tracker called Patch Repaired. Set the Report Type to Promise Repaired. Enter the same values as above for Watch and Start Time. Click Start to save the new tracker. This tracker allows you to see how the policy reacts as it is activated on your infrastructure.
Deploy changes (Enterprise and Community Users)
Always inspect what you expect. git status
shows the status of your current branch.
git status
Inspect the changes contained in each file. Once satisfied, add them to git's commit staging area.
git diff file
git add file
Iterate over using git diff, add, and status until all of the changes that you expected are listed as Changes to be committed. Check the status once more before you commit the changes.
git status
Commit the changes to your local repository.
git commit
Push the changes to the central repository so they can be pulled down to your policy server for distribution.
git push origin master
Tags for variables, classes, and bundles
Introduction
CFEngine 3.6.0 makes great use of the meta
attribute to set tags
on variables and classes. While that attribute was always in the
language, only in 3.6.0 is it fully utilized
Furthermore, bundles can now be tagged and found by tag, so you can construct a fully dynamic bundle sequence.
Problem statement
We'd like to apply tags to variables and classes for many purposes, from stating their provenance (whence they came, why they exist, and how they can be used) to filtering them based on tags.
We'd also like to be able to include all the files in a directory and then run all the discovered bundles if they are tagged appropriately.
Syntax
Tagging variables and classes is easy with the meta
attribute.
Here's an example that sets the inventory
tag on a variable and
names the attribute that it represents. This one is actually built
into the standard 3.6.0 inventory policy under
masterfiles/inventory/any.cf
, so you have it available out
of the box in either Community or Enterprise.
bundle agent cfe_autorun_inventory_listening_ports
# @brief Inventory the listening ports
#
# This bundle uses `mon.listening_ports` and is always enabled by
# default, as it runs instantly and has no side effects.
{
vars:
"ports" slist => { @(mon.listening_ports) },
meta => { "inventory", "attribute_name=Ports listening" };
}
In the Enterprise Mission Portal, you can then make a report for "Ports listening" across all your machines. For more details, see https://cfengine.com/docs/master/guide-enterprise-cfengine-guide-mission-portal-reporting.html
Class tags work exactly the same way, you just apply them to a
classes
promise with the meta
attribute.
Tagging bundles is different because you have to use the meta
promise type (different from the meta
attribute).
An example is easiest:
bundle agent run_deprecated
{
meta:
"tags" slist => { "deprecated" };
}
This declares an agent bundle with a single tag.
Functions
Several new functions exist to give you access to variable and class tags, and to find classes and variables with tags.
classesmatching
: this used to be somewhat available with theallclasses.txt
file. You can now call a function to get all the defined classes, optionally filtering by name and tags. See https://cfengine.com/docs/master/reference-functions-classesmatching.htmlgetvariablemetatags
: get the tags of a variable as an slist. See https://cfengine.com/docs/master/reference-functions-getvariablemetatags.htmlvariablesmatching
: just likeclassesmatching
but for variables. See https://cfengine.com/docs/master/reference-functions-variablesmatching.htmlgetclassmetatags
: get the tags of a class as an slist. See https://cfengine.com/docs/master/reference-functions-getclassmetatags.html
There is also a new function to find bundles.
bundlesmatching
: find the bundles matching some tags. See https://cfengine.com/docs/master/reference-functions-bundlesmatching.html (the example shows how you'd find adeprecated
bundle likerun_deprecated
earlier).
Module protocol
The module protocol has been extended to support tags. You set the tags on a line and they persist for every subsequent variable or class.
^meta=inventory
+x
=a=100
^meta=report,attribute_name=My vars
+y
=n=100
This will create class x
and variable a
with tag inventory
.
Then it will create class y
and variable b
with tags report
and
attribute_name=My vars
.
Enterprise Reporting with tags
In CFEngine Enterprise 3.6.0, you can build reports based on tagged variables and classes.
Please see https://cfengine.com/docs/master/guide-enterprise-cfengine-guide-mission-portal-reporting.html for a full tutorial, including troubleshooting possible errors. In short, this is extremely easy as long as you:
allow classes and variables with the
inventory
andreport
tags to be reported incontrols/cf_serverd.cf
. This is already the case if you install the stock CFEngine 3.6.0 packages.ensure there are no report collection issues due to firewalls (TCP port 5308 must be open from the hub to the agents, for instance)
Dynamic bundlesequence
Dynamic bundlesequences are extremely easy. You simply say
vars:
"bundles" slist => bundlesmatching("regex", "tag1", "tag2", ...);
methods:
"run $(bundles)" usebundle => $(bundles);
Then every bundle matching the regular expression regex
and all
the tags will be found and run.
Note that the discovered bundle names will have the namespace prefix,
e.g. default:mybundle
. The regular expression has to match that. So
mybundle
as the regular expression would not work. See
https://cfengine.com/docs/master/reference-functions-bundlesmatching.html
for another detailed example.
In fact we found this so useful we implemented services autorun in the masterfiles policy framework.
There is only one thing to beware. All the bundles have to have the same number of arguments (0 in the case shown). Otherwise you will get a runtime error and CFEngine will abort. We recommend only using 0-argument bundles in a dynamic sequence to reduce this risk.
Summary
Tagging variables and classes and bundles in CFEngine 3.6.0 is easy and allows more dynamic behavior than ever before. Try it out and see for yourself how it will change the way you use and think about system configuration policy and CFEngine.
Custom Inventory
This tutorial will show you how to add custom inventory attributes that can be leveraged in policy and reported on in the CFEngine Enterprise Mission Portal. For a more detailed overview on how the inventory system works please reference CFEngine 3 inventory modules.
Overview
This tutorial provides instructions for the following:
Choose an Attribute to Inventory
Writing inventory policy is incredibly easy. Simply add the inventory
and
attribute_name=
tags to any variable or namespace scoped
classes.
In this tutorial we will add Owner
information into the inventory. In this
example we will use a simple shared flat file data source
/vagrant/inventory_owner.csv
.
On your hosts create /vagrant/inventory_owner.csv
with the following contnet:
hub, Operations Team <ops@cfengine.com>
host001, Development <dev@cfengine.com>
Note: I am using the [CFEngine Enterprise Vagrant Environment][Installing Enterprise Vagrant Environemt] and files located in the vagrant project directory are automatically available to all hosts.
Create and Deploy Inventory Policy
Now that each of your hosts has access to a data source that provides the Owner information we will write an inventory policy to report that information.
Create /var/cfengine/masterfiles/services/tutorials/inventory/owner.cf
with the
following content:
bundle common tutorials_inventory_owner
# @brief Inventory Owner information
# @description Inventory owner information from `/vagrant/inventory_owner.csv`.
{
classes:
"have_owner"
expression => isvariable("my_owner");
vars:
"data_source" string => "/vagrant/inventory_owner.csv";
"owners" data => data_readstringarray( $(data_source), "", ", ", 100, 512 );
"my_owner"
string => "$(owners[$(sys.uqhost)][0])",
meta => { "inventory", "attribute_name=Owner" },
comment => "We need to tag the owner information so that it is correctly
reported via the UI.";
reports:
inform_mode.have_owner::
"$(this.bundle): Discovered Owner='$(my_owner)'";
}
Note: For the simplicity of this tutorial we assume that masterfiles is not configured for policy updates from a git repository. If it is, please add the policy to your repository and ensure it gets to its final destination as needed.
This policy will not be activated until it has been included in inputs. For simplicity we will be adding it to common control, but note that files can include other files through the use of inputs in file control.
Add 'services/tutorials/inventory/owner.cf' to inputs and
'tutorials_inventory_owner' to the bundlesequence in
common control found in /var/cfengine/masterfiles/promises.cf
as shown below.
body common control
{
bundlesequence => {
"inventory_control",
@(inventory.bundles),
tutorials_inventory_owner,
def,
cfe_internal_hub_vars,
# snipped for brevity
service_catalogue,
};
inputs => {
# File definition for global variables and classes
"def.cf",
# Inventory policy
@(inventory.inputs),
"services/tutorials/inventory/owner.cf",
# snipped for brevity
# List of services here
"services/init_msg.cf",
"services/file_change.cf",
};
}
When you finish its a good idea to run cf-promises
(syntax analyzer) to
ensure the policy validates. You can also perform a manual policy run and check
that the correct owner is discovered.
Syntax Check:
[root@hub ~]# cf-promises -f /var/cfengine/masterfiles/promises.cf
[root@hub ~]#
No output and return code 0 indicate the policy was successfully validated.
Manual Policy Run:
[root@hub ~]# cf-agent -KIf /var/cfengine/masterfiles/promises.cf -b tutorials_inventory_owner
2014-06-16T19:24:58+0000 info: Using command line specified bundlesequence
R: tutorials_inventory_owner: Discovered Owner='Operations Team <ops@cfengine.com>'
Here I have run the policy without locks (-K) in inform mode (-I), using a specific policy entry (-f) and activating only a specific bundle (-b). The inform output helps me confirm that my hubs owner is discovered from our CSV properly.
Run Reports
Once you have integrated the policy into promises.cf it will be distributed and run by all agents. Once the hub has had a chance to collect reports the 'Owner' attribute will be available to select as a Table column for Inventory Reports. Custom attributes appear under the 'User defined' section.
Note: It may take up to 15 minutes for your custom inventory attributes to be collected and made available for reporting.
You will see the host owner as shown in the following report.
Configure and Deploy a Policy Using Sketches
Note: This tutorial is for Enterprise users who have access to the Mission Portal application. CFEngine must be up and running in order to complete this tutorial.
Overview
In this tutorial, we want to implement the following policy: The iscsi-initatior-utils software package should be present/installed on all hosts. Since CFEngine has a sketch that can generate this policy, we will use it to deploy our policy. (Note that you may use an alternate package from your system's package repository.)
A sketch defines configurable and reusable policy. You can use sketches to implement, activate, or enforce policy. Sketches are written in the CFEngine policy language; you can use them simply by installing them, configuring them using the appropriate parameters and environments, and deploying them on your infrastructure.
This tutorial provides instructions for the following:
Configure and deploy a policy using sketches in the Design Center
Verify sketch deployment using Reports in the Mission Portal
Create a query to narrow results using Report Builder in the Mission Portal
Configure and deploy a policy using sketches in the Design Center
We will activate the Packages sketch which allows you to install selected software packages on specific hosts. A sketch must include a parameter set and an environment(s), both of which we will set in the example below. Make certain that the packages you select are included in the package repository. (The package in our example below is available in the Centos package repository. You can select any package that is available through your operating system's package repository.)
Log in to the Mission Portal. Select Design Center from the left sidebar.
Select the Packages::installed sketch. Use the following values:
a. Descriptive name: Enter Install iSCSI. This allows you to recognize the activation (and its goal) later, as the Design Center uses this name when it commits changes to Git.
b. Packages that should be installed: Fill in the name of the package that must be installed. For this example, use iscsi-initiator-utils. This is the parameter set.
c. Hosts to activate on: Click Select category to display host options. Select All hosts for our example. All host names appear. This is the environment in which the sketch must be activate.
Here is an example:
Click Activate. This deploys the sketch to all hosts.
Enter a description in the Commit your changes window that appears. The Design Center uses this comment for version control when it commits changes to Git. Click Commit to complete the change.
When a sketch is activated, the following occurs:
The policy that is generated when the sketch is activated gets committed to your Git repository. This allows you to keep track of who has made what changes, and when, and why.
The policy server is typically configured to check the Git repository every five minutes to ensure that it is running the latest version of available policies. This process can be handled manually as well.
The hosts check with the policy server for updated policy. They also work on default intervals of five minutes.
The policy server collects information from the agents on the hosts to obtain insight into the progress with executing the sketch. The information it collects is used to update the information in the Design Center.
In total, this process might take a few minutes to converge to the correct state for all hosts. The process is very scalable, so even if it takes a few minutes for the two servers in this example to be updated, it does not take much longer to update 2,000 servers. If you check back with the Packages sketch in the middle of the activation process, you will see a message that reads Status: Being Activated. Upon successful completion, the window should look like this:
Now that the sketch is deployed, CFEngine continuously verifies that it is maintained. It checks 365 days per year, 24 hours per day, 12 times per hour to make certain this package is on all of the hosts. If the package is removed, it is added within five minutes, and CFEngine creates reports that it made a repair. Thus, the state of the overall system is known and stable and system drift is avoided. This works for 2, 20, 200, 2,000 or 20,000 servers.
Verify sketch deployment using Reports in the Mission Portal
The Mission Portal contains standard Reports to facilite systems monitoring and management. We will use the Software installed Report to verify that the Packages sketch we just activated has been deployed.
Log in to the Mission Portal. Select Reports from the left sidebar.
Select the Software installed report from the list of reports that appear.
Scroll through the Software installed report to find the iscsi-initiator-utils software. To hasten your search, click the SoftwareName column in order to alphabetize the results. Another option is to create a query in the Report Builder, which is described below.
The table shows that the iscsi-initiator-utils software is installed on both hosts:
Create a query to narrow results using Report Builder in the Mission Portal
Use the Report builder to create queries. In our example, we will create a query to verify that the Packages sketch we just activated was deployed and that our software was installed.
Log in to the Mission Portal. Select Reports from the left sidebar.
Click New report to open the Report builder.
Under Tables, select Hosts and then Software.
Enter the following Field values for the Hosts and Software tables:
a. For Hosts Fields, select Host name.
b. For Software Fields, select, and in the order shown: Software name, Software version, Software architecture. Note that the Mission Portal creates Fields in the order that you select them.
Under Filters, click Add. Under Software, select Software name. For the Software name filter, select equals = and then enter iscsi-initiator-utils.
Click Show Query to view the SQL query that is generated from your selection.
Your completed query should look similar to this example:
Click Run. The Results reveal that the policy was generated when the sketch was deployed and activated on both hosts.
File Comparison
- Add the policy contents (also can be downloaded from file_compare_test.cf) to a new file, such as /var/cfengine/masterfiles/file_test.cf.
Run the following commands as root on the command line:
export AOUT_BIN="a.out" export GCC_BIN="/usr/bin/gcc" export RM_BIN="/bin/rm" export WORK_DIR=$HOME export CFE_FILE1="test_plain_1.txt" export CFE_FILE2="test_plain_2.txt" /var/cfengine/bin/cf-agent /var/cfengine/masterfiles/file_test.cf --bundlesequence robot,global_vars,packages,create_aout_source_file,create_aout,test_delete,do_files_exist_1,create_file_1,outer_bundle_1,copy_a_file,do_files_exist_2,list_file_1,stat,outer_bundle_2,list_file_2
Here is the order in which bundles are called in the command line above (some other support bundles are contained within file_test.cf but are not included here):
- robot - demonstrates use of
reports
. - global_vars - sets up some global variables for later use.
- packages - installs packages that will be used later on.
- create_aout_source_file - creates a source file.
- create_aout - compiles the source file.
- test_delete - deletes a file.
- do_files_exist_1 - checks the existence of files.
- create_file_1 - creates a file.
- outer_bundle_1 - adds text to a file.
- copy_a_file - copies the file.
- do_files_exist_2 - checks the existence of both files.
- list_file_1 - shows the contents of each file.
- stat - uses the stat command and the aout application to compare modified times of both files.
- outer_bundle_2 - modifies the contents of the second file.
- list_file_2 - shows the contents of both files and uses CFEngine functionality to compare the modified time for each file.
robot
Demonstrates use of reports
, using an ascii art representation of the CFEngine robot.
global_vars
Sets up some global variables that are used frequently by other bundles.
bundle common global_vars
{
vars:
"gccexec" string => getenv("GCC_BIN",255);
"rmexec" string => getenv("RM_BIN",255);
"aoutbin" string => getenv("AOUT_BIN",255);
"workdir" string => getenv("WORK_DIR",255);
"aoutexec" string => "$(workdir)/$(aoutbin)";
"file1name" string => getenv("CFE_FILE1",255);
"file2name" string => getenv("CFE_FILE2",255);
"file1" string => "$(workdir)/$(file1name)";
"file2" string => "$(workdir)/$(file2name)";
classes:
"gclass" expression => "any";
}
packages
Ensures that the gcc package is installed, for later use by the create_aout bundle.
bundle agent packages
{
vars:
"match_package" slist => {
"gcc"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => yum;
reports:
gclass::
"Package gcc installed";
"*********************************";
}
create_aout_source_file
Creates the c source file that will generate a binary application in create_aout.
bundle agent create_aout_source_file
{
# This bundle creates the source file that will be compiled in bundle agent create_aout.
# See that bunlde's comments for more information.
vars:
# An slist is used here instead of a straight forward string because it doesn't seem possible to create
# line endings using \n when using a string to insert text into a file.
"c" slist => {"#include <stdlib.h>","#include <stdio.h>","#include <sys/stat.h>","#include <string.h>","void main()","{char file1[255];strcpy(file1,\"$(global_vars.file1)\");char file2[255];strcpy(file2,\"$(global_vars.file2)\");struct stat time1;int i = lstat(file1, &time1);struct stat time2;int j = lstat(file2, &time2);if (time1.st_mtime < time2.st_mtime){printf(\"Newer\");}else{if(time1.st_mtim.tv_nsec < time2.st_mtim.tv_nsec){printf(\"Newer\");}else{printf(\"Not newer\");}}}"};
files:
"$(global_vars.workdir)/a.c"
perms => system,
create => "true",
edit_line => Insert("@(c)");
reports:
"The source file $(global_vars.workdir)/a.c has been created. It will be used to compile the binary a.out, which will provide more accurate file stats to compare two files than the built in CFEngine functionality for comparing file stats, including modification time. This information will be used to determine of the second of the two files being compared is newer or not.";
"*********************************";
}
create_aout
This bundle creates a binary application from the source in create_aout_source_file that uses the stat library to compare two files, determine if the modified times are different, nd whether the second file is newer than the first.
The difference between this application and using CFEngine's built in support for getting file stats is that normally the accuracy is only to the second of the modified file time but in order to better compare two files requires parts of a second as well. The stat library provides the extra support for retrieving the additional information required.
bundle agent create_aout
{
classes:
"doesfileacexist" expression => fileexists("$(global_vars.workdir)/a.c");
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
# Removes any previous binary
"rmaout" string => execresult("$(global_vars.rmexec) $(global_vars.aoutexec)","noshell");
doesfileacexist::
"compilestr" string => "$(global_vars.gccexec) $(global_vars.workdir)/a.c -o $(global_vars.aoutexec)";
"gccaout" string => execresult("$(compilestr)","noshell");
reports:
doesfileacexist::
"gcc output: $(gccaout)";
"Creating aout using $(compilestr)";
!doesfileacexist::
"Cannot compile a.out, $(global_vars.workdir)/a.c does not exist.";
doesaoutexist::
"The binary application aout has been compiled from the source in the create_aout_source_file bundle. It uses the stat library to compare two files, determine if the modified times are different, and whether the second file is newer than the first. The difference between this application and using CFEngine's built in support for getting file stats (e.g. filestat, isnewerthan), which provides file modification time accurate to a second. However, in order to better compare two files might sometimes require parts of a second as well. The stat library provides the extra support for retrieving the additional information required to get better accuracy (down to parts of a second), and is utilized by the binary application a.out that is compiled within the create_aout bundle.";
"*********************************";
}
test_delete
Deletes any previous copy of the test files used in the example.
bundle agent test_delete
{
files:
"$(global_vars.file1)"
delete => tidy;
}
do_files_exist_1
Verifies whether the test files exist or not.
bundle agent do_files_exist_1
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
methods:
doesfile1exist::
"any" usebundle => delete_file("$(global_vars.file1)");
doesfile2exist::
"any" usebundle => delete_file("$(global_vars.file2)");
reports:
!doesfile1exist::
"$(global_vars.file1) does not exist.";
doesfile1exist::
"$(global_vars.file1) did exist. Call to delete it was made.";
!doesfile2exist::
"$(global_vars.file2) does not exist.";
doesfile2exist::
"$(global_vars.file2) did exist. Call to delete it was made.";
}
create_file_1
Creates the first test file, as an empty file.
bundle agent create_file_1
{
files:
"$(global_vars.file1)"
perms => system,
create => "true";
reports:
"$(global_vars.file1) has been created";
}
outer_bundle_1
Adds some text to the first test file.
bundle agent outer_bundle_1
{
files:
"$(global_vars.file1)"
create => "false",
edit_line => inner_bundle_1;
}
copy_a_file
Makes a copy of the test file.
bundle agent copy_a_file
{
files:
"$(global_vars.file2)"
copy_from => local_cp("$(global_vars.file1)");
reports:
"$(global_vars.file1) has been copied to $(global_vars.file2)";
}
do_files_exist_2
Verifies that both test files exist.
bundle agent do_files_exist_2
{
methods:
"any" usebundle => does_file_exist($(global_vars.file1));
"any" usebundle => does_file_exist($(global_vars.file2));
}
list_file_1
Reports the contents of each test file.
bundle agent list_file_1
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
reports:
"*********************************";
}
exec_aout
bundle agent exec_aout
{
classes:
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
doesaoutexist::
"aout" string => execresult("$(global_vars.aoutexec)","noshell");
reports:
doesaoutexist::
"*********************************";
"$(global_vars.aoutbin) determined that $(global_vars.file2) is $(aout) than $(global_vars.file1)";
"*********************************";
!doesaoutexist::
"Executable $(global_vars.aoutbin) does not exist.";
}
stat
Compares the modified time of each test file using the binary application compiled in create_aout to see if it is newer.
bundle agent stat
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
vars:
doesfile1exist::
"file1" string => "$(global_vars.file1)";
"file2" string => "$(global_vars.file2)";
"file1_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file1)","noshell");
"file1_split1" slist => string_split($(file1_stat)," ",3);
"file1_split2" string => nth("file1_split1",1);
"file1_split3" slist => string_split($(file1_split2),"\.",3);
"file1_split4" string => nth("file1_split3",1);
"file2_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file2)","noshell");
"file2_split1" slist => string_split($(file2_stat)," ",3);
"file2_split2" string => nth("file2_split1",1);
"file2_split3" slist => string_split($(file2_split2),"\.",3);
"file2_split4" string => nth("file2_split3",1);
methods:
"any" usebundle => exec_aout();
reports:
doesfile1exist::
"Parts of a second extracted extracted from stat for $(file1): $(file1_split4). Full stat output for $(file1): $(file1_stat)";
"Parts of a second extracted extracted from stat for $(file2): $(file2_split4). Full stat output for $(file2): $(file2_stat)";
"Using the binary Linux application stat to compare two files can help determine if the modified times between two files are different. The difference between the stat application using its additional flags and using CFEngine's built in support for getting and comparing file stats (e.g. filestat, isnewerthan) is that normally the accuracy is only to the second of the file's modified time. In order to better compare two files requires parts of a second as well, which the stat command can provide with some additional flags. Unfortunately the information must be extracted from the middle of a string, which is what the stat bundle accomplishes using the string_split and nth functions.";
"*********************************";
!doesfile1exist::
"stat: $(global_vars.file1) and probably $(global_vars.file2) do not exist.";
}
outer_bundle_2
Modifies the text in the second file.
bundle agent outer_bundle_2
{
files:
"$(global_vars.file2)"
create => "false",
edit_line => inner_bundle_2;
}
list_file_2
Uses filestat
and isnewerthan
to compare the two test files to see if the second one is newer. Sometimes the modifications already performed, such as copy and modifying text, happen too quickly and filestat and isnewerthan may both report that the second test file is not newer than the first, while the more accurate stat based checks in the stat bundle (see step 12) will recognize the difference.
bundle agent list_file_2
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
classes:
"ok" expression => isgreaterthan(filestat("$(global_vars.file2)","mtime"),filestat("$(global_vars.file1)","mtime"));
"newer" expression => isnewerthan("$(global_vars.file2)","$(global_vars.file1)");
reports:
"*********************************";
ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
newer::
"Using isnewerthan determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!newer::
"Using isnewerthan determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
}
Full Policy
body common control {
inputs => {
"libraries/cfengine_stdlib.cf",
};
}
bundle agent robot
{
reports:
" 77777777777";
" 77777777777777";
" 777 7777 777";
" 7777777777777";
" 777777777777";
" 777 7777 77";
" ";
" ZZZZ ZZZ ZZZZ ZZZ ZZZZ";
" ZZZZZ ZZZZZZZZZZZZZZ ZZZZZ ";
" ZZZZZZZ ZZZZZZZZZZZZZ ZZZZZZZ";
" ZZZZ ------------- ZZZZZZ";
" ZZZZZ !CFENGINE! ZZZZZ";
" ZZZZ ------------- ZZZZZ";
" ZZZZZ ZZZZZZZZZZZZZZ ZZZZZ";
" ZZZ ZZZZZZZZZZZZZ ZZZ";
" ZZZZZ ZZZZZZZZZZZZZ ZZZZZ";
" ..?ZZZ+,,,,, ZZZZZZZZZZZZZZ ZZZZZ";
" ...ZZZZ~ ,:: ZZZZZZZZZZZZZ ZZZZ";
" ..,ZZZZZ,:::::: ZZZZZ";
" ZZZ ZZZ";
" ~ ===+";
" ZZZZZZZZZZZZZI??";
" ZZZZZZZZZZZZZ$???";
" 7Z$+ ZZ ZZZ???II";
" ZZZZZ+ ZZZZZIIII";
" ZZZZZ ZZZZZ III77";
" +++ +$ZZ??? ZZZ";
" +++??ZZZZZIIIIZZZZZ";
" ????ZZZZZIIIIZZZZZ";
" ??IIZZZZ 7777ZZZ";
" IIZZZZZ 77ZZZZZ";
" I$ZZZZ $ZZZZ";
}
bundle common global_vars
{
vars:
"gccexec" string => getenv("GCC_BIN",255);
"rmexec" string => getenv("RM_BIN",255);
"aoutbin" string => getenv("AOUT_BIN",255);
"workdir" string => getenv("WORK_DIR",255);
"aoutexec" string => "$(workdir)/$(aoutbin)";
"file1name" string => getenv("CFE_FILE1",255);
"file2name" string => getenv("CFE_FILE2",255);
"file1" string => "$(workdir)/$(file1name)";
"file2" string => "$(workdir)/$(file2name)";
classes:
"gclass" expression => "any";
}
bundle agent packages
{
vars:
"match_package" slist => {
"gcc"
};
packages:
"$(match_package)"
package_policy => "add",
package_method => yum;
reports:
gclass::
"Package gcc installed";
"*********************************";
}
bundle agent create_aout_source_file
{
# This bundle creates the source file that will be compiled in bundle agent create_aout.
# See that bunlde's comments for more information.
vars:
# An slist is used here instead of a straight forward string because it doesn't seem possible to create
# line endings using \n when using a string to insert text into a file.
"c" slist => {"#include <stdlib.h>","#include <stdio.h>","#include <sys/stat.h>","#include <string.h>","void main()","{char file1[255];strcpy(file1,\"$(global_vars.file1)\");char file2[255];strcpy(file2,\"$(global_vars.file2)\");struct stat time1;int i = lstat(file1, &time1);struct stat time2;int j = lstat(file2, &time2);if (time1.st_mtime < time2.st_mtime){printf(\"Newer\");}else{if(time1.st_mtim.tv_nsec < time2.st_mtim.tv_nsec){printf(\"Newer\");}else{printf(\"Not newer\");}}}"};
files:
"$(global_vars.workdir)/a.c"
perms => system,
create => "true",
edit_line => Insert("@(c)");
reports:
"The source file $(global_vars.workdir)/a.c has been created. It will be used to compile the binary a.out, which will provide more accurate file stats to compare two files than the built in CFEngine functionality for comparing file stats, including modification time. This information will be used to determine of the second of the two files being compared is newer or not.";
"*********************************";
}
bundle edit_line Insert(name)
{
insert_lines:
"$(name)";
}
bundle agent create_aout
{
classes:
"doesfileacexist" expression => fileexists("$(global_vars.workdir)/a.c");
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
# Removes any previous binary
"rmaout" string => execresult("$(global_vars.rmexec) $(global_vars.aoutexec)","noshell");
doesfileacexist::
"compilestr" string => "$(global_vars.gccexec) $(global_vars.workdir)/a.c -o $(global_vars.aoutexec)";
"gccaout" string => execresult("$(compilestr)","noshell");
reports:
doesfileacexist::
"gcc output: $(gccaout)";
"Creating aout using $(compilestr)";
!doesfileacexist::
"Cannot compile a.out, $(global_vars.workdir)/a.c does not exist.";
doesaoutexist::
"The binary application aout has been compiled from the source in the create_aout_source_file bundle. It uses the stat library to compare two files, determine if the modified times are different, and whether the second file is newer than the first. The difference between this application and using CFEngine's built in support for getting file stats (e.g. filestat, isnewerthan), which provides file modification time accurate to a second. However, in order to better compare two files might sometimes require parts of a second as well. The stat library provides the extra support for retrieving the additional information required to get better accuracy (down to parts of a second), and is utilized by the binary application a.out that is compiled within the create_aout bundle.";
"*********************************";
}
bundle agent test_delete
{
files:
"$(global_vars.file1)"
delete => tidy;
}
bundle agent delete_file(fname)
{
files:
"$(fname)"
delete => tidy;
reports:
"Deleted $(fname)";
}
body contain del_file
{
useshell => "useshell";
}
bundle agent do_files_exist_1
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
methods:
doesfile1exist::
"any" usebundle => delete_file("$(global_vars.file1)");
doesfile2exist::
"any" usebundle => delete_file("$(global_vars.file2)");
reports:
!doesfile1exist::
"$(global_vars.file1) does not exist.";
doesfile1exist::
"$(global_vars.file1) did exist. Call to delete it was made.";
!doesfile2exist::
"$(global_vars.file2) does not exist.";
doesfile2exist::
"$(global_vars.file2) did exist. Call to delete it was made.";
}
bundle agent create_file_1
{
files:
"$(global_vars.file1)"
perms => system,
create => "true";
reports:
"$(global_vars.file1) has been created";
}
bundle agent outer_bundle_1
{
files:
"$(global_vars.file1)"
create => "false",
edit_line => inner_bundle_1;
}
bundle agent copy_a_file
{
files:
"$(global_vars.file2)"
copy_from => local_cp("$(global_vars.file1)");
reports:
"$(global_vars.file1) has been copied to $(global_vars.file2)";
"*********************************";
}
bundle agent do_files_exist_2
{
methods:
"any" usebundle => does_file_exist($(global_vars.file1));
"any" usebundle => does_file_exist($(global_vars.file2));
}
bundle agent does_file_exist(filename)
{
vars:
"filestat" string => filestat("$(filename)","mtime");
classes:
"fileexists" expression => fileexists("$(filename)");
reports:
fileexists::
"$(filename) exists. Last Modified Time = $(filestat).";
!fileexists::
"$(filename) does not exist";
}
bundle agent list_file_1
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
reports:
"*********************************";
}
bundle agent exec_aout
{
classes:
"doesaoutexist" expression => fileexists("$(global_vars.aoutbin)");
vars:
doesaoutexist::
"aout" string => execresult("$(global_vars.aoutexec)","noshell");
reports:
doesaoutexist::
"*********************************";
"$(global_vars.aoutbin) determined that $(global_vars.file2) is $(aout) than $(global_vars.file1)";
"*********************************";
!doesaoutexist::
"Executable $(global_vars.aoutbin) does not exist.";
}
bundle agent stat
{
classes:
"doesfile1exist" expression => fileexists("$(global_vars.file1)");
"doesfile2exist" expression => fileexists("$(global_vars.file2)");
vars:
doesfile1exist::
"file1" string => "$(global_vars.file1)";
"file2" string => "$(global_vars.file2)";
"file1_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file1)","noshell");
"file1_split1" slist => string_split($(file1_stat)," ",3);
"file1_split2" string => nth("file1_split1",1);
"file1_split3" slist => string_split($(file1_split2),"\.",3);
"file1_split4" string => nth("file1_split3",1);
"file2_stat" string => execresult("/usr/bin/stat -c \"%y\" $(file2)","noshell");
"file2_split1" slist => string_split($(file2_stat)," ",3);
"file2_split2" string => nth("file2_split1",1);
"file2_split3" slist => string_split($(file2_split2),"\.",3);
"file2_split4" string => nth("file2_split3",1);
methods:
"any" usebundle => exec_aout();
reports:
doesfile1exist::
"Parts of a second extracted extracted from stat for $(file1): $(file1_split4). Full stat output for $(file1): $(file1_stat)";
"Parts of a second extracted extracted from stat for $(file2): $(file2_split4). Full stat output for $(file2): $(file2_stat)";
"Using the binary Linux application stat to compare two files can help determine if the modified times between two files are different. The difference between the stat application using its additional flags and using CFEngine's built in support for getting and comparing file stats (e.g. filestat, isnewerthan) is that normally the accuracy is only to the second of the file's modified time. In order to better compare two files requires parts of a second as well, which the stat command can provide with some additional flags. Unfortunately the information must be extracted from the middle of a string, which is what the stat bundle accomplishes using the string_split and nth functions.";
"*********************************";
!doesfile1exist::
"stat: $(global_vars.file1) and probably $(global_vars.file2) do not exist.";
}
bundle agent outer_bundle_2
{
files:
"$(global_vars.file2)"
create => "false",
edit_line => inner_bundle_2;
}
bundle edit_line inner_bundle_1
{
vars:
"msg" string => "Helloz to World!";
insert_lines:
"$(msg)";
reports:
"inserted $(msg) into $(global_vars.file1)";
}
bundle edit_line inner_bundle_2
{
replace_patterns:
"Helloz to World!"
replace_with => hello_world;
reports:
"Text in $(global_vars.file2) has been replaced";
}
body replace_with hello_world
{
replace_value => "Hello World";
occurrences => "all";
}
bundle agent list_file_2
{
methods:
"any" usebundle => file_content($(global_vars.file1));
"any" usebundle => file_content($(global_vars.file2));
classes:
"ok" expression => isgreaterthan(filestat("$(global_vars.file2)","mtime"),filestat("$(global_vars.file1)","mtime"));
"newer" expression => isnewerthan("$(global_vars.file2)","$(global_vars.file1)");
reports:
"*********************************";
ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!ok::
"Using isgreaterthan+filestat determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
newer::
"Using isnewerthan determined that $(global_vars.file2) was modified later than $(global_vars.file1).";
!newer::
"Using isnewerthan determined that $(global_vars.file2) was not modified later than $(global_vars.file1).";
}
bundle agent file_content(filename)
{
vars:
"file_content" string => readfile( "$(filename)" , "0" );
"file_stat" string => filestat("$(filename)","mtime");
reports:
"Contents of $(filename) = $(file_content). Last Modified Time = $(file_stat).";
#"The report on contents will only show new content and modifications. Even if the method is called more than once, if the evaluation is exactly the same as the previous call then there will be no report (possibly because the bundle is not evaluated a second time?).";
}
body perms system
{
mode => "0640";
}
Enterprise API Examples
- Check installation status
- Manage users, roles
- Managing Settings
- Browse host information
- Issue flexible SQL queries against data collected from hosts by the CFEngine Server
- Schedule reports for email and later download
- Enterprise API interaction with Design Center
- Tracking changes performed by CFEngine
See Also: Enterprise API Reference
SQL Query Examples
Synchronous Example: Listing Hostname and IP for Ubuntu Hosts
Request:
curl -k --user admin:admin https://test.cfengine.com/api/query -X POST -d '{ "query": "SELECT Hosts.HostName, Hosts.IPAddress FROM Hosts"}'
Response:
{
"data": [
{
"header": [
{
"columnName": "hostname",
"columnType": "STRING"
},
{
"columnName": "ipaddress",
"columnType": "STRING"
}
],
"query": "select hostname, ipaddress from hosts",
"queryTimeMs": 152,
"rowCount": 1001,
"rows": [
[
"ubuntu10-2.stage.cfengine.com",
"172.20.100.1"
],
[
"ubuntu10-3.stage.cfengine.com",
"172.20.100.2"
],
[
"ubuntu10-4.stage.cfengine.com",
"172.20.100.3"
]
],
}
],
"meta": {
"count": 1,
"page": 1,
"timestamp": 1437051092,
"total": 1
}
}
Subscribed Query Example: Creating A Subscribed Query
Here we create a new query to count file changes by name and have the result
sent to us by email. The schedule field is any CFEngine context expression.
The backend polls subscriptions in a loop and checks whether it's time to
generate a report and send it out. In the following example, user milton
creates a new subscription to a report which he names file-changes-report
,
which will be sent out every Monday night. His boss will get an email with a
link to a PDF version of the report.
Request:
curl -k --user admin:admin https://test.cfengine.com/api/user/milton/ subscription/query/file-changes-report -X PUT -d '{"to": "boss@megaco.com", "query": "SELECT FileName, Count(*) FROM FileChangesLog GROUP BY FileName", "schedule": "Monday.Hr23.Min59", "title": "A very important file changes report""description": "Text that will be included in email""outputTypes": [ "pdf" ] }'
Response:
204 No Content
Subscribed Query Example: Listing Report Subscriptions
Milton can list all his current subscriptions by issuing the following.
Request:
curl -k --user admin:admin https://test.cfengine.com/api/user/milton/subscription/query
Response:
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1351003514
},
"data": [
{
"id": "file-changes-report",
"to": "boss@megaco.com",
"query": "SELECT FileName, Count(*) FROM FileChangesLog GROUP BY FileName",
"title": "A very important file changes report",
"description": "Text that will be included in email",
"schedule": "Monday.Hr23.Min59",
"outputTypes": [
"pdf"
]
}
]
}
Subscribed Query Example: Removing A Report Subscription
Request:
curl -k --user admin:admin https://test.cfengine.com/api/user/milton/subscription/query/file-changes-report -X DELETE
Response:
204 No Content
Checking Status
You can get basic info about the API by issuing /api. This status information may also be useful if you contact support, as it gives some basic diagnostics.
Request
curl -k --user admin:admin https://test.cfengine.com/api/
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1351154889
},
"data": [
{
"apiName": "CFEngine Enterprise API",
"apiVersion": "v1",
"enterpriseVersion": "3.0.0a1.81c0d4c",
"coreVersion": "3.5.0a1.f3649b2",
"databaseHostname": "127.0.0.1",
"databasePort": 27017,
"authenticated": "internal",
"license": {
"expires": 1391036400,
"installTime": 1329578143,
"owner": "Stage Environment",
"granted": 20,
"licenseUsage": {
"lastMeasured": 1351122120,
"samples": 1905,
"minObservedLevel": 7,
"maxObservedLevel": 30,
"meanUsage": 21.9689,
"meanCumulativeUtilization": 109.8446,
"usedToday": 7
}
}
}
]
}
Managing Settings
Most of the settings configurable in the API relate to LDAP authentication of users. Settings support two operations, GET (view settings) and POST (update settings). When settings are updated, they are sanity checked individually and as a whole. All or no settings will be updated for a request.
Example: Viewing settings
Request
curl --user admin:admin http://test.cfengine.com/api/settings
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350992335
},
"data": [
{
"ldapPort": 389,
"ldapPortSSL": 636,
"hostIdentifier": "default.sys.fqhost",
"rbacEnabled": true,
"logLevel": "error",
"ldapEnabled": true,
"ldapUsername": "",
"ldapPassword": "",
"ldapEncryption": "ssl",
"ldapLoginAttribute": "uid",
"ldapHost": "ldap.example.com",
"ldapBaseDN": "ou=people,dc=example,dc=com",
"ldapFilter": "(objectClass=inetOrgPerson)",
"blueHostHorizon": 900,
"sketchActivationAlertTimeout": 60
}
]
}
Example: Configuring LDAP
The setting ldapEnabled
turns external authentication on or off. When turned
on, the API will check to see that the other LDAP related settings make sense,
and attempt to authenticate using the configured credentials. If it is not
successful in doing this, no settings will be changed. The API will notify you
with a return code an a message describing the error.
Request
curl --user admin:admin http://test.cfengine.com/api/settings -X POST -d
{
"ldapEnabled": true,
"ldapBaseDN": "DC=ldap,DC=example,DC=com",
"ldapEncryption": "ssl",
"ldapHost": "ldap.example.com",
"ldapLoginAttribute": "uid",
"ldapFilter": "(objectClass=inetOrgPerson)",
"ldapPassword": "password",
"ldapUsername": "test",
}
Response
204 No Content
Example: Changing The Log Level
The API uses standard Unix syslog to log a number of events. Additionally, log
events are sent to stderr
, which means they may also end up in your Apache
log. Log events are filtered based on the log level in settings. Suppose you
wanted to have greater visibility into the processing done at the backend. The
standard log level is error
. Changing it to info
is done as follows.
Request
curl --user admin:admin http://test.cfengine.com/api/settings -X POST -d
{
"logLevel": "info"
}
Response
204 No Content
Enterprise API interaction with Design Center
This is a simple walkthrough of a System::motd sketch activation against CFEngine Enterprise Hub 192.168.33.2. There are several steps. The most important step is where we define the sketch parameters. Those are exactly what the Mission Portal Design Center App generates through the GUI.
Sample API call to Install sketch
- Action:
PUT
- URL: http://192.168.33.2/api/dc/sketch/System::motd
- HTTP return code: 200
- RESULT:
{
"System::motd":
{
"params\/debian_squeeze.json":"sketches\/system\/motd\/params\/debian_squeeze.json",
"README.md":"sketches\/system\/motd\/README.md",
"params\/example.json":"sketches\/system\/motd\/params\/example.json",
"sketch.json":"sketches\/system\/motd\/sketch.json",
"params\/debian_wheezy.json":"sketches\/system\/motd\/params\/debian_wheezy.json",
"main.cf":"sketches\/system\/motd\/main.cf",
"params\/simple.json":"sketches\/system\/motd\/params\/simple.json"
},
"sketches":{"System::motd":1}
}
Sample API call to Define sketch parameters
- Action:
PUT
- URL: http://192.168.33.2/api/dc/definition/myfirstsketchdefinition
- HTTP return code: 200
- POST data:
{
'params': {'prepend_command': '/bin/uname -snrvm', 'motd_path': '/etc/motd', 'motd': 'hello dude2'},
'sketchName': 'System::motd'
}
- RESULT:
{"myfirstsketchdefinition":1}
Sample API call to Define environment
- Action:
PUT
- URL: http://192.168.33.2/api/dc/environment/mymotdenvironment
- HTTP return code: 200
- POST data:
{'environment': ['linux']}
- RESULT:
{"mymotdenvironment":1}
Sample API call to Activate sketch
- Action:
PUT
- URL: http://192.168.33.2/api/dc/activation/myfirstsketchActivation/System::motd
HTTP return code: 200
POST data:
{
'environmentName': 'mymotdenvironment',
'bundleName': 'entry',
'paramName': 'myfirstsketchdefinition',
'sketchName': 'System::motd'
}
- RESULT:
{
"System::motd":
{
"hash":"6cefdfedd0de3933c2ecac420b8d2aad",
"params":["myfirstsketchdefinition"],
"environment":"mymotdenvironment",
"identifier":"myfirstsketchActivation",
"target":"sketches",
"bundle":"entry",
"metadata":{"identifier":"myfirstsketchActivation","timestamp":1402657585}
}
}
Sample API call to Commit changes
- Action:
POST
- URL: http://192.168.33.2/api/dc/workspace/commit
- HTTP return code: 200
- POST data:
{'message': 'First-dc-api-commit', 'userEmail': 'test@test.com'}
- RESULT:
null
Managing Users and Roles
Users and Roles determine who has access to what data from the API. Roles are defined by regular expressions that determine which hosts the user can see, and what policy outcomes are restricted.
Example: Listing Users
Request
curl --user admin:admin http://test.cfengine.com/api/user
Response
{
"meta": {
"page": 1,
"count": 2,
"total": 2,
"timestamp": 1350994249
},
"data": [
{
"id": "calvin",
"external": true,
"roles": [
"Huguenots", "Marketing"
]
},
{
"id": "quinester",
"name": "Willard Van Orman Quine",
"email": "noreply@@aol.com",
"external": false,
"roles": [
"admin"
]
}
]
}
Example: Creating a New User
All users will be created for the internal user table. The API will never attempt to write to an external LDAP server.
Request
curl --user admin:admin http://test.cfengine.com/api/user/snookie -X PUT -d
{
"email": "snookie@mtv.com",
"roles": [
"HR"
]
}
Response
201 Created
}
Example: Updating an Existing User
Both internal and external users may be updated. When updating an external users, the API will essentially annotate metadata for the user, it will never write to LDAP. Consequently, passwords may only be updated for internal users. Users may only update their own records, as authenticated by their user credentials.
Request
curl --user admin:admin http://test.cfengine.com/api/user/calvin -X POST -d '{ "name": "Calvin" }'
Response
204 No Content
Example: Retrieving a User
It is possible to retrieve data on a single user instead of listing
everything. The following query is similar to issuing GET
/api/user?id=calvin
, with the exception that the previous query accepts
a regular expression for id
.
Request
curl --user admin:admin http://test.cfengine.com/api/user/calvin
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350994249
},
"data": [
{
"id": "calvin",
"name": "Calvin",
"external": true,
"roles": [
"Huguenots", "Marketing"
]
},
]
}
Example: Adding a User to a Role
Adding a user to a role is just an update operation on the user. The full role-set is updated, so if you are only appending a role, you may want to fetch the user data first, append the role and then update. The same approach is used to remove a user from a role.
Request
curl --user admin:admin http://test.cfengine.com/api/user/snookie -X POST -d
{
"roles": [
"HR", "gcc-contrib"
]
}
Response
204 No Content
}
Example: Deleting a User
Users can only be deleted from the internal users table.
Request
curl --user admin:admin http://test.cfengine.com/api/user/snookie -X DELETE
Response
204 No Content
Browsing Host Information
A resource /api/host is added as an alternative interface for browsing host
information. For full flexibility we recommend using SQL
reports via /api/query for this. however, currently vital signs (data
gathered from cf-monitord
) is not part of the SQL reports data model.
Example: Listing Hosts With A Given Context
Request
curl --user admin:admin http://test.cfengine.com/api/host?context-include=windows.*
Response
{
"meta": {
"page": 1,
"count": 2,
"total": 2,
"timestamp": 1350997528
},
"data": [
{
"id": "1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8",
"hostname": "windows2008-2.test.cfengine.com",
"ip": "172.20.100.43"
},
{
"id": "dddc95486d97e4308f164ddc1fdbbc133825f35254f9cfbd59393a671015ab99",
"hostname": "windows2003-2.test.cfengine.com",
"ip": "172.20.100.42"
}
]
}
Example: Looking Up Hosts By Hostname
Contexts, also known as classes, are powerful. You can use them to
categorize hosts according to a rich set of tags. For example, each
host is automatically tagged with a canonicalized version of its
hostname and IP-address. So we could lookup the host with hostname
windows2003-2.test.cfengine.com
as follows (lines split and indented
for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host?context-include=
windows2003_2_stage_cfengine_com
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350997528
},
"data": [
{
"id": "dddc95486d97e4308f164ddc1fdbbc133825f35254f9cfbd59393a671015ab99",
"hostname": "windows2003-2.test.cfengine.com",
"ip": "172.20.100.42"
}
]
}
Example: Looking Up Hosts By IP
Similarly we can lookup the host with hostname
windows2008-2.test.cfengine.com
by IP as follows (lines split and indented
for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host?
context-include=172_20_100_43
Response
{
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1350997528
},
"data": [
{
"id": "1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8",
"hostname": "windows2008-2.stage.cfengine.com",
"ip": "172.20.100.43"
}
]
}
Example: Removing Host Data
If a host has been decommissioned from a Hub, we can explicitly remove data associated with the host from the Hub, by issuing a DELETE request (lines split and indented for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=1c8fafe478e05eec60fe08d2934415c81a51d2075aac27c9936e19012d625cb8 -X DELETE
Response
204 No Content
Example: Listing Available Vital Signs For A Host
Each host record on the Hub has a set of vital signs collected by cf-monitord
on the agent. We can view the list of vitals signs from as host as follows
(lines split and indented for presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a/vital
Response
{
"meta": {
"page": 1,
"count": 4,
"total": 4,
"timestamp": 1351001799
},
"data": [
{
"id": "messages",
"description": "New log entries (messages)",
"units": "entries",
"timestamp": 1351001400
},
{
"id": "mem_swap",
"description": "Total swap size",
"units": "megabytes",
"timestamp": 1351001400
},
{
"id": "mem_freeswap",
"description": "Free swap size",
"units": "megabytes",
"timestamp": 1351001400
},
{
"id": "mem_free",
"description": "Free system memory",
"units": "megabytes",
"timestamp": 1351001400
},
}
Example: Retrieving Vital Sign Data
Each vital sign has a collected time series of values for up to one week. Here
we retrieve the time series for the mem_free
vital sign at host
4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a
for October
23rd 2012 12:20pm to 12:45pm GMT (lines split and indented for
presentability).
Request
curl --user admin:admin http://test.cfengine.com/api/host/\
SHA=4e913e2f5ccf0c572b9573a83c4a992798cee170f5ee3019d489a201bc98a1a/vital/\
mem_free?from=1350994800&to=1350996300
Response
"meta": {
"page": 1,
"count": 1,
"total": 1,
"timestamp": 1351002265
},
"data": [
{
"id": "mem_free",
"description": "Free system memory",
"units": "megabytes",
"timestamp": 1351001700,
"values": [
[
1350994800,
36.2969
],
[
1350995100,
36.2969
],
[
1350995400,
36.2969
],
[
1350995700,
36.2969
],
[
1350996000,
36.1758
],
[
1350996300,
36.2969
]
]
}
]
Tracking changes
Changes REST API allows to track the changes made by cf-agent in the infrastructure.
Example: Count changes
This examples shows how to count changes performed by cf-agent within last 24h hours.
Example is searching for changes that are performed by linux machines within generate_repairs bundle.
Request
curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy/count?nodegroup=linux&bundlename=generate_repairs'
Response
{
"count": 381
}
Example: Show vacuum command executions
Show all vacuumdb executions within last 24 hours executed on policy server.
Example is searching for changes that are performed by policy_server machines that execute commands promise with command /var/cfengine/bin/vacuumdb% - there is '%' sign at the end which is a wildcard as vacuumdb is executed with different options across policy.
Request
curl --user admin:admin 'https://test.cfengine.com/api/v2/changes/policy?nodegroup=policy_server&promisetype=commands&promiser=/var/cfengine/bin/vacuumdb%'
Response
{
"data": [
{
"bundlename": "cfe_internal_postgresql_vacuum",
"changetime": 1437642099,
"hostkey": "SHA=6ddfd5eaa85ee681ec12ce833fd7206e4d21c76e496be5f8b403ad0ead60a6ce",
"hostname": "hub.provisioned.1436361289.cfengine.com.com",
"logmessages": [
"Executing 'no timeout' ... '/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'",
"Completed execution of '/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'"
],
"policyfile": "/var/cfengine/inputs/lib/3.7/cfe_internal_hub.cf",
"promisees": [],
"promisehandle": "cfe_internal_postgresql_maintenance_commands_run_vacuumdb",
"promiser": "/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb",
"promisetype": "commands",
"stackpath": "/default/cfe_internal_management/methods/'CFEngine_Internals'/default/cfe_internal_enterprise_main/methods/'hub'/default/cfe_internal_postgresql_vacuum/commands/'/var/cfengine/bin/vacuumdb --analyze --quiet --dbname=cfdb'[0]"
}
],
"total": 1,
"next": null,
"previous": null
}