About a year ago, I wrote a small python script to automate installing and bootstrapping CFEngine on virtual machines in AWS. It had some hard coded IP addresses that I needed to update when I spawned new hosts, but other than that, it worked well. During manual testing, it saved me a lot of time instead of having to do things manually. Deploying CFEngine normally consists of these steps:
- Determine what CFEngine package to use.
- Download appropriate package if you haven’t already -
curl. - Copy the package to the host -
scp. - Log into the host -
ssh. - Install the package -
rpm/dpkg. - Bootstrap CFEngine -
cf-agent -B.
At a company hackathon I decided to make my script into something
better, something that would be useful to my colleagues, and maybe even
CFEngine users in general. Enter cf-remote.
Info
cf-remote can be used to show information about a system before
installing CFEngine. The info command logs into the system, runs a few
commands and parses
/etc/os-release
to present relevant information:
$ cf-remote info -H 34.252.28.73
ec2-user@34.252.28.73
OS : rhel (fedora)
Architecture : x86_64
CFEngine : Not installed
Policy server : None
Binaries : rpm, yum
The cf-remote command line tool is written in Python, and uses
Fabric to log into the system via SSH. (Add
--log-level debug to see all commands cf-remote runs). Note that
there are almost no dependencies on the target system. You only need ssh
access and a shell which can perform basic UNIX commands like ls,
cat, which. The printout shows that it’s a Red Hat machine, and
CFEngine is not installed yet.
Install and bootstrap
cf-remote can install CFEngine on the system above. The only thing you
really need to specify is IP address(es):
$ cf-remote install --hub 34.252.28.73 --bootstrap 172.31.30.237
Here is the output from the example above:
$ cf-remote install --hub 34.252.28.73 --bootstrap 172.31.30.237
ec2-user@34.252.28.73
OS : rhel (fedora)
Architecture : x86_64
CFEngine : Not installed
Policy server : None
Binaries : rpm, yum
Package already downloaded: '/Users/olehermanse/.cfengine/cf-remote/packages/cfengine-nova-hub-3.12.1-1.x86_64.rpm'
Copying: '/Users/olehermanse/.cfengine/cf-remote/packages/cfengine-nova-hub-3.12.1-1.x86_64.rpm' to '34.252.28.73'
Installing: 'cfengine-nova-hub-3.12.1-1.x86_64.rpm' on '34.252.28.73'
CFEngine 3.12.1 was successfully installed on '34.252.28.73'
Bootstrapping: '34.252.28.73' -> '172.31.30.237'
Bootstrap successful: '34.252.28.73' -> '172.31.30.237'
We can guess the username and CFEngine version if not specified. Many hosts can be specified in a single command:
$ cf-remote install --hub 34.252.28.73 --bootstrap 172.31.30.237 --clients 52.212.51.201,52.212.51.202,52.212.51.203
Or using a newline delimited file:
$ cat ./clients
52.212.51.201
52.212.51.202
52.212.51.203
$ cf-remote install --hub 34.252.28.73 --bootstrap 172.31.30.237 --clients ./clients
Other useful commands
The tooling includes a few other useful utilities. Some of these things
can be done in CFEngine policy, but cf-remote doesn’t assume that
CFEngine is installed. Thus, these commands can be useful both before
and after installing CFEngine.
File copy
The scp command can be used to transfer files to the host.
$ echo "Hello, world" > txt
$ cf-remote -H 34.252.28.73 scp txt
Copying: 'txt' to '34.252.28.73'
The destination defaults to the home folder of the SSH user
(/home/ec2-user/txt in this example). The command doesn’t actually use
an scp executable, but Fabric’s Connection.put().
Arbitrary command execution
We can also run commands as if we were logged in via ssh:
$ cf-remote -H 34.252.28.73 run "cat txt"
34.252.28.73: 'cat txt' -> 'Hello, world'
This is especially useful to run commands on many hosts. -H accepts a
comma separated list or a path to a file (beginning with ./, /,
../ or ~/).
Videos
cf-remote was introduced, with some live demos, in my talk at
CfgMgmtCamp 2019. The video recording is
available on YouTube (cf-remote part starts at
20:51):
We also made some videos
showing how to install and use cf-remote: Install
cf-remote Deploying CFEngine using
cf-remote
Getting started
The tool is completely open source, and available on GitHub:
https://github.com/cfengine/cf-remote
The README has installation instructions. It should work on any system with
curl, Python(3), Fabric, and
Requests. We’ve tested it
on Linux(Fedora) and OS X. Currently, RHEL and Ubuntu targets are
supported. Debian, Fedora, and CentOS should also work as they use
dpkg / rpm. The target must be running an SSH server with your key
in authorized_keys, so Fabric can log in.
Contributing
Anyone can contribute to cf-remote, we have a curated list of tickets
to pick up here: https://northerntech.atlassian.net/issues/?filter=10068
Additional platform support (beyond dpkg and rpm) and parallellized
installation are good next features.
