CFEngine 3.10.0 LTS released!
Posted by: Eystein Maloy Stenberg
We’re happy to announce that CFEngine 3.10.0 LTS now is released! One of the key advantages of CFEngine is its unparalleled performance, and this has received further attention in 3.10.0!
A big thanks to everyone testing the 3.10.0 beta release!
Being an LTS release, 3.10 will be supported until December 2019.
If you are using the previous LTS, 3.7, you will also benefit from all the new features, improvements and testing of all 3.8 and 3.9 releases, which you can read more about in the CFEngine 3.8.0 and 3.9.0 release posts.
Double query speed for Enterprise API
Enterprise hubs running at scale in the thousands of clients sometimes took quite a few seconds to respond to complex API queries.
One of the reasons for this taking a long time is the need for finding the number of results returned along with the actual results themselves. This number is used by the Mission Portal and API clients in order to split the result up into pages so that not the whole data set needs to be returned at the same time.
In previous versions of CFEngine Enterprise, this was done by first issuing a SQL COUNT query before the actual data query. However, utilizing some smart logic in PostrgreSQL, CFEngine 3.10 has reduced these two queries into one. This means that you can expect all queries, including Mission Portal reports, alerts and the Inventory reporting to return twice as fast as previous versions of CFEngine!
New variable expansion engine
A high-demand improvement included in 3.10.0 relates to speeding up variable expansion over large data structures. This means that working with large JSON-files or nested “classic” arrays is now much more efficient. We can let the numbers speak for themselves.
In this simple test, we are using a JSON file with about 2 KB of size (locations.json – placed in
/tmp), together with a policy that deeply iterates over it in order to define classes based on subnets (ipranges.cf).
To test the performance we wrap cf-agent in time:
/usr/bin/time -f "Elapsed (sec) %e\nMaxRSS (KB) %M" cf-agent -K ipranges.cf
The run with CFEngine 3.7.4 yields the following:
R: CFEngine Version 3.7.4
Elapsed (sec) 59.34
MaxRSS (KB) 32960
While CFEngine 3.10.0 gives:
R: CFEngine Version 3.10.0b1
Elapsed (sec) 0.08
MaxRSS (KB) 32224
The difference is almost hard to believe; in this case CFEngine 3.10 is 750 times faster than 3.7, while the consumed memory is about the same. What is more, the performance improvement in 3.10 will be even larger as the size of the data grows!
Please note that in order to make this vast improvement possible, it was necessary to replace the variable expansion engine in CFEngine. We have been very careful to ensure that policy is backward-compatible in 3.10, including running our some 1500 policy-based acceptance tests (which all pass), and Neil Watson tells us the EFL acceptance test (which all pass). The 3.10.0 beta did not expose any serious incompatibility issues, but other minor issues have been addressed in 3.10.0.
One notable change in order of operations is that an additional pass resolving vars was added in pre-evaluation. Now pre-evaluation does vars, classes, vars; previously it was only classes, vars.
Expanded Ubuntu and Solaris support
We are happy to announce that 3.10 have official support for the latest LTS releases from Canonical: Ubuntu 14.04 LTS and Ubuntu 16.04 LTS. This includes both host and Enterprise hub support on these platforms!
In addition, CFEngine Enterprise customers can expect official support for Solaris 10 x86 in 3.10. There is no package for Solaris 10 x86 yet, but we will make a package available as soon as it has passed internal testing. You can find the list of currently supported platforms here.
systemd unit improvements
The systemd support has received a major overhaul in CFEngine 3.10. All CFEngine services, like cf-serverd, cf-hub, cf-execd, now have their own independent systemd unit. This means that systemd will manage them independently, including monitor and restart them immediately if they stop for any reason. Please note currently systemctl stop cfengine3 will stop all other cfengine services. It returns immediately and systemd takes care of bringing down the other services.
3.10 also introduces the ability to bootstrap to a hostname and specify the port used by cf-agent to fetch policy (e.g.
cf-agent --bootstrap myhub.cfengine.com:5309). This information is written to
$(sys.workdir)/policy_server.dat. The new variable
sys.policy_hub_port will contain the port to use. Prior to 3.10 the IP address of the host you bootstrapped to was recorded and used as the source for
$(sys.policy_hub). Now the hostname itself is stored so that agents can seamlessly follow a hub when it’s IP changes.
This integrates seamlessly with any existing policy because the
sys.policy_hub variable is still an IP address, but now resolved via DNS at the start of the agent run instead of being resolved at bootstrap time.
You will also note that all the Known issues in 3.10.0 beta are fixed.
You can download the CFEngine community 3.10.0 packages and source code.
If you are an Enterprise customer, the Enterprise packages can be downloaded here.
We hope you enjoy 3.10.0, and we look forward to hearing about your experience in the CFEngine Mailing List!