Today we are very proud and happy to launch our latest non-supported release, CFEngine 3.14.0. 3.14 is a great number, being the closest we will get to π, we also wanted to introduce something very special this time around, and we did!

New features

Let’s start with an overview of some new changes debuting in CFEngine 3.14.

Improved Role Based Access Control (RBAC)

In CFEngine 3.14 we have introduced a new backend for managing RBAC settings, as well as a whole new UI in the Mission Portal to manage this. This allows for more granular RBAC settings and makes it simple to set up roles with very limited and specific access. This new Mission Portal & API RBAC is based on existing roles. RBAC is a tricky topic, and we advise to create specific roles when users should have specific access. The permissions are purely additive, i.e. they give permission to access something. Every role has a set of permissions, and in the case where a user has more than one role, she has access to all the permissions of those roles.

Other new additions

In addition to the major news about introducing a Single Pane of Glass for all your reporting needs, there are also a number of small new features and bug fixes in 3.14. We have done a lot of work on the Mission Portal UI. We have for example made an improvement in Mission Portal, that makes it easier to see when a specific package was actually installed, and last modified, making software updating easier. For overall faster reporting, we have spent a significant amount of time improving the CFEngine hub main loop, to make it both more robust and faster. We have made several improvements to the Policy Analyser introduced in CFEngine 3.13, amongst others it will not also show the content of def.json and all other files that are a part of the distribution, not only .cf files. For those customers using the CFEngine High Availability mode, we have improved the setup documentation significantly, so this is now simpler than ever to use. Also, a bug that prevented files_single_copy from being defined from Augments has been fixed.

Introducing Federated Reporting

In this release, we launch the beta version of CFEngine Federated reporting. This is a new feature that we have worked on for a while, that make our reporting capabilities even better, and allows you to get a full 360˚ view of your entire infrastructure. Each Policy Hub will now be able, in addition to hosting its own Mission Portal, also to report all its data to a central Reporting Hub. This single hub runs a Mission Portal instance that will provide you with all the data from your entire infrastructure. Aggregated and presented just as you want it.

Why use multiple policy hubs?

CFEngine supports a large number of hosts per hub, usually in the realm of 5000 hosts per hub, but this will depend on many factors, and can be both higher or lower. There are many good reasons to set up an architecture with multiple hubs, and we will quickly go through some of them here.

Global Diversity

Servers might reside in many different locations. Due to issues like network latency, firewall configurations, etc. But also who is managing that datacenter, and issues like when servers are taken offline for maintenance, you want a local Policy Hub in the datacenter, to manage the infrastructure.

Multiple Security Zones

Your infrastructure might reside in multiple different security zones, where the end nodes - the hosts - might not all be able to connect to the same Hub. It is then a great choice to set up a separate Hub in each of these security zones to reduce the traffic between them.

Different network availability

We have customers that are running CFEngine in remote devices, that only have internet connection at a certain, sometimes even unpredictable, times. For this infrastructure, they often set up a separate Hub to deal with these types of devices. This includes CFEngine running in what we typically think of IoT devices, vehicles and so on. CFEngine is an excellent configuration management solution for such infrastructure.

Different Timescales

We have many CFEngine customers using CFEngine for infrastructures with inherently different timescales for different aspects of their infrastructure. Some parts of their infrastructure can work at an entirely different time scale than other parts, and you can save a lot of work by allocating for an appropriately scaled Hub.

Large Scale Infrastructure

CFEngine is the only automation solution that truly supports managing a large scale infrastructure. Other solutions can be great for a few hundred hosts under management, but CFEngine thrives with thousands of servers. However, at a certain scale, a single Hub might not be able to handle your entire infrastructure. We support 5000 hosts per Hub in a standard configuration, but through the help and assistance of our excellent support team, we can also go way beyond that. But when you manage many close to 100 000 servers or more, like several CFEngine users do, you need multiple Hubs. With Federated Reporting, however, you can connect them all together to have a single pane of glass to view reporting from your entire infrastructure.

How does it work?

Importing large scale data sets can be a challenge, and with very large scale infrastructure, accurate and detailed reporting and inventory data, this is a challenging, yet important, problem to solve. We have long been working with a client on a similar solution. While they were only able to do a few rounds of importing data to their central server every day, we have reduced the full round time with their data set to around 20 minutes. This means we can go from 5-6 updates of the data each day to a complete refresh twice an hour. The architecture we here introduce works in a 3 layered way. First, all of the hosts, hundreds or more often thousands or tens of thousands that are under management are all running the light-weight CFEngine agent. These hosts are each bootstrapped to one CFEngine hub. A system can often have several Hubs, and these Hubs do not traditionally talk to each other or share information. We now introduce the third layer, the Single Pane of Glass Reporting Hub, often referred to as a Superhub. This Hub collects and collates all the reporting data from each of the Hubs. This allows you to use a single interface to access the full reporting capabilities and inventory data from your entire infrastructure. Our main priority for the first release is that this new SPOG Hub will work with existing installations of CFEngine, without interfering with the existing setup. Connect the existing Hubs to the newly set up SPOG Hub. The collection mechanism will depend on the CFEngine version of the Hub in questions. With an existing CFEngine setup, we use a simple ETL (Extract, Transform, Load) process, copying data from the CFEngine hub and integrating it to the SPOG Hub at a given interval. When you also upgrade the other components in your infrastructure to CFEngine 3.14 or higher more opportunities will open. Newer versions of the CFEngine hub will support more advanced data synchronization mechanisms. This will allow for simplified setup, even faster synchronization of data and reduced bandwidth consumption. If there are other features or ideas you have, please share them with us, we are happy to work your priorities into our roadmap.

Hub Management

CFEngine 3.14 introduces a new Hub Management module. This allows you to set up a new Hub as the Single Pane of Glass Reporting Server - aka a Superhub, as well as enable a regular Hub to be a Feeder Hub, that will feed data to the Superhub. In the Hub Management module, you will also be able to tweak other settings such as the refresh rate of the data. Depending on how many Hubs, and how many hosts in total you want to show reports from, and how often you want to update the reporting data on the Single Pane of Glass reporting Hub, the load on this will vary, and you should make sure to use a machine that can meet the requirements of your infrastructure. We will get back with more specific HW requirements once it is launched as a supported feature of CFEngine 3.15.

The path to support

We are hoping for as much customer feedback as possible. This will be sifted through, made sense of, and implemented to the best of our ability in the coming months. Towards the end of the year, we will be ready to launch our next LTS series, CFEngine 3.15 LTS, that will feature a fully supported version of CFEngine Federated Reporting. In order to get as much feedback as possible, we encourage all our enterprise users to test this, and let us know how it goes. Even if you only collect data from a single Hub, your feedback is valuable.

Updated dependencies

We have updated some dependencies since CFEngine 3.13. Here is a list of the new requirements.

Try CFEngine 3.14 today

We have been working hard on CFEngine 3.14 so that it can work hard for you. Download CFEngine Enterprise or CFEngine Community today, and make your automation even more awesome!