Today we are happy to announce the general availability of CFEngine 3.12.0 LTS beta. The release of 3.12.0 beta took longer than expected. As a result we have decided to adjust our release schedule, and we releasing 3.12.0 beta now and later this year 3.12.0 will be the next official LTS release. If you are planning to contribute features or fixes to the next set of releases (we warmly welcome that !), we are always accepting those. The window for features is not closed for 3.12, but we appreciate help with fixing potential bugs in the beta, improving the performance if there are any identified issues, or generally implementations of suggestions moving forwards. If you want to start contributing but are unsure how? - Send documentation updates as pull requests to cfengine/documentation. - Search for issues labeled easy that are good candidates for new contributors to cfengine/core.

Beta Program

We appreciate all feedback we can get on this beta release. If you test it, you can provide any and all feedback here.

Notable changes in CFEngine 3.12.0 LTS beta

A detailed description of what has changed in 3.12.0 LTS beta can be found in the 3.12 changelogs.

Supported platforms

We have recently been conducting both a Platform Survey, as well as trying to analyse the download numbers and see what platforms are still being widely used and if some platforms are no longer as relevant to our users are they once were. We want to support as many platforms as possible, and try to keep all changes and new features compatible with all both supported and unsupported platforms. But maintaining support involves both a long term commitment as well as investments in HW, SW and time. For 3.12 LTS series we therefor suggest to drop the support for the following platforms.

• Solaris 9
• AIX 5.3

We will add support for Ubuntu 18.04 in CFEngine 3.12 LTS series. We would want to hear directly from you if you have a different perspective on this. Are you using any of the platforms we are planning to drop support for, or if you want us to add support for another platform, now is the time to tell us.

Multiple augments files can now be used

NOTE: Discussion about current implementation in progress in CFE-2741. Please consider contributing to the discussion. You merge more specific augments on top those set first from def.json. For example create /tmp/example.cf with the following content.

bundle agent main
{
  reports:
    "def.my_var == $(def.my_var)";
    "def.my_other_var == $(def.my_other_var)";
    "def.centos_6_var == $(def.centos_6_var)";
}

Create /tmp/def.json with the following content. Note the augments keys position currently affects how this functions. See CFE-2741 for details.

{
  "vars":{
    "my_var": "defined in def.json",
    "my_other_var": "Defined ONLY in def.json"
  },
  "augments": [
    "/tmp/$(sys.flavor).json"
  ]
}

Create an additional augments file to merge on top. /tmp/$(sys.flavor).json where $(sys.flavor) is the platform you are running. For examle create /tmp/centos_6.json with the following content.

{
  "vars": {
    "my_var": "Overridden in centos_6.json",
    "centos_6_var": "Defined ONLY in centos_6.json"
  }
}

Running the policy will produce output that looks like this:

[root@hub tmp]# cf-agent -KIf ./example.cf
R: def.my_var == Overridden in centos_6.json
R: def.my_other_var == Defined ONLY in def.json
R: def.centos_6_var == Defined ONLY in centos_6.json

New template method for inline mustache

You are no longer required to use an external file template. Now you can provide the template directly within the policy. For example:

bundle agent main
{
  vars:
    "d" data => '{ "hello": "world", "feature": [ "render", "inline", "mustache" ] }';

  files:

    "/tmp/example.txt"
      create => "true",
      template_method => "inline_mustache",
      edit_template_string => "{{%-top-}}
",
      template_data => @(d);

}

Results in /tmp/example.txt having this content.

{
  "feature": [
    "render",
    "inline",
    "mustache"
  ],
  "hello": "world"
}

Improved UI responsiveness at large scale

In a response to customers of large scale environments, Mission Portal and APIs now support up to 50,000 hosts in a single hub, and the most common queries will respond within 10 seconds at this scale! Note these improvements are intended to be used with a dedicated “query hub” where multiple hubs data sets have been coalesced.

New Inventory API

Mission Portals Inventory reporting interface is popular for reporting standard inventory like OS distribution across hosts. However often these data needs to be integrated with external systems. 3.12.0 LTS beta brings a new REST API to better support extracting and using this information.

curl --user admin -X POST \
    -H 'content-type: application/json' \
    https://hub/api/inventory -d '{ "select":[ "Host name", "OS type"]}'

{
    "data": [
        {
            "header": [
                {
                    "columnName": "Host name",
                    "columnType": "STRING"
                },
                {
                    "columnName": "OS type",
                    "columnType": "STRING"
                }
            ],
            "queryTimeMs": 11,
            "rowCount": 2,
            "rows": [
                [
                    "host001",
                    "linux"
                ],
                [
                    "hub",
                    "linux"
                ]
            ]
        }
    ],
    "meta": {
        "count": 1,
        "page": 1,
        "timestamp": 1515607751,
        "total": 1
    }
}

New LDAP settings API

CFEngine Enterprise 3.12.0 LTS beta also introduces a new LDAP settings implementation with accompanying API supporting new highly requested features, including custom LDAP ports and default roles. By using a default role you can create a role that new users of Mission Portal (and APIs) will get, both if they are created in the local database or come from the LDAP integration. This enables you to decide which hosts all new users will see upfront, before they log in to their account for the first time.

New email settings

CFEngine Enterprise 3.12.0 LTS beta brings new email settings that can configured in Mission Portal. Authenticated SMTP support is added, and exported reports (CSV/PDF) can now be attached to outbound emails if they do not exceed the configured maximum attachment size (default max size is 10MB).

New Host count dashboard widget

The new Host count widget allows you to visualize the number of agents reporting over a specifed range of time. Bootstrap and decomission events can now be found in the Event log.

Improved Health Diagnostics in Mission Portal

When there are reports with an unclear origin, this has caused some confusion in the past. We have continuously improved how reposting is done, and among other things we have had a Health indicator already, however we have now extended this, and clarified its usage and meaning. The current two report categories that are available, have been renamed as follows:

• “Hosts not reporting” have been renamed “Hosts not recently collected from” and
• “Agents not running” have been renamed “Agents not recently run”

The categories are in priority order and non-overlapping, so any host can maximum be in one of these two categories. For each category there is a report of hosts in it, together with diagnostics information about what exactly it means and probable root causes with remediation recommendations. This should make reporting more reliable, and issues that arise should be simpler to fix. We have also added a new health category for host that are misconfigured so that they use same key and appear to be the same host. This has created some unreliable reporting in the past, that we now mitigate.

New Host Exclusion List

Many of our customers are using reporting extensively, and have many different reports In mission portal it is now straight forward to create a report for a set of hosts, but add a list of hosts to be excluded from the search criteria directly from the Mission Portal UI. This makes it significantly simpler to create and maintain such exclusion lists on top of dynamic host groups.

Removal of Redis database

In 3.12.0 LTS beta we have removed the Redis database structure from our reporting pipeline, to make sure that we have a more robust, unified database structure. This is now only using Postgres. This ensures that the consistency of reports is higher, that all the reports have correct time stamps and arrive in the correct order. This is a significant improvement to our reporting, and will benefit all the customers of CFEngine that uses reporting. This allows us to improve the performance of reporting, and is a key factor in enabling a single pane of glass from all systems in a large infrastructure. This change will allow many changes to be done faster in the lifetime of 3.12 and the subsequent non-LTS releases.

Dependencies

As can be seen from the lists below, we have spent a lot of time on updating the dependencies. The largest changes are that we have moved from PHP 5 to PHP 7, OpenSSL 1.0 to OpenSSL 1.1, we have removed Redis and have moved from Postgres 9 to Postgres 10. We therefore encourage every user of CFEngine to test this beta release as much as possible to make sure we find all potential issues before the final release before the summer.

Core Agent Dependencies

Enterprise Hub Dependencies

Get it!

CFEngine Enterprise packages can be downloaded here or you can take a quick spin with the CFEngine Enterprise 3.12 Vagrant environment. Community Edition is released as source code, packages and Linux package repositories – to make installation as easy as possible! We hope you enjoy the new releases, and we look forward to hearing about your experience in the CFEngine Google Group!