In the latest version of the CFEngine network protocol (filestream - v4), we leveraged librsync for efficient file copying using their Streaming API.
While implementing the file streaming in CFEngine, I found that the documentation on the Streaming API was a bit unclear. Thus, I created two example programs to experiment with how it works. I thought I’d share them in this blog post as a tutorial to help other developers get up to speed faster.
Over three years ago we introduced git and systemd custom propmise types.
While these are quite functional I recently needed to manage a git repository that was private and so needed a way to authenticate to a git server, in this case gitlab.
Gitlab has project access tokens but we didn’t directly support any sort of authentication attributes on the promise type.
I ran across git credential as a way to provide authentication and thought it was a good fit.
I promised more Build modules in my previous monthly Monday module blog post: package-method-winget.
And here they are: windows-capability and windows-optional-feature.
Inventory Both of these modules use similar usage details to control whether to inventory and to promise the state for specific capabilities and optional features.
By default inventory is taken in the form of a classic array which ends up in the Mission Portal as a comma separated list:
OpenSSH.Server~~~~0.0.1.0:Installed, OpenSSH.Client~~~~0.0.1.0:Installed, etc. To disable this inventory, define the class disable_windows_capability_inventory or disable_windows_optional_feature_inventory in the data namespace. The data namespace is the default if you use Host specific data or Group data. If you want to set these in augments you will need to specify the namespace explicitly like this:
As a developer and user of CFEngine I want to use policy to manage the software on my systems so that I can switch operating systems, distributions, computers and have all my normal tools available wherever I go.
Towards this end I searched for a Windows package manager and found one in winget. I showed a prototype in Agent Is In - Episode 37 - Windows package management as well as refined the whole process in Agent Is In - Episode 40 - Windows module workshop.
Editing and copying large files or large numbers of files is slow. For a configuration management tool, it is probably one of the slowest things we do, apart from waiting for other programs to finish or waiting for network communication. In this blog post, we look at how to copy files. More specifically, the most performant approaches available on modern Linux systems. We are working on implementing these techniques so CFEngine and all your policy will copy files more efficiently.
To manage large infrastructures, efficient solutions for both making changes and observing the current state are necessary. As most information (inventory) about hosts is quite predictable and static, there are many opportunities for optimizations in terms of compression and avoiding re-transmission of the same data. In the CFEngine team, we are improving our reporting systems with a focus on correctness and low bandwidth consumption. This will benefit many users, both large data centers where bandwidth (networking equipment) is costly, as well as small IoT devices with limited connectivity. Inspired by git, we are implementing commits of reporting data, with table-based diffs, and compression of multiple changes, akin to squashing git commits.
For CFEngine we manage several public and private repositories of code in GitHub for our Open Source and Enterprise products. In order to ensure quality we run many checks on the code both with nightly builds as well as on each pull request. We use a Jenkins server for nightlies which also includes more extensive deployment tests on all of the platforms we support. Previously we had used Travis for many of these checks but that system started to show its age and limitations.
The license of our in-house C utility and compatibility library libntech was recently changed from GPLv3 to Apache License Version 2.0 which makes the library suitable for more projects thanks to the more permissive license. While GPLv3 practically required any project using libntech to be licensed under GPLv3 as well, the Apache License v2.0 allows any open source as well as proprietary software to utilize our utility library, keeping the copyright attributions.
Opening and reading files may cause your program to block indefinitely. In this blogpost we'll discuss how to work around this issue.
This guide is designed for the novice user of CFEngine who wishes to explore the power of Emacs while developing CFEngine policy files – and will introduce the use of some Emacs features and plugins along the way.
There are two types of editors available in the Unix and Linux world: line and visual. Examples of line editors are ed and sed. These allow you to edit a file one line at a time.
