Today we released our first monthly release! The January edition of CFEngine Enterprise and Community is now available to those who want to see what we are currently working on. The releases are called 3.1 alpha1 and 3.5 alpha1, respectively - and as the name suggests, this is work in progress. We have built packages for Enterprise and tested a small set of platforms: CentOS 5/6, RHEL 5/6, Ubuntu 10.04. Customers find the packages in a subdirectory “alpha_releases(unstable)” in their download area on cfengine.com.
If you are not a customer, but want to evaluate this release, please sign up here to download a free version of CFEngine Enterprise. Once registered, you will have access to the latest stable release (currently 3.0.0) as well as to this monthly snapshot release. These free releases can be used to manage up the 25 nodes.
Community users can clone our github repository - check out tag 3.5.0a1, we don’t branch off for these releases. There are also tar balls with sources. At this point, a big Thank You! to all of you who have contributed with patches to the Community code and to our policies and promise libraries in the last couple of months! Thanks for caring about CFEngine and for getting involved.
So, what has happened in January? Let’s start with…
Community
Those of you who followed the commit log on github will have noticed that a lot of structural changes have been made to the CFEngine core code. We have introduced a few implementations of basic data structures, separated the code into agent-specific directories, libpromises and a few utility libraries, and worked on separating the parser from the evaluation process. From a technical perspective this will make the individual components of CFEngine more testable on a unit-level, which allows us to write better code, fix bugs with more confidence, and add shiny new features later on. However, we also see a lot of goodness for our users from this:
The new data structures will allow us to get rid of numerous limitations regarding maximum sizes of input buffers and hash tables. Plenty of you have reported bugs and had come up with workaround to those limitations. The risk that through dynamic allocation cf-agent becomes a resource hog is mitigated by giving all these data structures a dynamically specified upper limit.
With a separated parser we open up for numerous integration possibilities. Syntax trees can be exported and imported from other formats, allowing significantly improved tooling support. Validation of policies can happen on the fly and without side-effects, which allows us to simplify the policy deployment process and improve the Design Center tools. Implementing new language features, such as support for structured data, becomes easier and less error prone as well.
A clearly defined parser process and clearer separation to the evaluator will also allow us to define a plugin interface for promise types in the future. Plugins that implement new promise types need to be able to validate the policy language, add their branches to the syntax tree, and then validate and modify (create, update, delete) the respective objects in the system. Building higher-level abstractions of promise types could become a very powerful concept both in terms of capabilities, and in terms of customization of CFEngine: on embedded or mobile devices we see very different configurables than on servers.
A new feature we have implemented in the Core is zeroconf-based discovery of policy hubs, using Avahi. This allows for a simplified bootstrapping process, if Avahi is present on your system. cf-serverd can generate an Avahi configuration file, and cf-agent will use libavahi to detect the policy hub. We detect and load libavahi at runtime (using dlopen), so no additional configuration flags or runtime dependencies are added. This is a first step, we plan to investigate fallback solutions to this that don’t have the limitations of Avahi at a later stage.
Enterprise
On Enterprise, we have focused on three major projects: Completing data access through SQL reports and Enterprise APIs, collecting diagnostic data about the policy hub, and investigating a modular architecture for the Mission Portal.
Via the SQL Reports app you have now access to promise logs, software reports, file diffs and compliance data. We have integrated PCRE support into SQL, so you can now use regular expressions in your SQL statements. The reports in the Hosts app have been rewritten to use the SQL backend, and will at a later stage probably disappear. In this particular release, the policy context (all promises, user promises, system promises) will not be functional for these reports, it will however come back at a later stage when we have implemented support for this in the SQL Report backend.
In addition to collecting data from the hosts, the Enterprise Hub now also collects diagnostic data from MongoDB into a history of data points, which allows us to identify trends in the database, and correlate changes in the system to what’s happening with the hub infrastructure. A next step is to collect similar data for cf-serverd on the hub so that we get information about throughput of data, responsiveness of the server, connection logs and stability, which we can again correlate with changes in the system to learn about the scalability of the existing infrastructure. This is not only useful information for those of you who operate Enterprise hubs, it also allows us to identify bottlenecks in the existing design, and to validate our models when we make future additions, such as building a reporting infrastructure to access aggregate data from multiple hubs.
Lastly, we have started to investigate a modularized architecture of the Mission Portal tool. This allows more flexible development of specialized applications, which can then provide specific knowledge about certain aspects of your system. These applications will be able to create a mesh of information through well-defined interfaces so that an app that knows how to interpret the git commit history of your policy files can enrich information provided by a policy browser or host explorer. Observations like “Host as89 is not compliant and has unusually high CPU load ever since user X made commit foo to promises.cf” become possible if apps that know about git, compliance and vital signs start to talk to each other. By making the architecture open to everybody, others could develop additional integrations.
These releases are snapshot releases of work in progress. Most things are not finished, some things might not work. They are not designed for production environments and are not covered by our service level agreements. However, we hope that we made some of you curious enough about the contents of the packages to give them a go in a test environment, and tell us what you think! The bug tracker is over here. As mentioned in my last blog post, we plan to have a production release twice a year - the next one before summer.
Volker Hilsheimer
I’m a software engineer turned team lead and project manager. I believe that there are universal patterns that we can observe in all systems, be it software, hardware or human, and that the principles of a shared purpose, autonomy of the individual and the desire to achieve mastery give us the energy to make progress.
Twitter: @vhilsheimer
