This post was syndicated with permission from the original source.
CFEngine 3.12.0 introduced the augments key to the Augments file format. If you are not already familiar with Augments, check it out. It’s a very easy way to define classes and variables very early during agent execution, before policy.
The new augments key allows you to merge additional data in the augments format on top of the base augments. I However, there is, I think, still a simple way to accomplish this. This can provide a flexible way of providing different data to different sets of machines.
This post was syndicated with permission from the original source.
How do you deal with config files that need different settings based on various services that are running on a host and cooperate with other teams? It’s a common question, and it came up on in #cfengine on libera.chat recently.
The issue is that team A might be working on package A, which requires some environment variables set. But team B might be working on a totally different thing – and want to achieve the same thing. I hoped to give them a bit of ’library’ code to take care of it, rather than have them touch a centralized environment-setting policy file.
We’re happy to announce that CFEngine 3.7.6 is released! With 3.7 being a stable LTS branch, 3.7.6 brings bug fixes and stability enhancements to the CFEngine customers and community. Looking at the CFEngine release schedule, we can see:
3.7 LTS is maintained (and supported for enterprise customers) until July 17th 2018. 3.10 LTS is maintained (and supported for enterprise customers) until December 27th 2019. If you are planning to contribute feature to the next feature release (thank you!), please note that we wold need the pull requests ready for merging by the end of September for incorporation into 3.12. If you are planning to contribute fixes to 3.10 or 3.7 LTS please note that we would need the pull requests ready for merging by the end of September for incorporation into 3.7.7 and 3.10.3. RPM packages now respect the chkconfig specified state when stopping a service. Now if the cfengine3 service is off for runlevel 2 the CFEngine services are stopped when you switch to runlevel 2. cf-monitord now correctly detects the usernames for processes on AIX. Classification when running under the Xen Hypervisor was also fixed. Masterfiles got fixes to the apt_get package module so that it works correctly when more than one source repository contains the package. Masterfiles also saw the addition of oslevel (on AIX), mailx (on Linux, Darwin, OpenBSD, NetBSD, and FreeBSD) to the paths bundle. The prunetree bundle was added to the standard library making it easier to recursively delete files and directories up to a specified depth older than a specified number of days. Enterprise gets bug fixes related to exporting reports and sharing host categorization views and reports. Additionally when hostnames are displayed in reports they now link to the individual host info page and usernames are now allowed to contain dots (.). Masterfiles now ensures the postrgres log file is rotated. The verbosity of some maintenance policy was increased and the policy to clear a build up of unreported data now includes previous_state and untracked reports. Some Enterprise dependencies were updated:
With more and more services using SSL keeping track of the certificates in use across a global infrastructure can be challenging. The inventory reporting features in CFEngine Enterprise can be useful in identifying services using SSL as well as when their certificates will expire. cf-monitord provides lists of ports that are listening. We can use openssl to connect to each listening port and if successful we can extract the certificate information for inventory. We won’t be able to find ALL certificates like this. This policy only covers up-front SSL/TLS. From Serverfault:
We’re happy to announce that CFEngine 3.11.0 Beta (non-LTS) is now ready. Thanks to everyone for all of the contributions! Please test extensively and submit bug reports. 3.11.x introduces some new features and deprecates some underutilized functionality. Please note that 3.11.0 will be a non-LTS release, which means that it will be maintained only for 6 months from the release date and not supported for CFEngine Enterprise customers (but Enterprise packages will be available). Looking at the CFEngine release schedule, we can see:
We’re happy to announce maintenance releases for both supported CFEngine release branches today! Being maintenance (aka patch) releases, the goal is to increase stability and reliability for CFEngine users and enable a safe upgrade-path. As such, the releases primarily include bugfixes and low-risk changes that do not impact the compatibility between previous patch releases. Looking at the CFEngine release schedule, we can see that
3.7 LTS is maintained (and supported for Enterprise customers) until July 17th 2018 3.9 non-LTS is no longer maintained 3.10 LTS is maintained (and supported for Enterprise customers) until December 27th 2019 If you are planning to contribute features to the next feature release (thank you!), please note that we would need the pull requests ready for merging by early-April in order to have time to incorporate them into 3.11. If you are planning to contribute fixes to 3.10 LTS please note that we would need the pull requests ready for merging by early-May in order to have time to incorporate them into 3.10.2 LTS.
Last week we attended Config Management Camp in Gent Belgium.
How did automation start? In the beginning Mark split light from dark…
There were 7 talks in the CFEngine devroom. Christian Linden kicked things off with his presentation “Get set for getting work done by CFEngine”. He covered the basics of what you need to know when getting started with CFEngine as well as some tips to help in debugging policies. After Christians Kickoff I talked about “Testing CFEngine Policy”. Why you should test, currently available testing frameworks, how to get started writing tests that produce TAP and or JUnit output and finally how to expand CFEngines test coverage while contributing to examples in the documentation. Martin Simons wrapped up the first day with his talk “CFEngine Hero of IoT”. Martin brought a Raspberry Pi running Raspbian told us how he is using it to implement a motion-activated security system built with commodity hardware at home and then demonstrated how CFEngine can easily manage it and deployed a tomcat application. The main topic of conversation on Tuesday was data. How to separate it from policy, how to make use of external data, and how it can make your life easier in SURFsara. Bas van der Vlies started us off with his talk “How to use the augments file (def.json)”. He showed how he defined a host specific data file that let him easily override default values for a specific service/bundle. He discussed how this has had great benefits for non CFEngine users in his environment, allowing them to easily identify which service/bundles are activated for a host and exceptions that each host is overriding. Next Jurica Borozan talked about “Merging Technologies, ideas on using CFEngine with cloud and container technologies”. He discussed how his tag based classification system enabled host specific policies to be delivered and the varying ways that this common model could be applied to physical systems, docker containers, and cloud hosts leveraging the features available in each technology stack. He finished up by demonstrating his policy that managed maintaining a specific number of AWS instances for a given image and how a simple change in his data source (a json file) resulted in nodes being provisioned or destroyed within 5 minutes. Neil Watson gave a short remote talk “CFEngine Simplified with EFL”. He discussed the design philosophy of EFL and how you can leverage the framework to get things accomplished with CFEngine without having to know any policy. I wrapped up talks with a short presentation on “The Masterfiles policy Framework: A short history and future direction”. I presented the lineage of the current policy framework, how we got to where we are today and how data (the augments file def.json) is very much at the core of how we expect to improve usability and ease future policy framework upgrades without having to edit policy. Special thanks to Christian, Martin, Bas, and Neil for sharing your thoughts and presentations. We look forward to seeing everyone again next year!
The notes from the Q4 2016 meeting can be found in the github repository.
This meeting marks 2 years for the Community Advisory Board. As a reminder the current board members include:
Bas van der Vlies Jonathan Clarke Mike Svoboda Ted Zlatanov Neil Watson Eystein Stenberg Nick Anderson The general consensus was that the meetings have been valuable and we should continue the efforts. We have decided to pilot having the next advisory board meeting in #cfengine instead of #cfengine-cab in order to increase community awareness and get additional input from other users with different perspectives.
This interview was conducted by Aleksey Tsalolikhin / 3 Aug 2016 Q: What can you tell our readers about yourself? How did you start using CFEngine? A: I am a Spanish sysadmin living and working in The Netherlands (Dutch spouse). All round (multidisciplinary) sysadmin, with a focus on automation, bootstrapping infrastructures, high availability, disaster recovery, storage. I started using CFEngine about 8/9 years ago (or was it 10?). Back in the day I was the only admin doing Linux at our company, and the Linux infrastructure was growing every week and it was getting impossible to manage everything by hand. So CFEngine to the rescue. I found the tutorials on the debian-administration.org site, looking at the publication dates, it must have been 10 years ago, time flies) and it seemed quite doable. Later I came across the Campi/Bauer book: Automating Linux and Unix System Administration and re-implemented quite a bit of stuff with their tips, not only CFEngine but also FAI/Kickstart. Q: Could you give us an example of something you’ve automated with CFEngine that stands out as having made your life easier, helps you to sleep better at night, or has really made a difference for the business? A: Stuff I’ve used CFEngine for: - disabling/enabling services; - configuration mail routing; - seLinux; - iptables; - yum/apt repositories; - installation software; - configuration syslogd/journald/logstash forwarders for ELK; - distribution of sysadmin scripts/configuration files/motd’s; - distribution known_hosts files for ssh clients; - distribution and configuration of monitoring software (nagios: nrpe/snmpd); - configuration ntpd/chronyd; Anything that needs doing basically. We have a number of host types (apache2, tftpd, mysqld, java application servers, etc). Our Linux infra is coupled to a freeipa environment with the automember plugin, so if the host is called tftp\* (where \* is ‘whatever’ ) then that host will be member of an LDAP hostgroup, which in FreeIPA gets populated as a netgroup as well. We can use those netgroups to automatically define classes in CFEngine and those classes are coupled to actions (install apache2, configure it according to x, y, z, etc.). My colleagues install the OS using PXE and the only thing they need to do once the installation is done is give the host a name (part of joining the FreeIPA domain). The rest gets done automatically. So this is about it in a nutshell. Without CFEngine this company would not have the Linux infrastructure it now has and I would not have had time to accomplish many other projects (I became much more efficient, so I got assigned to many more projects). Q: You said you have about 150 Linux nodes in your network. How many are still on cf2 and how many are on cf3? A: cf2: 70%, cf3 the rest (growing). Q: When we corresponded earlier, you said you “left the old cf2 environment alone (it basically never dies and does not need new modifications)”. What led you to start using cf3? A: Lack of support for cf2. We use CentOS and with CentOS 7 the cf2 binaries were a bit long in the tooth. Q: What’s the biggest advantage of cf3 over cf2 for you? A: cf3 is more consistent in its syntax. Plus the standard library has a lot of scaffolding already set up for us (services, for instance). The packages method is much nicer as well. Q: What are you most proud of about your setup? A: Difficult to say, in the end this is just a tool. It’s nice to see things work the way they are supposed to, but that is our work. Q: What do you enjoy most about the CFEngine community? A: Very fast (good) answers and advice for alternative solutions to the issues one posts to the mailing list. Nice, polite, civil, discussions. All communities should be like this. Q: Any advice for the CFEngine 2 users out there? a: Even though cf2 is an excellent tool, the fact that it is no longer supported means that you will be on your own if something happens to your setup. This could be an issue. if you have a very stable and static environment where cf2 just works, leave that alone and set a new cf3 environment next to it. Sooner or later you will get a system where cf2 will not run without much effort, and at that time you will have a problem. Q: Anything else you’d like to share with the CFEngine community? A: I learnt a lot using your tutorial and Brian Bennett’s (@bahamat) posts, specially the cf-primer: from zero to hero. Those are the hands on tutorials that the community needs. CFEngine has a reputation for being hard, but it’s not really true. Aleksey Tsalolikhin is a CFEngine consultant and trainer at Vertical Sysadmin.
A few months ago I posted a link on the help list to the CFEngine layer for spacemacs. Since then I have learned there are a few other org-mode users so I wanted to share how I got cfengine3 src blocks execution working. I added the following to my dotspacemacs/user-init.
(defcustom org-babel-cfengine3-command "/var/cfengine/bin/cf-agent" "Name of command to use for executing CFEngine policy.") (defvar org-babel-cfengine3-command-options "--no-lock" "Option string that should be passed to the agent. Note that --file will be appended to the options.") (defvar org-babel-cfengine3-file-control-stdlib "body file control{ inputs => { '$(sys.libdir)/stdlib.cf' };}\n" "File control body to include the standard library from $(sys.libdir). It is useful to inject into an example source block before execution so that bundles and bodies from the standard library are automatically available.") (defun org-babel-execute:cfengine3 (body params) "Actuate a block of CFEngine 3 policy. This function is called by `org-babel-execute-src-block'. A temporary file is constructed containing `org-babel-cfengine3-file-control-stdlib and the body of the src block. `org-babel-cfengine3-command' is used to execute the temporary file." (let* ((temporary-file-directory ".") (tempfile (make-temp-file "cfengine3-"))) (with-temp-file tempfile ;; TODO Consider making automatic stdlib inclusion optional (insert org-babel-cfengine3-file-control-stdlib) (insert body)) (unwind-protect (shell-command-to-string (concat org-babel-cfengine3-command " " ;; TODO Consider adding a header option to specify bundlesequence org-babel-cfengine3-command-options " " (format " --file %s" tempfile))) (delete-file tempfile)))) Now any time I have a cfengine3 SRC block in org-mode I can simply run org-babel-execute-src-block ( CTRL-c CTRL-c in emacs/spacemacs documentation this is written as C-c C-c ) to have the block written to a temporary file, executed and the temporary file deleted. For example with my insertion placed inside the following SRC block.
