This interview was conducted by Aleksey Tsalolikhin / 3 Aug 2016 Q: What can you tell our readers about yourself? How did you start using CFEngine? A: I am a Spanish sysadmin living and working in The Netherlands (Dutch spouse). All round (multidisciplinary) sysadmin, with a focus on automation, bootstrapping infrastructures, high availability, disaster recovery, storage. I started using CFEngine about 8/9 years ago (or was it 10?). Back in the day I was the only admin doing Linux at our company, and the Linux infrastructure was growing every week and it was getting impossible to manage everything by hand. So CFEngine to the rescue. I found the tutorials on the debian-administration.org site, looking at the publication dates, it must have been 10 years ago, time flies) and it seemed quite doable. Later I came across the Campi/Bauer book: Automating Linux and Unix System Administration and re-implemented quite a bit of stuff with their tips, not only CFEngine but also FAI/Kickstart. Q: Could you give us an example of something you’ve automated with CFEngine that stands out as having made your life easier, helps you to sleep better at night, or has really made a difference for the business? A: Stuff I’ve used CFEngine for: - disabling/enabling services; - configuration mail routing; - seLinux; - iptables; - yum/apt repositories; - installation software; - configuration syslogd/journald/logstash forwarders for ELK; - distribution of sysadmin scripts/configuration files/motd’s; - distribution known_hosts files for ssh clients; - distribution and configuration of monitoring software (nagios: nrpe/snmpd); - configuration ntpd/chronyd; Anything that needs doing basically. We have a number of host types (apache2, tftpd, mysqld, java application servers, etc). Our Linux infra is coupled to a freeipa environment with the automember plugin, so if the host is called tftp\* (where \* is ‘whatever’ ) then that host will be member of an LDAP hostgroup, which in FreeIPA gets populated as a netgroup as well. We can use those netgroups to automatically define classes in CFEngine and those classes are coupled to actions (install apache2, configure it according to x, y, z, etc.). My colleagues install the OS using PXE and the only thing they need to do once the installation is done is give the host a name (part of joining the FreeIPA domain). The rest gets done automatically. So this is about it in a nutshell. Without CFEngine this company would not have the Linux infrastructure it now has and I would not have had time to accomplish many other projects (I became much more efficient, so I got assigned to many more projects). Q: You said you have about 150 Linux nodes in your network. How many are still on cf2 and how many are on cf3? A: cf2: 70%, cf3 the rest (growing). Q: When we corresponded earlier, you said you “left the old cf2 environment alone (it basically never dies and does not need new modifications)”. What led you to start using cf3? A: Lack of support for cf2. We use CentOS and with CentOS 7 the cf2 binaries were a bit long in the tooth. Q: What’s the biggest advantage of cf3 over cf2 for you? A: cf3 is more consistent in its syntax. Plus the standard library has a lot of scaffolding already set up for us (services, for instance). The packages method is much nicer as well. Q: What are you most proud of about your setup? A: Difficult to say, in the end this is just a tool. It’s nice to see things work the way they are supposed to, but that is our work. Q: What do you enjoy most about the CFEngine community? A: Very fast (good) answers and advice for alternative solutions to the issues one posts to the mailing list. Nice, polite, civil, discussions. All communities should be like this. Q: Any advice for the CFEngine 2 users out there? a: Even though cf2 is an excellent tool, the fact that it is no longer supported means that you will be on your own if something happens to your setup. This could be an issue. if you have a very stable and static environment where cf2 just works, leave that alone and set a new cf3 environment next to it. Sooner or later you will get a system where cf2 will not run without much effort, and at that time you will have a problem. Q: Anything else you’d like to share with the CFEngine community? A: I learnt a lot using your tutorial and Brian Bennett’s (@bahamat) posts, specially the cf-primer: from zero to hero. Those are the hands on tutorials that the community needs. CFEngine has a reputation for being hard, but it’s not really true. Aleksey Tsalolikhin is a CFEngine consultant and trainer at Vertical Sysadmin.
A few months ago I posted a link on the help list to the CFEngine layer for spacemacs. Since then I have learned there are a few other org-mode users so I wanted to share how I got cfengine3 src blocks execution working. I added the following to my dotspacemacs/user-init.
(defcustom org-babel-cfengine3-command "/var/cfengine/bin/cf-agent" "Name of command to use for executing CFEngine policy.") (defvar org-babel-cfengine3-command-options "--no-lock" "Option string that should be passed to the agent. Note that --file will be appended to the options.") (defvar org-babel-cfengine3-file-control-stdlib "body file control{ inputs => { '$(sys.libdir)/stdlib.cf' };}\n" "File control body to include the standard library from $(sys.libdir). It is useful to inject into an example source block before execution so that bundles and bodies from the standard library are automatically available.") (defun org-babel-execute:cfengine3 (body params) "Actuate a block of CFEngine 3 policy. This function is called by `org-babel-execute-src-block'. A temporary file is constructed containing `org-babel-cfengine3-file-control-stdlib and the body of the src block. `org-babel-cfengine3-command' is used to execute the temporary file." (let* ((temporary-file-directory ".") (tempfile (make-temp-file "cfengine3-"))) (with-temp-file tempfile ;; TODO Consider making automatic stdlib inclusion optional (insert org-babel-cfengine3-file-control-stdlib) (insert body)) (unwind-protect (shell-command-to-string (concat org-babel-cfengine3-command " " ;; TODO Consider adding a header option to specify bundlesequence org-babel-cfengine3-command-options " " (format " --file %s" tempfile))) (delete-file tempfile)))) Now any time I have a cfengine3 SRC block in org-mode I can simply run org-babel-execute-src-block ( CTRL-c CTRL-c in emacs/spacemacs documentation this is written as C-c C-c ) to have the block written to a temporary file, executed and the temporary file deleted. For example with my insertion placed inside the following SRC block.
This post was authored by Aleksey Tsalolikhin and has been re-published with his consent. I recently spoke at Digital Media Educators Conference (DMEC) on Infrastructure Management at Scale and the skills educators need to impart to up and coming system administrators. This conference serves the California community college system, which is dear to my heart. My mother worked at West Los Angeles College library her entire professional life in America, since we arrived in 1988. I used to volunteer and help her out with shelving in the summer. I was a very poor helper since I kept getting distracted by all the delicous books and did more reading than shelving. While in high school I took computer programming, math and English at West Los Angeles College and at Santa Monica Community College, at first during summer break and then concurrent with eleventh grade, which allowed me to go to University instead of going to 12th grade. So I have a personal connection to the California community college system and I jumped at the chance to contribute a talk: Because my presentation was in the Data Representation track, I focused on Inventory and Compliance Reporting so I could show off CFEngine’s slick UI. I started by laying out CFEngine’s philosophic groundwork: - Promise Theory and the advantages of voluntary cooperation and distributed work over the limitations of imposed direct control. - The advantages of pull over push (see “Push versus pull” in Deconstructing the `CAP theorem’ for CM and DevOps by the author of CFEngine for more on this), and - The Dunbar numbers which constrain the quality and quantity of relationships sysadmins are able to have with their infrastructures. The rest of the talk demonstrated how the design of CFEngine uses Dunbar numbers to focus the information it presents. We also talked about what computer system administration IS, and what the challenges are and how we handle them. Then I introduced the CFEngine dashboard: I pointed out the header which holds the host count (2, including the hub itself) and the health indicator (OK); the graph of Changes made by CFEngine, the fact that both of our hosts have Software Updates available (1 alert triggered on 2 hosts), and that we have 100% compliance on promise compliance and system health (green check-marks). The next slide, adding a third host (notice the hosts indicator up top), shows how the Alert for Software Updates changes to a 2/3 arc, as, right after adding the host, as at this point the hub knows 2 out of 3 hosts are missing software updates. Once the agent runs on the third host and the hub collects the report, the Alert will change back to a full circle with 3 out of 3 hosts are missing software updates. The next slide illustrates how CFEngine communicates the severity of the alert: critical issues are indicated in red, less severe in orange (amber for you Aussies), and mildest level is yellow. I induced a policy non-compliance situation on one of the three hosts (e.g., promised a file edit but prevented CFEngine from accessing the file by filling up the disk), so the Promise Compliance alert spans 1/3 of the circle (1 out of 3 hosts). Notice also that if CFEngine is unable to collect reports from a host or if an agent stops running on a host, the health indicator at the top of the screen changes from OK to a red number indicating the number of issues: You can see the number and type of issues: Notice that the Dunbar numbers are in play here: CFEngine tells you there are issues, and if you want more data, then you can have it. But it doesn’t throw all the detail at you at once, that would be too much. You can get more detail on which hosts are not reporting by selecting “Hosts not reporting” from the health indicator menu: You can then select a host in the list of hosts not reporting to see the info for that host (host detail). That actually takes us to the “Hosts” tab. The “Hosts” tab starts in the “all hosts” view, where you see the promise compliance summary for your infrastructure: You can list the hosts that have less than 100% compliance: You can see which promises were not kept on each host: And that takes us to the “Reports” tab. There are many reports available but let’s take a look at the Inventory Report. It starts out with four basic columns but you can add more: You can extend inventory collection by writing CFEngine promises, for example, here I’ve added inventory of the host’s timezone: Let’s say our company policy says all hosts must be in the UTC timezone. But in reality we have this: You can sort the column contents by selecting the column heading, this groups the outliers and brings them into view: You can graphically summarize column contents by selecting “Chart Data”: Voila! Hover over a slice to get more detail: Or switch to column view: Here is another example: The charts can be exported and embedded in reports to management, auditors, etc. Want to give CFEngine Enterprise a try? It’s very easy to download and install the hub package. Feel free to email me if you have any questions!
We’re happy to announce maintenance releases for both supported CFEngine release branches today! Being maintenance (aka patch) releases, the goal is to increase stability and reliability for CFEngine users and enable a safe upgrade-path. As such, the releases primarily include bugfixes and low-risk changes that do not impact the compatibility between previous patch releases. Looking at the CFEngine release schedule, we can see that
3.6 and 3.8 are no longer maintained. If you are still on any of these versions, please consider upgrading. 3.7 LTS is maintained (and supported for Enterprise customers) until July 17th 2018 3.9 non-LTS is maintained until December 2016 For CFEngine Enterprise customers, the only supported release today is 3.7. The next supported release will be 3.10 LTS due end of 2016, which will be supported through the end of 2019. If you are planning to contribute improvements to 3.10 (thank you!), please note that we would need the pull requests ready for merging by mid-September in order to have time to incorporate them into 3.10.
We’re happy to announce that CFEngine 3.9.0 non-LTS now is released! A big thanks to everyone testing the 3.9.0 beta release! Please note that this is a non-LTS release, which means that it is maintained for 6 months from the release date and not supported for CFEngine Enterprise customers (but packages are available for testing). The established CFEngine release schedule gives an overview over the release timelines for all releases.
Running remote bundles CFEngine 3.9 introduces a simple way to trigger bundles to run remotely by using cf-runagent –remote-bundles to ask a remote cf-serverd to run a given set of bundles. For example:
As someone responsible for IT operations, you have probably spent most of your time thinking of ways to improve your team’s effectiveness and efficiency to make it more competitive. At CFEngine, we have witnessed the management of IT infrastructure and organizations operating across the spectrum of the performance scale. Where are you on that scale and how can you improve? In the world of IT operations, a negatively skewed distribution seems to exist. The majority of companies, despite their admirable DevOps initiatives, struggle. The great minority who are masters of execution continuously increase their relative competitiveness leaving the others behind. In my previous post, “The painful road to the digital economy,” I pointed out some ideas as to why becoming more competitive seems so hard for Fortune 2000 companies and suggested ways to improve. It might be worth a read if you find yourself too far on the left side in the illustration below. The characteristics of a poor and reactive versus proactive and high-performing IT organization can be many. Below you will find some of the most common observed with our own users. Have a look and compare them with your own state of operations.
We’re happy to announce that CFEngine 3.9.0 non-LTS beta is now ready for testing! The established CFEngine release schedule shows that the 3.9.0 final version is due in June 2016, so it’s time to test and fix any remaining issues. Please note that this is a non-LTS release, which means that it is maintained for 6 months from the release date and not supported for CFEngine Enterprise customers, but packages are available for testing.
By now you have probably heard about the Badlock vulnerability (CVE-2016-2118)in DCE/RPC-based SAMR and LSA protocols used in the Microsoft Windows ActiveDirectory infrastructure as well as other critical security flows in Samba. With CFEngine Enterprise you can simply tag any variable or class and MissionPortals Inventory reporting interface will be automatically extended with the new attributes. This makes it easy to identify vulnerable hosts. Dashboard alerts can be created to alert on vulnerable hosts for specific subsets of infrastructure. Dashboard alerts can be integrated with other systems. For example you could automatically open an issue in Jira when vulnerable hosts are found. If you would like to use CFEngine to detect, repair and report on Badlock in your infrastructure, we have prepared some policies you can use: - Badlock reporting and remediation policy - Implementation Tutorial
We’re happy to announce maintenance releases for all supported CFEngine release branches today! Being maintenance (aka patch) releases, the goal is to increase stability and reliability for CFEngine users and enable a safe upgrade-path. As such, the releases primarily include bugfixes and low-risk changes that do not impact the compatibility between previous patch releases. Looking at the CFEngine release schedule, we can see that
3.7 LTS is maintained until July 17th 2018 3.6 is maintained until July 17th 2016 (released before CFEngine incorporated the LTS model) 3.8 (being a non-LTS release) is maintained until 3.9.0 is released For CFEngine Enterprise customers, the supported releases as of today are 3.7 and 3.6. If you are on 3.6, it is a good time to prepare for an upgrade to 3.7, as the next supported minor release will be 3.10 LTS due end of 2016. If you are planning to contribute improvements to 3.9 (thank you!), please note that we would need the pull requests by beginning of April in order to have time to incorporate them into 3.9.
CFEngine is one of the most widely used infrastructure automation solutions in the world, being leveraged by some of the biggest organizations in the world for infrastructure management. eZuce is an Enterprise provider of Unified Communications, Voice and Video Conferencing solutions and is being used by many of their customers to improve business processes. eZuce has been shipping CFEngine inside of UniteMe since 2012 and have benefited from the following by using the popular configuration management solution:
