With the recent release of build.cfengine.com and cfbs I have been thinking about the process of converting a traditionally manged policy set. I consider a traditionally manged policy set one where you have a repo with the root of masterfiles being the root of the repository, or even having no repository at all and managing masterfiles by editing directly in the distribution point (e.g. /var/cfengine/masterfiles). Before jumping in with both feet and converting to a cfbs managed policy set you might want a hybrid situation where you can leverage some of the benefits of cfbs but without making drastic changes to the way policy is currently managed. That’s what this post is about, using cfbs with your traditionally manged policy set. Note: This post assumes that you already have cfbs installed and understand the basics of how it works. Check out our previous blog posts if you want to review how to get started with cfbs.
When running cfbs init it’s suggested that you cfbs add masterfiles as your first module. While modules can declare dependencies, none of the modules in the default index currently depend on masterfiles.
So, if we don’t cfbs add masterfiles, running cfbs add on other modules won’t result in the MPF (Masterfiles Policy Framework) being included in the cfbs build.
Let’s see:
Here we make a directory for our cfbs project, switch into it and run cfbs init.
cfbs_dir=/tmp/cfbs-playground
mkdir -p ${cfbs_dir}; cd ${cfbs_dir}
cfbs initInitialized - edit name and description cfbs.json
To add your first module, type: cfbs add masterfilesSo, the project was successfully initialized and we are instructed to cfbs add masterfiles but we will skip that bit since we are looking to use cfbs with a traditionally manged policy set, or even a policy set that’s not based on the MPF. Next we can add autorun, a module that simply defines the class services_autorun to enable a feature to automatically load policy files in services/autorun and run any bundles tagged with autorun (a feature of the MPF).
cfbs add autorun && cfbs buildAdded module: autorun
Modules:
001 autorun @ c3b7329b240cf7ad062a0a64ee8b607af2cb912a (Downloaded)
Steps:
001 autorun : json 'def.json' 'masterfiles/def.json'
Generating tarball...
Build complete, ready to deploy 🐿
-> Directory: out/masterfiles
-> Tarball: out/masterfiles.tgz
To install on this machine: cfbs install
To deploy on remote hub(s): cf-remote deploy --hub hub out/masterfiles.tgzThe build completed successfully and we can see that the directory out/masterfiles contains the resulting assets (as well an accompanying archive at out/masterfiles.tgz).
Let’s take a look at the artifacts produced by the build.
find out/masterfiles/*out/masterfiles/def.jsonWe see just one file, def.json.
Looking at the content we can see that services_autorun gets defined and nothing else.
jq < out/masterfiles/def.json{
"classes": {
"services_autorun": [
"any"
]
}
}So, by simply adding and building the autorun module we get an augments file defining the class to enable the feature.
This could be merged with the augments as they sit in your traditionally managed policy set.
Let’s look at a def.json from a contrived policy set based on the MPF.
jq < /tmp/contrived-masterfiles/def.json{
"inputs": [
"services/my_custom.cf"
],
"vars": {
"msg": "Hello"
},
"classes": {
"contrived_policy_set": [
"any::"
]
}
}From the output we can see that there is a custom policy file (services/my_custom) added to inputs a variable, msg in the def bundle scope is defined with the value Hello and the class contrived_policy_set is defined for any hosts.
We can use jq to merge the cfbs built def.json into the one that already exists in our traditionally manged contrived policy set.
jq -n 'reduce inputs as $i ({}; .* $i)' \
/tmp/contrived-masterfiles/def.json \
/tmp/cfbs-playground/out/masterfiles/def.json \
| tee /tmp/cfbs-playground/out/masterfiles/def.json.merged{
"inputs": [
"services/my_custom.cf"
],
"vars": {
"msg": "Hello"
},
"classes": {
"contrived_policy_set": [
"any::"
],
"services_autorun": [
"any"
]
}
}We can diff the original with the merged result and see that the classes object was extended to include the services_autorun class definition.
diff -u \
/tmp/contrived-masterfiles/def.json \
/tmp/cfbs-playground/out/masterfiles/def.json.merged | cat--- /tmp/contrived-masterfiles/def.json 2022-01-28 13:27:04.845285598 -0600
+++ /tmp/cfbs-playground/out/masterfiles/def.json.merged 2022-01-28 13:27:08.681439446 -0600
@@ -8,6 +8,9 @@
"classes": {
"contrived_policy_set": [
"any::"
+ ],
+ "services_autorun": [
+ "any"
]
}
}We can continue down this path, let’s add another module that vendors more than augments.
In the cfbs project directory run let’s add and build with tmp-file-age, a module to inventory and manage old files in /tmp.
cfbs add tmp-file-age && cfbs buildAdded module: tmp-file-age
Modules:
001 autorun @ c3b7329b240cf7ad062a0a64ee8b607af2cb912a (Downloaded)
002 tmp-file-age @ 56a7c149f33808db6796de77eff6eb0502c01e61 (Downloaded)
Steps:
001 autorun : json 'def.json' 'masterfiles/def.json'
002 tmp-file-age : copy './tmp-file-age.cf' 'masterfiles/services/security-hardening/tmp-file-age/'
002 tmp-file-age : json 'cfbs/def.json' 'masterfiles/def.json'
Generating tarball...
Build complete, ready to deploy 🐿
-> Directory: out/masterfiles
-> Tarball: out/masterfiles.tgz
To install on this machine: cfbs install
To deploy on remote hub(s): cf-remote deploy --hub hub out/masterfiles.tgzLooking at the artifacts resulting from the build we can see that a policy file services/security-hardening/tmp-file-age/tmp-file-age.cf is added.
tree /tmp/cfbs-playground/out/masterfiles//tmp/cfbs-playground/out/masterfiles/
├── def.json
└── services
└── security-hardening
└── tmp-file-age
└── tmp-file-age.cf
3 directories, 2 filesLooking at the def.json resulting from the cfbs build we can see that the policy file was added to inputs and a bundle was added to the bundlesequence.
jq < /tmp/cfbs-playground/out/masterfiles/def.json{
"classes": {
"services_autorun": [
"any"
]
},
"vars": {
"control_common_bundlesequence_end": [
"northerntech_security_hardening:tmp_file_age"
]
},
"inputs": [
"services/security-hardening/tmp-file-age/tmp-file-age.cf"
]
}As done previously, we can merge this with our preexisting def.json.
jq -n 'reduce inputs as $i ({}; .* $i)' \
/tmp/contrived-masterfiles/def.json \
/tmp/cfbs-playground/out/masterfiles/def.json \
| tee /tmp/cfbs-playground/out/masterfiles/def.json.merged{
"inputs": [
"services/security-hardening/tmp-file-age/tmp-file-age.cf"
],
"vars": {
"msg": "Hello",
"control_common_bundlesequence_end": [
"northerntech_security_hardening:tmp_file_age"
]
},
"classes": {
"contrived_policy_set": [
"any::"
],
"services_autorun": [
"any"
]
}
}Again we can diff to see the detailed changes.
diff -u \
/tmp/contrived-masterfiles/def.json \
/tmp/cfbs-playground/out/masterfiles/def.json.merged | cat--- /tmp/contrived-masterfiles/def.json 2022-01-28 13:27:04.845285598 -0600
+++ /tmp/cfbs-playground/out/masterfiles/def.json.merged 2022-01-28 14:12:24.653577092 -0600
@@ -1,13 +1,19 @@
{
"inputs": [
- "services/my_custom.cf"
+ "services/security-hardening/tmp-file-age/tmp-file-age.cf"
],
"vars": {
- "msg": "Hello"
+ "msg": "Hello",
+ "control_common_bundlesequence_end": [
+ "northerntech_security_hardening:tmp_file_age"
+ ]
},
"classes": {
"contrived_policy_set": [
"any::"
+ ],
+ "services_autorun": [
+ "any"
]
}
}Oh no! Good thing we checked, that’s not quite right. The inputs array was replaced instead of extended. We need a different incantation for jq. Let’s try again …
jq -s 'def deepmerge(a;b):
reduce b[] as $item (a;
reduce ($item | keys_unsorted[]) as $key (.;
$item[$key] as $val | ($val | type) as $type | .[$key] = if ($type == "object") then
deepmerge({}; [if .[$key] == null then {} else .[$key] end, $val])
elif ($type == "array") then
(.[$key] + $val | unique)
else
$val
end)
);
deepmerge({}; .)' \
/tmp/contrived-masterfiles/def.json \
/tmp/cfbs-playground/out/masterfiles/def.json \
| tee /tmp/cfbs-playground/out/masterfiles/def.json.merged{
"inputs": [
"services/my_custom.cf",
"services/security-hardening/tmp-file-age/tmp-file-age.cf"
],
"vars": {
"msg": "Hello",
"control_common_bundlesequence_end": [
"northerntech_security_hardening:tmp_file_age"
]
},
"classes": {
"contrived_policy_set": [
"any::"
],
"services_autorun": [
"any"
]
}
}Let’s check that our merge behaved as desired.
diff -u \
/tmp/contrived-masterfiles/def.json \
/tmp/cfbs-playground/out/masterfiles/def.json.merged | cat--- /tmp/contrived-masterfiles/def.json 2022-01-28 14:33:45.518200635 -0600
+++ /tmp/cfbs-playground/out/masterfiles/def.json.merged 2022-01-28 14:34:41.658773202 -0600
@@ -1,13 +1,20 @@
{
"inputs": [
- "services/my_custom.cf"
+ "services/my_custom.cf",
+ "services/security-hardening/tmp-file-age/tmp-file-age.cf"
],
"vars": {
- "msg": "Hello"
+ "msg": "Hello",
+ "control_common_bundlesequence_end": [
+ "northerntech_security_hardening:tmp_file_age"
+ ]
},
"classes": {
"contrived_policy_set": [
- "any::"
+ "any::"
+ ],
+ "services_autorun": [
+ "any"
]
}
}Now that we are confident in the changes to def.json we can put the merged result into place and copy the policy built by cfbs into our traditionally managed contrived policy set.
mv -v \
/tmp/cfbs-playground/out/masterfiles/def.json.merged \
/tmp/contrived-masterfiles/def.json
rsync -av\
/tmp/cfbs-playground/out/masterfiles/ \
/tmp/contrived-masterfiles/ \
--exclude def.jsonrenamed '/tmp/cfbs-playground/out/masterfiles/def.json.merged' -> '/tmp/contrived-masterfiles/def.json'
sending incremental file list
./
services/
services/security-hardening/
services/security-hardening/tmp-file-age/
services/security-hardening/tmp-file-age/tmp-file-age.cf
sent 3,097 bytes received 54 bytes 6,302.00 bytes/sec
total size is 2,832 speedup is 0.90That’s it, we integrated both the autorun module and the tmp-file-age module into our traditionally manged policy set. This is a good way to easily consume content with cfbs before moving to a fully managed cfbs policy set. Moving fully to cfbs brings other great features like removing modules you no longer want and cleaning up unused dependencies.
