The Policy Framework

Table of Contents

This documentation is deprecated in favor of reference/masterfiles-policy-framework*

The CFEngine policy framework is called the Masterfiles Policy Framework, MPF, or simply masterfiles because the files live in /var/cfengine/masterfiles on the policy server (on the clients, and note the policy server is typically also a client, they are cached in /var/cfengine/inputs).

The following configuration files are part of the default CFEngine installation in /var/cfengine/masterfiles, and have special roles.

The Masterfiles Policy Framework is continually updated. You can track its development on github. Notable changes to the framework are documented in the changelog.

Setting up

First, review the update_def and def bundles found in controls/update_def.cf and controls/def.cf respectively. Many settings you need to change will live here.

It is recommended to use an augments file to specify things traditionally set in update_def.cf and def.cf.

We will cover the policy in the order it is activated, starting with update.cf and its bundlesequence followed by promises.cf and its bundlesequence.

update.cf

Synchronizing clients with the policy server happens here, in update.cf. Its main job is to copy all the files on the policy server (usually the hub) under $(sys.masterdir) (usually /var/cfengine/masterfiles) to the local host into $(sys.inputdir) (usually /var/cfengine/inputs).

This file should rarely if ever change. Should you ever change it (or when you upgrade CFEngine), take special care to ensure the old and the new CFEngine can parse and execute this file successfully. If not, you risk losing control of your system (that is, if CFEngine cannot successfully execute update.cf, it has no mechanism for distributing new policy files).

By default, the policy defined in update.cf is executed at the beginning of a cf-execd scheduled agent run (see schedule and exec_command as defined in body executor control in controls/cf_execd.cf). When the update policy completes (regardless of success or failure) the policy defined in promises.cf is activated.

This is a standalone policy file. You can actually run it with cf-agent -KI -f ./update.cf but if you don't understand what that command does, please hold off until you've gone through the CFEngine documentation. The contents of update.cf duplicate other things under lib sometimes, in order to be completely standalone.

To repeat, when update.cf is broken, things go bonkers. CFEngine will try to run a backup failsafe.cf you can find in the C core under libpromises/failsafe.cf (that .cf file is written into the C code and can't be modified). If things get to that point, you probably have to look at why corrupted policies made it into production.

As is typical for CFEngine, the policy and the configuration are mixed. In controls/update_def.cf you'll find some very useful settings. Keep referring to controls/update_def.cf as you read this. We are skipping the nonessential ones.

How it works

There are multiple stages in update.cf. This document covers each bundle in the order defined by the bundlesequence.

update_def (bundle)

This bundle is defined in controls/update_def.cf. bundle common update_def defines settings and variables that are used throughout the update policy.

As of CFEngine version 3.7 it is recommended that these setting changes are specified in def.json to ease policy framework updates.

input_name_patterns (variable)

A list of regular expressions defining which files should be considered for copying during update.

masterfiles_perms_mode (variable)

Usually you want to leave this at 0600 meaning the inputs will be readable only by their owner.

trigger_upgrade (class)

Off by default

When this class is set, the internal CFEngine upgrade mechanism is enabled. Currently this upgrade policy is specific to CFEngine Enterprise.

cfengine_internal_masterfiles_update (class)

Off by default.

This class enables masterfiles automatic update from a version control repository. Currently this policy relies on tooling available in CFEngine Enterprise.

Turn this on (set to any) to auto-deploy policies on the policy server, it has no effect on clients. See Version Control and Configuration Policy for details on how to use it.

This may result in DATA LOSS.

cfengine_internal_encrypt_transfers (class)

Off by default.

This class enables encryption during policy updates. If you are running CFEngine versions 3.6 with protocol => "2" or protocol => "latest" this settings is unnecessary as all traffic will be encapsulated inside of TLS. CFEngine Version 3.7 uses protocol => "2" by default.

Turn this on (set to any) to encrypt your policy transfers.

Note it has a duplicate in def.cf, see below. If they are not synchronized, you may get unexpected behavior.

cfengine_internal_purge_policies (class)

Off by default.

This class causes the update behavior to change from only copying changed files down to performing a synchronization by purging files on the client that do not exist on the server.

Turn this on (set to any) to delete any files in your $(sys.inputdir) that are not in the policy server's masterfiles.

This may result in DATA LOSS.

Note it has a duplicate in def.cf, see below. If they are not synchronized, you may get unexpected behavior.

cfengine_internal_preserve_permissions (class)

Off by default.

Turn this on (set to any) to preserve the permissions of the policy server's masterfiles when they are copied.

This may result in FUNCTIONALITY LOSS if your scripts lose their exec bits unexpectedly

Note it has a duplicate in def.cf, see below. If they are not synchronized, you may get unexpected behavior.

cfengine_internal_disable_cf_promises_validated (class)

Off by default.

Turn this on (set to any) to have remote agents always scan all of masterfiles for changes and update accordingly.

This is not recommended as it both removes a safety mechanism that checks for policy to be valid before allowing clients to download updates, and the increased load on the hub will affect scalability.

Consider using time_based, select_class or dist based classes instead of any to retain some of the benefits.

enable_cfengine_enterprise_hub_ha (class)

Off by default.

This class enables the HA policy for CFEngine Enterprise hubs. This class is not set by default.

cfe_internal_dc_workflow (bundle)

This bundle implements the auto-deployment of policies. See Version Control and Configuration Policy and cfengine_internal_masterfiles_update below for details. This policy is currently specific to CFEngine Enterprise.

cfe_internal_update_policy (bundle)

This bundle is defined in cfe_internal/update/update_policy.cf. It updates the policy files themselves. Basically it's a check step that looks at $(sys.inputdir)/cf_promises_validated and compares it with the policy server's $(sys.masterdir)/cf_promises_validated. Then there's the actual copy, which happens only if the cf_promises_validated file was updated in the check step. You can bypass this check and perform a full scan by running cf-agent -KIf update.cf -D validated_updates_ready.

Implementation (warning: advanced usage):

bundle agent cfe_internal_update_policy
{
  classes:

      # Define classes if we see a user is requesting a custom policy update bundle

      "have_user_specified_update_bundle"
        expression => isvariable( "def.mpf_update_policy_bundle" );

      # Define classes if we are able to find the specific bundle they requested
      # (otherwise we may get an error about undefined bundle)

      "have_found_user_specified_update_bundle"
        expression  => some(".*", "found_matching_user_specified_bundle");

      "missing_user_specified_update_bundle"
       not  => some(".*", "found_matching_user_specified_bundle");



  vars:
      "default_policy_update_bundle" string => "cfe_internal_update_policy_cpv";

      # Look for a bundle that matches what the user wants
      "found_matching_user_specified_bundle"
        slist => bundlesmatching( "$(def.mpf_update_policy_bundle)" );

  methods:

    # Use the user specified bundle when it's found
    have_found_user_specified_update_bundle::

        "User specified policy update bundle"
          usebundle => $(found_matching_user_specified_bundle);


    # Fall back to stock policy update bundle if we have not found one
    # specified by user

    !have_found_user_specified_update_bundle::

      "Stock policy update"
        usebundle => cfe_internal_update_policy_cpv;

  reports:

    inform_mode|verbose_mode|DEBUG|DEBUG_cfe_internal_update_policy::
      # Report a human readable way to understand the policy behavior

      "Found user specified update bundle."
        if => "have_user_specified_update_bundle";

      "User specified update bundle: $(def.mpf_update_policy_bundle)"
        if => "have_user_specified_update_bundle";

      "User specified update bundle MISSING! Falling back to $(default_policy_update_bundle)."
        if => and( "have_user_specified_update_bundle",
                   "missing_user_specified_update_bundle"
                  );


}

bundle agent cfe_internal_setup_python_symlink(symlink_path)
{
  vars:
      "path" string => getenv("PATH", 1024);

      "_path_folders" slist => splitstring("$(path)", ":", 128);

      "path_folders" slist => filter("$(sys.bindir)",
                                     @(_path_folders),
                                     false, true, 128);

    windows::
      "abs_path_folders" -> {"CFE-2309"}
        slist => filter("([A-Z]|//):.*", path_folders, "true", "false", 128),
        comment => "findfiles() complains about relative directories";

    !windows::
      "abs_path_folders" -> {"CFE-2309"}
        slist => filter("/.*", path_folders, "true", "false", 128),
        comment => "findfiles() complains about relative directories";

    any::
      "exact_version_globs" slist => maplist("$(this)/python[23]", @(abs_path_folders)),
        comment => "Looking for Python 2 and/or Python 3 in the $PATH folders";

      "generic_python_globs" slist => maplist("$(this)/python", @(abs_path_folders)),
        comment => "Looking for the 'python' symlink/executable which can be any
                    version of Python (usually Python 2 for backwards compatibility)";

      "python_exact[$(exact_version_globs)]" slist => findfiles("$(exact_version_globs)");
      "python_generic[$(generic_python_globs)]" slist => findfiles("$(generic_python_globs)");
      "python_platform_fallback[/usr/libexec/platform-python]" -> { "CFE-3291" }
        slist => { "/usr/libexec/platform-python" };

      "python_exact_sorted" slist => reverse(sort(getvalues(@(python_exact)), "lex")),
        comment => "Prefer higher major versions of Python";

      "pythons" slist => getvalues(mergedata(@(python_exact_sorted),
                                             getvalues(@(python_generic)),
                                             getvalues(@(python_platform_fallback)))),
        comment => "Prefer exact versions over unknown";

      "python" string => nth(@(pythons), 0),
        if => isgreaterthan(length(@(pythons)), 0),
        comment => "Taking the first item from the list (sorted by preference)";

  files:
      "$(symlink_path)"
        delete => u_tidy,
        if => not(isvariable("python"));

      "$(symlink_path)"
        link_from => u_ln_s("$(python)"),
        move_obstructions => "true",
        if => isvariable("python");
}

bundle agent cfe_internal_update_policy_cpv
{
  vars:
      "inputs_dir"         string => translatepath("$(sys.inputdir)"),
      comment => "Directory containing CFEngine policies",
      handle => "cfe_internal_update_policy_vars_inputs_dir";

      "master_location" -> { "ENT-3692" }
        string => "$(update_def.mpf_update_policy_master_location)",
        comment => "The path to request updates from the policy server.",
        handle => "cfe_internal_update_policy_vars_master_location";

    windows::

      "modules_dir"        string => "/var/cfengine/modules",      # NB! NOT $(sys.workdir) on Windows !
      comment => "Directory containing CFEngine modules",
      handle => "cfe_internal_update_policy_vars_modules_dir_windows";

      "plugins_dir"        string => "/var/cfengine/plugins",      # NB! NOT $(sys.workdir) on Windows !
      comment => "Directory containing CFEngine plugins",
      handle => "cfe_internal_update_policy_vars_plugins_dir_windows";

    !windows::

      "modules_dir"        string => translatepath("$(sys.workdir)/modules"),
      comment => "Directory containing CFEngine modules",
      handle => "cfe_internal_update_policy_vars_modules_dir";

      "plugins_dir"        string => translatepath("$(sys.workdir)/plugins"),
      comment => "Directory containing CFEngine plugins",
      handle => "cfe_internal_update_policy_vars_plugins_dir";

    any::

      "file_check"         string => translatepath("$(inputs_dir)/promises.cf"),
      comment => "Path to a policy file",
      handle => "cfe_internal_update_vars_file_check";

      "ppkeys_file"        string => translatepath("$(sys.workdir)/ppkeys/localhost.pub"),
      comment => "Path to public key file",
      handle => "cfe_internal_update_policy_vars_ppkeys_file";

      "postgresdb_dir"        string => "$(sys.workdir)/state/pg/data",
      comment => "Directory where Postgres database files will be stored on hub -",
      handle => "cfe_internal_update_policy_postgresdb_dir";

      "postgresdb_log"        string => "/var/log/postgresql.log",
      comment => "File where Postgres database files will be logging -",
        handle => "cfe_internal_update_policy_postgresdb_log_file";

      "python_symlink"        string => "$(sys.bindir)/python",
        comment => "Symlink to Python we found (if any)",
        handle => "cfe_internal_update_policy_python_symlink";

    cfredis_in_enterprise::

      # TODO Remove from MPF after 3.12 EOL

      "redis_conf_file" -> { "ENT-2797" }
        string => translatepath("$(sys.workdir)/config/redis.conf"),
        comment => "Path to Redis configuration file",
        handle => "cfe_internal_update_policy_redis_conf_file";

  classes:

      "validated_updates_ready"
        expression => "cfengine_internal_disable_cf_promises_validated",
        comment => "If cf_promises_validated is disabled, then updates are
                    always considered validated.";

    any::

      "local_files_ok" expression => fileexists("$(file_check)"),
      comment => "Check for $(sys.masterdir)/promises.cf",
      handle => "cfe_internal_update_classes_files_ok";

      # create a global files_ok class
      "cfe_internal_trigger" expression => "local_files_ok",
      classes => u_if_else("files_ok", "files_ok");

  files:

    !am_policy_hub::  # policy hub should not alter inputs/ uneccessary

      "$(inputs_dir)/cf_promises_validated"
      comment => "Check whether a validation stamp is available for a new policy update to reduce the distributed load",
      handle => "cfe_internal_update_policy_check_valid_update",
      copy_from => u_rcp("$(master_location)/cf_promises_validated", @(update_def.policy_servers)),
      action => u_immediate,
      classes => u_if_repaired("validated_updates_ready");

    !am_policy_hub.!windows::

      "$(modules_dir)"
      comment => "Always update modules files on client side",
      handle => "cfe_internal_update_policy_files_update_modules",
      copy_from => u_rcp("$(modules_dir)", @(update_def.policy_servers)),
      depth_search => u_recurse("inf"),
      perms => u_m("755"),
      action => u_immediate;

      "$(plugins_dir)"
      comment => "Always update plugins files on client side",
      handle => "cfe_internal_update_policy_files_update_plugins",
      copy_from => u_rcp("$(plugins_dir)", @(update_def.policy_servers)),
      depth_search => u_recurse("inf"),
      perms => u_m("755"),
      action => u_immediate;

    !am_policy_hub.windows::

      "$(sys.workdir)\modules"
      comment => "Always update modules files on client side (Windows)",
      handle => "cfe_internal_update_policy_files_update_modules_windows",
      copy_from => u_rcp("$(modules_dir)", @(update_def.policy_servers)),
      depth_search => u_recurse("inf"),
      perms => u_m("755"),
      action => u_immediate;

      "$(sys.workdir)\plugins"
      comment => "Always update plugins files on client side (Windows)",
      handle => "cfe_internal_update_policy_files_update_plugins_windows",
      copy_from => u_rcp("$(plugins_dir)", @(update_def.policy_servers)),
      depth_search => u_recurse("inf"),
      perms => u_m("755"),
      action => u_immediate;

    am_policy_hub|validated_updates_ready::  # policy hub should always put masterfiles in inputs in order to check new policy

      "$(inputs_dir)"
      comment => "Copy policy updates from master source on policy server if a new validation was acquired",
      handle => "cfe_internal_update_policy_files_inputs_dir",
      copy_from => u_rcp("$(master_location)", @(update_def.policy_servers)),
      depth_search => u_recurse("inf"),
      file_select  => u_input_files,
      action => u_immediate,
      classes => u_results("bundle", "update_inputs");

    update_inputs_not_kept::

      "$(inputs_dir)/cf_promises_validated" -> { "CFE-2587" }
        delete => u_tidy,
        comment => "If there is any problem copying to $(inputs_dir) then purge
                    the cf_promises_validated file must be purged so that
                    subsequent agent runs will perform a full scan.";

    !policy_server.enable_cfengine_enterprise_hub_ha::
      "$(sys.workdir)/policy_server.dat"
      comment => "Copy policy_server.dat file from server",
      handle => "cfe_internal_update_ha_policy_server",
      copy_from => u_rcp("$(sys.workdir)/state/master_hub.dat", @(update_def.policy_servers)),
      action => u_immediate,
      classes => u_if_repaired("replica_failover");  # not needed ?

    am_policy_hub::

      "$(master_location)/."
      comment => "Make sure masterfiles folder has right file permissions",
      handle => "cfe_internal_update_policy_files_sys_workdir_masterfiles",
      perms => u_m($(update_def.masterfiles_perms_mode)),
      depth_search => u_recurse_basedir("inf"),
      action => u_immediate;

  methods:
    debian|redhat|amazon_linux|suse|sles|opensuse::
      # Only needed on distros with Python-based package modules
      "setup_python_symlink" usebundle => cfe_internal_setup_python_symlink("$(python_symlink)");
}


body perms u_m(p)
{
      mode  => "$(p)";
}


body perms u_mo(p,o)
{
      mode   => "$(p)";
      owners => {"$(o)"};
}


body perms u_shared_lib_perms
{
    !hpux::
      mode => "0644";
    hpux::
      mode => "0755"; # Mantis 1114, Redmine 1179
}


body file_select u_cf3_files
{
      leaf_name => { "cf-.*" };
      file_result => "leaf_name";
}


body file_select u_input_files
{
      leaf_name => { @(update_def.input_name_patterns) };
      file_result => "leaf_name";
}


body copy_from u_rcp(from,server)
{
      source      => "$(from)";
      compare     => "digest";
      trustkey    => "false";


    !am_policy_hub::
      servers => { "$(server)" };

    !am_policy_hub.sys_policy_hub_port_exists::
      portnumber => "$(sys.policy_hub_port)";

    cfengine_internal_encrypt_transfers::
      encrypt => "true";

    cfengine_internal_purge_policies::
      purge => "true";

    cfengine_internal_preserve_permissions::
      preserve => "true";

    cfengine_internal_verify_update_transfers::
      verify      => "true";
}


body copy_from u_cp(from)
{
      source  => "$(from)";
      compare => "digest";
}


body copy_from u_cp_nobck(from)
{
      source      => "$(from)";
      compare     => "digest";
      copy_backup => "false";
}


body action u_immediate
{
      ifelapsed => "0";
}


body depth_search u_recurse(d)
{
      depth => "$(d)";
      exclude_dirs => { "\.svn", "\.git", "git-core" };
}


body depth_search u_recurse_basedir(d)
{
      include_basedir => "true";
      depth => "$(d)";
      exclude_dirs => { "\.svn", "\.git", "git-core" };
}


body classes u_if_repaired(x)
{
      promise_repaired => { "$(x)" };
}


body classes u_if_repaired_then_cancel(y)
{
      cancel_repaired => { "$(y)" };
}


body classes u_if_else(yes,no)
{
      promise_repaired => { "$(yes)" };
      repair_failed    => { "$(no)" };
      repair_denied    => { "$(no)" };
      repair_timeout   => { "$(no)" };
}


body contain u_in_shell
{
      useshell => "true";
}


body contain u_in_shell_and_silent
{
      useshell => "true";
      no_output => "true";
}


body contain u_postgres
{
  useshell   => "useshell";
  exec_owner => "cfpostgres";
  exec_group => "cfpostgres";
  chdir      => "/tmp";
  no_output  => "true";
}


body action u_ifwin_bg
{
    windows::
      background => "true";
}


body service_method u_bootstart
{
      service_autostart_policy => "boot_time";
}


body contain u_in_dir(s)
{
      chdir => "$(s)";
}


body contain u_silent_in_dir(s)
{
      chdir => "$(s)";
      no_output => "true";
}


body link_from u_ln_s(x)
{
      link_type => "symlink";
      source => "$(x)";
      when_no_source => "force";
}


body delete u_tidy
{
      dirlinks => "delete";
      rmdirs   => "true";
}

cfe_internal_update_bins (bundle)

This step does a self-update of CFEngine. See the Enterprise documentation for details; this functionality is unsupported in CFEngine Community.

cfe_internal_update_processes (bundle)

This step manages the running processes, ensuring cf-execd and cf-serverd and cf-monitord are running and doing some other tasks.

promises.cf

How it works

promises.cf is your main run file. Keep referring to your installation's promises.cf as you read this.

promises.cf is the first file that cf-agent with no arguments will try to look for. So whenever you see cf-agent with no flile parameter, read it as "run my promises.cf".

It should contain all of the basic configuration settings, including a list of other files to include. In normal operation, it must also have a bundlesequence.

bundlesequence

The bundlesequence acts like the 'genetic makeup' of the configuration. Edit the bundlesequence to add any bundles you have defined, or are pre-defined. Consider using the services_autorun facility, an augments file or integrating your bundles as methods type promises so you don't have to edit this setting at all.

BEWARE THAT ONLY VALID (KNOWN) BUNDLES CAN BE ADDED.

By default, the inventory modules, then internal hub modules, then the autorun services, and finally internal management bundles are in the bundlesequence.

In a large configuration, you might want to have a different bundlesequence for different classes of host, so that you can build a complete system like a check-list from different combinations of building blocks. You can construct different lists by composing them from other lists, or you can use methods promises as an alternative for composing bundles for different classes. This is an advanced topic and a risky area (if you get it wrong, your policies will not validate) so make sure you test your changes carefully!

inventory_control (bundle)

The inventory policy was added in CFEngine version 3.6. Inventory modules are desinged to define classes and variables based on the inspected state of the system. These classes and variales can be levereged when writing policy, and in CFEngine Enterprise they can be reported on from Mission Portal. You can disable pieces of it (inventory modules) or the whole thing if you wish.

disable_inventory (class)

This class is off by default (meaning the inventory is on by default). Here's the master switch to disable all inventory modules.

disable_inventory_lsb (class)

LSB is the Linux Standard Base, see https://wiki.linuxfoundation.org/en/LSB

By default, this class is turned off (and the module is on) if the LSB executable /usr/bin/lsb_release can be found. This inventory module will populate inventory reports and variables for you with LSB details. For details, see [LSB][The Policy Framework#LSB]

disable_inventory_dmidecode (class)

By default, this class is turned off (and the module is on) if the executable /usr/sbin/dmidecode can be found. This inventory module will populate inventory reports and variables for you. For details, see [DMI decoding][The Policy Framework#DMI decoding]

disable_inventory_LLDP (class)

LLDP is a protocol for Link Layer Discovery. See http://en.wikipedia.org/wiki/Link_Layer_Discovery_Protocol

By default, this class is turned off (and the module is on) if the executable /usr/bin/lldpctl can be found. This inventory module will populate variables for you. For details, see [LLDP][The Policy Framework#LLDP]

disable_inventory_package_refresh (class)

By default, this class is turned off (and the module is on). This inventory module will populate the installed packages for you. On CFEngine Enterprise, the available packages will also be populated.

disable_inventory_mtab (class)

By default, this class is turned off (and the module is on) if /etc/mtab exists. This inventory module will populate variables for you based on the mounted filesystems. For details, see [mtab][The Policy Framework#mtab]

disable_inventory_fstab (class)

By default, this class is turned off (and the module is on) if $(sys.fstab) (usually /etc/fstab or /etc/vfstab) exists. This inventory module will populate variables for you based on the defined filesystems. For details, see [fstab][The Policy Framework#fstab]

disable_inventory_proc (class)

By default, this class is turned off (and the module is on) if /proc is a directory. This inventory module will populate variables for you from some of the contents of /proc. For details, see [procfs][The Policy Framework#procfs]

disable_inventory_cmdb (class)

By default, this class is turned on (and the module is off).

Turn this on (set to any) to allow each client to load a me.json file from the server and load its contents. For details, see [CMDB][The Policy Framework#CMDB]

@(inventory.bundles) (bundle)

This bundle is defined as bundle common inventory in promises.cf.

Inventory bundles may vary between platform and other classes.

def (bundle)

This bundle is defined as bundle common def in controls/def.cf

def has some crucial settings used by the rest of CFEngine. It's expected that users may edit it, but won't normally change the rest of masterfiles except in services or if they know it's necessary. This bundle should be configured in conjunction with update_def as there are some settings that should be kept in sync between the two policies.

As of CFEngine version 3.7 it is recommended that these setting changes are specified in def.json to ease policy framework updates.

Keep referring to def.cf as you read this.

Implementation (warning: advanced usage):

bundle common def
{
  classes:
    "_workaround_CFE_2333" -> { "https://tracker.mender.io/browse/CFE-2333" }
      or => { "cfengine_3_7_3", "cfengine_3_8_1", "cfengine_3_8_2" };

    # If the augments_file is parsed from C then we do not need ot do this work
    # from policy
    !(feature_def_json_preparse)|(_workaround_CFE_2333)::
      "have_augments_file" expression => fileexists($(augments_file)), scope => "bundle";
      "have_augments_classes" expression => isvariable("augments[classes]"), scope => "bundle";
      "have_augments_inputs" expression => isvariable("augments[inputs]"), scope => "bundle";

    have_augments_classes.!(feature_def_json_preparse)|(_workaround_CFE_2333)::
      "$(augments_classes_data_keys)"
        expression => classmatch("$(augments[classes][$(augments_classes_data_keys)])"),
        meta => { "augments_class", "derived_from=$(augments_file)" };

  vars:

    !(feature_def_json_preparse)|(_workaround_CFE_2333)::
      "augments_file" string => "$(this.promise_dirname)/../../def.json";

      "defvars" slist => variablesmatching("default:def\..*", "defvar");

    have_augments_file.!feature_def_json_preparse|(_workaround_CFE_2333)::
      "augments" data => readjson($(augments_file), 100k), ifvarclass => "have_augments_file";

      "augments_inputs" slist => getvalues("augments[inputs]");
      "override_vars" slist => getindices("augments[vars]");
      "override_data_$(override_vars)" data => mergedata("augments[vars][$(override_vars)]");
      "override_data_s_$(override_vars)" string => format("%S", "override_data_$(override_vars)");

    any::
      "augments_inputs"
        slist => {},
        ifvarclass => not( isvariable( "augments_inputs" ) ),
        comment => "It's important that we define this list, even if it's empty
                    or we get errors about the list being unresolved.";

    have_augments_classes.!(feature_def_json_preparse)|(_workaround_CFE_2333)::
      "augments_classes_data" data => mergedata("augments[classes]");
      "augments_classes_data_keys" slist => getindices("augments_classes_data");

    any::
      # Begin change

      # Your domain name, for use in access control
      # Note: this default may be inaccurate!
      "domain"
        string => "$(sys.domain)",
        comment => "Define a global domain for all hosts",
        handle => "common_def_vars_domain",
        ifvarclass => not(isvariable("domain"));

      # Mail settings used by body executor control found in controls/cf_execd.cf
      "mailto"
        string => "root@$(def.domain)",
        ifvarclass => not(isvariable("mailto"));

      "mailfrom"
        string => "root@$(sys.uqhost).$(def.domain)",
        ifvarclass => not(isvariable("mailfrom"));

      "smtpserver"
        string => "localhost",
        ifvarclass => not(isvariable("smtpserver"));

      # List here the IP masks that we grant access to on the server

      # Only define here if we are not capable of parsing augments from C
      "acl"
        slist => getvalues("override_data_acl"),
        comment => "JSON-sourced: Define an acl for the machines to be granted accesses",
        handle => "common_def_json_vars_acl",
        ifvarclass => and(isvariable("override_data_acl"), "!feature_def_json_preparse"),
        meta => { "defvar" };

      "acl"
        slist => {
                   # Allow everything in my own domain.

                   # Note that this:
                   # 1. requires def.domain to be correctly set
                   # 2. will cause a DNS lookup for every access
                   # ".*$(def.domain)",

                   # Assume /16 LAN clients to start with
                   "$(sys.policy_hub)/16",

                   # Uncomment below if HA is used
                   #"@(def.policy_servers)"

                   #  "2001:700:700:3.*",
                   #  "217.77.34.18",
                   #  "217.77.34.19",
        },
        comment => "Define an acl for the machines to be granted accesses",
        handle => "common_def_vars_acl",
        ifvarclass => and(not(isvariable("override_data_acl")), not(isvariable("acl"))),
        meta => { "defvar" };

      # Out of the hosts in allowconnects, trust new keys only from the
      # following ones.  This is open by default for bootstrapping.

      # Only define here if we are not capable of parsing augments from C
      "trustkeysfrom"
        slist => getvalues("override_data_trustkeysfrom"),
        comment => "JSON-sourced: define from which machines keys can be trusted",
        ifvarclass => and(isvariable("override_data_trustkeysfrom"), "!feature_def_json_preparse"),
        meta => { "defvar" };

      "trustkeysfrom"
        slist => {
                   # COMMENT THE NEXT LINE OUT AFTER ALL MACHINES HAVE BEEN BOOTSTRAPPED.
                   "0.0.0.0/0", # allow any IP
        },
        comment => "Define from which machines keys can be trusted",
        ifvarclass => and(not(isvariable("override_data_trustkeysfrom")),
                          not(isvariable("trustkeysfrom"))),
        meta => { "defvar" };

      ## List of the hosts not using the latest protocol that we'll accept connections from
      ## (absence of this option or empty list means allow none)
      "control_server_allowlegacyconnects"
        slist => {},
        ifvarclass => not( isvariable( "control_server_allowlegacyconnects" ) );


      # Users authorized to request executions via cf-runagent
      "control_server_allowusers_non_policy_server"
        slist => { "root" },
        ifvarclass => not( isvariable( "control_server_allowusers_non_policy_server" ) );

      "control_server_allowusers_policy_server"
        slist => {},
        ifvarclass => not( isvariable( "control_server_allowusers_policy_server" ) );

      # Executor Controls

      ## Default splaytime to 4 unless it's already defined (via augments)
      "control_executor_splaytime"
        string => "4",
        if => not( isvariable( "control_executor_splaytime" ) ),
        comment => "Splaytime controls the number of minutes hosts execution
                    should be splayed over. This value should be less than the
                    number of minutes between scheduled executions";

      ## Default agent_expireafter to 120 unless it's already defined (via augments)
      "control_executor_agent_expireafter" -> { "ENT-4208" }
        string => "120",
        if => not( isvariable( "control_executor_agent_expireafter" ) ),
        comment => "This controls the number of minutes after no data has been
                    recieved by cf-execd from a cf-agent process before that
                    cf-agent process is killed.";

      ## Default schedule unless it's already defined (via augments)
      "control_executor_schedule_value"
        slist => {
                   "Min00", "Min05", "Min10", "Min15",
                   "Min20", "Min25", "Min30", "Min35",
                   "Min40", "Min45", "Min50", "Min55",
                 },
        if => not( isvariable( control_executor_schedule) ),
        comment => "This variable defines the list of classes that should
                    trigger exec_command if any of them are defined.";

      # schedule cant use a data structure directly, so we must use an
      # intermediary variable to convert it to list
      "control_executor_schedule_value"
        slist => getvalues(control_executor_schedule),
        if =>  not( isvariable( control_executor_schedule_value) ),
        comment => "This variable defines the list of classes that should
                    trigger exec_command if any of them are defined.";


      # Agent Controls

      "control_agent_abortclasses" -> { "ENT-4823" }
        slist => { },
        comment => "The control body has a variable, so a valid list must be defined or the agent will error",
        if => not( isvariable( $(this.promiser) ));

      "control_agent_abortbundleclasses" -> { "ENT-4823" }
        slist => { "abortbundle" },
        comment => "The control body has a variable, so a valid list must be defined or the agent will error",
        if => not( isvariable( $(this.promiser) ));

      "control_agent_default_repository"
        string => ifelse( isvariable( "control_agent_default_repository"),
                          $(control_agent_default_repository),
                          "$(sys.workdir)/backups"),
        if => "mpf_control_agent_default_repository";

      "control_agent_maxconnections"
        int => "30",
        ifvarclass => not( isvariable( "control_agent_maxconnections" ) );

      # Because in some versions of cfengine bundlesequence in body common
      # control does not support does not support iteration over data containers
      # we must first pick out the bundles into a shallow container that we can
      # then get a regular list from using getvalues().

      "tbse" data => mergedata( "def.control_common_bundlesequence_end" );

      # Since we have @(def.bundlesequence_end) in body common control
      # bundlesequence we must have a list variable defined. It can be empty, but it
      # must be defined. If it is not defined the agent will error complaining
      # that '@(def.bundlesequence_end) is not a defined bundle.

      # As noted in CFE-2460 getvalues behaviour varies between versions. 3.7.x
      # getvalues will return an empty list when run on a non existant data
      # container.  On 3.9.1 it does not return an empty list.
      # So we initialize it as an empty list first to be safe.

      "bundlesequence_end" slist => {};
      "bundlesequence_end" slist => getvalues( tbse );

      "control_common_ignore_missing_bundles" -> { "CFE-2773" }
        string => ifelse( strcmp( $(control_common_ignore_missing_bundles), "true" ),
                          "true",
                          "false");

      "control_common_ignore_missing_inputs" -> { "CFE-2773" }
        string => ifelse( strcmp( $(control_common_ignore_missing_inputs), "true" ),
                          "true",
                          "false");

      # Agent controls
@if minimum_version(3.18)
      # TODO When 3.18 is the oldest supported LTS, redact this macro and associated protections
      "control_agent_files_single_copy" -> { "CFE-3622" }
        slist => { },
        if => not( isvariable( "control_agent_files_single_copy" ) ),
        comment => "Default files_single_copy to an empty list if it is not
                    defined. It is expected that users can override the default
                    by setting this value from the augments file (def.json).";
@endif

      "control_server_maxconnections"
        int => "200",
        ifvarclass => not( isvariable( "control_server_maxconnections" ) );

      # Package inventory refresh
      "package_module_query_installed_ifelapsed" -> { "CFE-2771" }
        string => "60", # 1 hour
        if => not( isvariable( $(this.promiser) ));

      "package_module_query_updates_ifelapsed" -> { "CFE-2771" }
        string => "1440", # 1 day
        if => not( isvariable( $(this.promiser) ));

    debian::
      "environment_vars"
        handle => "common_def_vars_environment_vars",
        comment => "Environment variables of the agent process. The values
                    of environment variables are inherited by child commands.",
        slist => {
                   "DEBIAN_FRONTEND=noninteractive",
                  # "APT_LISTBUGS_FRONTEND=none",
                  # "APT_LISTCHANGES_FRONTEND=none",
                 };

      # End change #

    any::
      "dir_masterfiles" string => translatepath("$(sys.masterdir)"),
      comment => "Define masterfiles path",
      handle => "common_def_vars_dir_masterfiles";

      "dir_reports"     string => translatepath("$(sys.workdir)/reports"),
      comment => "Define reports path",
      handle => "common_def_vars_dir_reports";

      "dir_bin"         string => translatepath("$(sys.bindir)"),
      comment => "Define binary path",
      handle => "common_def_vars_dir_bin";

      "dir_data"
        string => ifelse( isvariable( "def.dir_data"),
                          $(def.dir_data),
                          "$(sys.workdir)/data"),
      comment => "Define data path",
      handle => "common_def_vars_dir_data";

      "dir_modules"     string => translatepath("$(sys.workdir)/modules"),
      comment => "Define modules path",
      handle => "common_def_vars_dir_modules";

      "dir_plugins"     string => translatepath("$(sys.workdir)/plugins"),
      comment => "Define plugins path",
      handle => "common_def_vars_dir_plugins";

      "dir_templates"
        string => ifelse( isvariable( "def.dir_templates"),
                          $(def.dir_templates),
                          "$(sys.workdir)/templates"),
      comment => "Define templates path",
      handle => "common_def_vars_dir_templates";


      "cf_apache_user" string => "cfapache",
      comment => "User that CFEngine Enterprise webserver runs as",
      handle => "common_def_vars_cf_cfapache_user";

      "cf_apache_group" string => "cfapache",
      comment => "Group that CFEngine Enterprise webserver runs as",
      handle => "common_def_vars_cf_cfapache_group";

    policy_server|am_policy_hub::

      # Only hubs serve software updates

      "dir_master_software_updates" -> { "ENT-4953" }
        string => "$(sys.workdir)/master_software_updates",
        handle => "common_def_vars_dir_serve_master_software_updates",
        comment => "Path where software updates are served from the policy hub.
        This variable is overridable via augments as
        vars.dir_master_software_updates. All remote agents request this path
        via the master_software_updates shortcut.",
        if => not( isvariable( "def.dir_master_software_updates" ));

    solaris::
      "cf_runagent_shell"
        string  => "/usr/bin/sh",
        comment => "Define path to shell used by cf-runagent",
        handle  => "common_def_vars_solaris_cf_runagent_shell";

    !(windows|solaris)::
      "cf_runagent_shell"
        string  => "/bin/sh",
        comment => "Define path to shell used by cf-runagent",
        handle  => "common_def_vars_cf_runagent_shell";

    any::
      "base_log_files" slist =>
      {
        "$(sys.workdir)/cf3.$(sys.uqhost).runlog",
        "$(sys.workdir)/promise_summary.log",
      };

      "enterprise_log_files" slist =>
      {
        "$(sys.workdir)/cf_notkept.log",
        "$(sys.workdir)/cf_repair.log",
        "$(sys.workdir)/state/cf_value.log",
        "$(sys.workdir)/outputs/dc-scripts.log",
        "$(sys.workdir)/state/promise_log.jsonl",
        "$(sys.workdir)/state/classes.jsonl",
      };

      "hub_log_files" slist =>
      {
        "$(sys.workdir)/httpd/logs/access_log", # Mission Portal
        "$(sys.workdir)/httpd/logs/error_log", # Mission Portal
        "/var/log/postgresql.log",
      };

      "max_client_history_size" -> { "cf-hub", "CFEngine Enterprise" }
        int => "50M",
        unless => isvariable(max_client_history_size),
        comment => "The threshold of report diffs which will trigger purging of
                    diff files.";

    enterprise.!am_policy_hub::
      # CFEngine's own log files
      "cfe_log_files" slist => { @(base_log_files), @(enterprise_log_files) };

    enterprise.am_policy_hub::
      # CFEngine's own log files
      "cfe_log_files" slist => { @(base_log_files), @(enterprise_log_files), @(hub_log_files) };

    !enterprise::
      # CFEngine's own log files
      "cfe_log_files" slist => { @(base_log_files) };

  # Directories where logs are rotated and old files need to be purged.

    any::
      "log_dir[outputs]" string => "$(sys.workdir)/outputs";
      "log_dir[reports]" string => "$(sys.workdir)/reports";

    # TODO ENT-6845 - move package module logs to $(sys.workdir)/log/something
    windows::
      "log_dir[package_logs]" string => "$(const.dirsep)cfengine_package_logs";

    enterprise.am_policy_hub::
      "log_dir[application]" string => "$(sys.workdir)/httpd/htdocs/application/logs";

    any::
      "cfe_log_dirs" slist => getvalues( log_dir );

  # MPF controls

      "mpf_log_dir_retention"
        string => "30",
        if => not( isvariable ( mpf_log_dir_retention ) ),
        comment => "The default log file retention in cfe_log_dirs is 30 days
                    unless it's already been defined (augments).";

      "mpf_log_file_retention"
        string => "10",
        if => not( isvariable( mpf_log_file_retention) ),
        comment => "This is the number of rotated versions of mpf log files to
                    save";

      "mpf_log_file_max_size"
        string => "1M",
        if => not( isvariable( mpf_log_file_max_size) ),
        comment => "When individual mpf log files reach this size they should be
                    rotated so that we don't fill up the disk";

      "purge_scheduled_reports_older_than_days" -> { "ENT-4404" }
        string => "30",
        if => not( isvariable( purge_scheduled_reports_older_than_days ) ),
        comment => "This controls the maximum age of artifacts generated by the
                    asynchronous query API and scheduled reports.";

  # Enterprise HA Related configuration
  # enable_cfengine_enterprise_hub_ha is defined below
  # Disabled by default

    enable_cfengine_enterprise_hub_ha::
      "standby_servers" slist => filter("$(sys.policy_hub)", "ha_def.ips", false, true, 10);
      "policy_servers" slist => { "$(sys.policy_hub)", "@(standby_servers)" };

    !enable_cfengine_enterprise_hub_ha::
      "policy_servers" slist => {"$(sys.policy_hub)"};

    enterprise_edition.policy_server::
      "control_hub_exclude_hosts"
        slist => { "" },
        unless => isvariable(control_hub_exclude_hosts);

      "control_hub_hub_schedule"
        comment => "By default Enterprise hubs initiate pull collection once every 5 minutes.",
        slist  => { "Min00", "Min05", "Min10", "Min15", "Min20",
                    "Min25", "Min30", "Min35", "Min40", "Min45",
                    "Min50", "Min55" },
        unless => isvariable(control_hub_hub_schedule);

      "control_hub_port"
        comment => "cf-hub performs pull collection on port 5308, unless
                    overridden by augments",
        string => "5308",
        unless => isvariable(control_hub_port);

      "control_hub_client_history_timeout"
        comment => "cf-hub instructs clients to discard reports older than this
                    many hours to avoid a condition where a build up of reports
                    causes a client to never be fully collected from",
        string => "6",
        unless => isvariable(control_hub_port);


      "mpf_access_rules_collect_calls_admit_ips"
        slist => { @(def.acl) },
        unless => isvariable(mpf_access_rules_collect_calls_admit_ips);

    enterprise_edition.client_initiated_reporting_enabled::
      "control_server_call_collect_interval"
        string => "5",
        unless => isvariable(control_server_call_collect_interval);

      "control_server_collect_window" -> { "ENT-4102" }
        string => "30",
        unless => isvariable(control_server_collect_window);

    enterprise_edition.policy_server::
      "default_data_select_host_monitoring_include"
        comment => "Most people have monitoring systems, so instead of collecting data people won't use we save the work unless its requested.",
        slist => { },
        unless => isvariable( default_data_select_host_monitoring_include );

      "default_data_select_policy_hub_monitoring_include"
        comment => "Collect all the monitoring data from the hub itself. It can be useful in diagnostics.",
        slist => { ".*" },
        unless => isvariable( default_data_select_policy_hub_monitoring_include );

  classes:

      "_have_control_agent_files_single_copy" -> { "CFE-3622"}
        expression => isvariable( "def.control_agent_files_single_copy" );
      "cfengine_recommendations_enabled"
        expression => "!cfengine_recommendations_disabled";

      ### Enable special features policies. Set to "any" to enable.

      # Auto-load files in "services/autorun" and run bundles tagged "autorun".
      # Disabled by default!
      "services_autorun" -> { "jira:CFE-2135" }
        comment => "This class enables the automatic parsing running of bundles
                    tagged with 'autorun'. Evaluation limitations require that
                    this class is set at the beginning of the agent run, so it
                    must be defined in the augments file (def.json), or as an
                    option to the agent with --define or -D. Changing the
                    expression here will *NOT* work correctly. Setting the class
                    here will result in an error due to the autorun bundle not
                    being found.",
        expression => "!any";

      # Internal CFEngine log files rotation
      "cfengine_internal_rotate_logs" expression => "any";

      # Enable or disable agent email output (also see mailto, mailfrom and
      # smtpserver)
      "cfengine_internal_agent_email" expression => "any";
      "cfengine_internal_disable_agent_email" expression => "!any";



      # Enable or disable external watchdog to ensure cf-execd is running
      "cfe_internal_core_watchdog_enabled" expression => "!any";
      "cfe_internal_core_watchdog_disabled" expression => "!any";

      # Transfer policies and binaries with encryption
      # you can also request it from the command line with
      # -Dcfengine_internal_encrypt_transfers

      # NOTE THAT THIS CLASS ALSO NEEDS TO BE SET IN update.cf

      "cfengine_internal_encrypt_transfers" expression => "!any";

      # Purge policies that don't exist on the server side.
      # you can also request it from the command line with
      # -Dcfengine_internal_purge_policies

      # NOTE THAT THIS CLASS ALSO NEEDS TO BE SET IN update.cf

      "cfengine_internal_purge_policies" expression => "!any";

      # Preserve permissions of the policy server's masterfiles.
      # you can also request it from the command line with
      # -Dcfengine_internal_preserve_permissions

      # NOTE THAT THIS CLASS ALSO NEEDS TO BE SET IN update.cf

      "cfengine_internal_preserve_permissions" expression => "!any";

      # Allow the hub to edit sudoers in order for the Apache user to
      # run passwordless sudo cf-runagent. Enable this if you want the
      # Mission Portal to be able to troubleshoot failed Design Center
      # sketch activations on a host.
      "cfengine_internal_sudoers_editing_enable" expression => "!any";

      # Class defining which versions of cfengine are (not) supported
      # by this policy version.
      # Also note that this policy will only be run on enterprise policy_server
      "postgresql_maintenance_supported"
        expression => "(policy_server.enterprise.!enable_cfengine_enterprise_hub_ha)|(policy_server.enterprise.enable_cfengine_enterprise_hub_ha.hub_active)";

      # This class is for PosgreSQL maintenance
      # pre-defined to every Sunday at 2 a.m.
      # This can be changed later on.
      "postgresql_full_maintenance" expression => "postgresql_maintenance_supported.Sunday.Hr02.Min00_05";

      # Run vacuum job on database
      # pre-defined to every night except Sunday when full cleanup is executed.
      "postgresql_vacuum" expression => "postgresql_maintenance_supported.!Sunday.Hr02.Min00_05";

      # Enable CFEngine Enterprise HA Policy
      "enable_cfengine_enterprise_hub_ha" expression => "!any";
      #"enable_cfengine_enterprise_hub_ha" expression => "enterprise_edition";

      # Enable failover to node which is outside cluster
      #"failover_to_replication_node_enabled" expression => "enterprise_edition";

      # Enable cleanup of agent report diffs when they exceed
      # `def.max_client_history_size`
      "enable_cfe_internal_cleanup_agent_reports" -> { "cf-hub", "CFEngine Enterprise" }
        expression => "enterprise_edition",
        comment => "If reports are not collected for an extended period of time
                    the disk may fill up or cause additional collection
                    issues.";

      # Enable paths to POSIX tools instead of native tools when possible.
      "mpf_stdlib_use_posix_utils" expression => "any";

      # TODO Remove from MPF after 3.12 EOL
      "cfredis_in_enterprise" -> { "ENT-2797" }
        or => { "cfengine_3_7", "cfengine_3_8", "cfengine_3_9", "cfengine_3_10", "cfengine_3_11" };
      "cfconsumer_in_enterprise"  -> { "ENT-2797" }
        or => { "cfengine_3_7", "cfengine_3_8", "cfengine_3_9", "cfengine_3_10", "cfengine_3_11" };

  reports:
    DEBUG|DEBUG_def::
      "DEBUG: $(this.bundle)";

      "$(const.t) def.json was found at $(augments_file)"
        ifvarclass => fileexists( $(augments_file) );

      "$(const.t) override request $(override_vars) to '$(override_data_s_$(override_vars))'; new value '$($(override_vars))'"
      ifvarclass => isvariable("override_data_$(override_vars)");

      "$(const.t) defined class '$(augments_classes_data_keys)' because of classmatch('$(augments[classes][$(augments_classes_data_keys)])')"
      ifvarclass => "$(augments_classes_data_keys)";

      "$(const.t) $(defvars) = $($(defvars))";
      "DEBUG $(this.bundle): Agent parsed augments_file"
        ifvarclass => "have_augments_file.feature_def_json_preparse";
      "DEBUG $(this.bundle): Policy parsed augments_file"
        ifvarclass => "have_augments_file.!feature_def_json_preparse";
}

bundle common inventory_control
{
  vars:
      "lldpctl_exec" string => ifelse(fileexists("/usr/sbin/lldpctl"), "/usr/sbin/lldpctl",
                                     fileexists("/usr/local/bin/lldpctl"), "/usr/local/bin/lldpctl",
                                     "/usr/sbin/lldpctl");

      "lldpctl_json" string => "$(lldpctl_exec) -f json",
        unless => isvariable("def.lldpctl_json");
      "lldpctl_json" string => "$(def.lldpctl_json)",
        if => isvariable("def.lldpctl_json");

      "lsb_exec" string => "/usr/bin/lsb_release";
      "mtab" string => "/etc/mtab";
      "proc" string => "/proc";

  vars:

    freebsd::

      "dmidecoder" string => "/usr/local/sbin/dmidecode";

    !freebsd::

      "dmidecoder" string => "/usr/sbin/dmidecode";

  classes:
      # setting this disables all the inventory modules except package_refresh
      "disable_inventory" expression => "!any";

      # disable specific inventory modules below

      # by default disable the LSB inventory if the general inventory
      # is disabled or the binary is missing.  Note that the LSB
      # binary is typically not very fast.
      "disable_inventory_lsb" expression => "disable_inventory";
      "disable_inventory_lsb" not => fileexists($(lsb_exec));

      # by default disable the dmidecode inventory if the general
      # inventory is disabled or the binary does not exist.  Note that
      # typically this is a very fast binary.
      "disable_inventory_dmidecode" expression => "disable_inventory";
      "disable_inventory_dmidecode" not => fileexists($(dmidecoder));

      # by default disable the LLDP inventory if the general inventory
      # is disabled or the binary does not exist.  Note that typically
      # this is a reasonably fast binary but still may require network
      # I/O.
      "disable_inventory_LLDP" expression => "disable_inventory";
      "disable_inventory_LLDP" not => fileexists($(lldpctl_exec));

      # by default run the package inventory refresh every time, even
      # if disable_inventory is set
      "disable_inventory_package_refresh" expression => "!any";

      # by default disable the mtab inventory if the general inventory
      # is disabled or $(mtab) is missing.  Note that this is very
      # fast.
      "disable_inventory_mtab" expression => "disable_inventory";
      "disable_inventory_mtab" not => fileexists($(mtab));

      # by default disable the fstab inventory if the general
      # inventory is disabled or $(sys.fstab) is missing.  Note that
      # this is very fast.
      "disable_inventory_fstab" expression => "disable_inventory";
      "disable_inventory_fstab" not => fileexists($(sys.fstab));

      # by default disable the proc inventory if the general
      # inventory is disabled or /proc is missing.  Note that
      # this is typically fast.
      "disable_inventory_proc" expression => "disable_inventory|freebsd";
      "disable_inventory_proc" not => isdir($(proc));

      # by default don't run the CMDB integration every time, even if
      # disable_inventory is not set
      "disable_inventory_cmdb" expression => "any";

  reports:
    verbose_mode.disable_inventory::
      "$(this.bundle): All inventory modules disabled";
    verbose_mode.!disable_inventory_lsb::
      "$(this.bundle): LSB module enabled";
    verbose_mode.!disable_inventory_dmidecode::
      "$(this.bundle): dmidecode module enabled";
    verbose_mode.!disable_inventory_LLDP::
      "$(this.bundle): LLDP module enabled";
    verbose_mode.!disable_inventory_mtab::
      "$(this.bundle): mtab module enabled";
    verbose_mode.!disable_inventory_fstab::
      "$(this.bundle): fstab module enabled";
    verbose_mode.!disable_inventory_proc::
      "$(this.bundle): proc module enabled";
    verbose_mode.!disable_inventory_package_refresh::
      "$(this.bundle): package_refresh module enabled";
    verbose_mode.!disable_inventory_cmdb::
      "$(this.bundle): CMDB module enabled";

    DEBUG|DEBUG_def::

      "Executor Schedule: $(def.control_executor_schedule_value)";
}
augments_file (variable)

This variable defines the path to a JSON file used to "augment" the current definitions.

The augments_file is intended to ease policy framework upgrades by providing a standard location for site specific settings to be defined. Currently the augments file supports defining additional inputs and classes as well as overriding variables matching defvars.

By default the augments_file is expected to exist in the root of your policy. Should you want to deliver different augments files to different clients, you may consider pointing this to a file outside of the masterfiles tree that is downloaded by the individual clients.

defvars (variable)

This variable defines the list of variables that are allowed to be overridden by the augments_file.

augments (variable)

This variable contains the augments data as loaded from the augments_file. By default this is limited to 100k. If your augments_file is larger than 100k you will want to adjust this limit.

Here is an example augments_file:

{
    "classes":
    {
        "my_apache": [ "server1", "server2", "redhat.*" ],
        "my_other_apache": [ "server[34]", "debian.*" ],
        "my_filehost": [ "server3" ],
        "my_gateway": [ "server3" ],
        "my_yum_role": [ "redhat.*" ],
        "my_redhat_role": [ "redhat.*" ],
        "my_apt_role": [ "debian.*" ],
        "my_debian_role": [ "debian.*" ],
        "services_autorun": [ "any" ],
        "cfengine_internal_disable_agent_email": [ "enterprise_edition" ],
        "cfe_internal_core_watchdog_enabled": [ "any" ],
        "mpf_control_agent_default_repository": [ "any" ]
    },

    "inputs": [ "$(sys.libdir)/bundles.cf" ],
    "vars": {

        "control_agent_default_repository": "/var/cfengine/edit_backups",
        "acl": [ ".*$(def.domain)", "$(sys.policy_hub)/16" ],
        "control_agent_maxconnections": 100,
        "control_common_bundlesequence_end": [ "my_bundle", "my_other_bundle" ],
        "control_agent_files_single_copy": [ "/etc/.*", "/tmp/special_file" ],
        "input_name_patterns": [ ".*\\.cf",".*\\.dat",".*\\.txt", ".*\\.conf", ".*\\.mustache",
                                 ".*\\.sh", ".*\\.pl", ".*\\.py", ".*\\.rb",
                                 "cf_promises_release_id", ".*\\.json", ".*\\.yaml", ".*\\.js" ],

        "dir_templates": "/var/cfengine/templates_tests"
    }
}
domain (variable)

Set your domain to the right value. By default it's used for mail and to deduce your file access ACLs.

mailto (variable)

This variable defines the email address that agent run output is sent to.

mailfrom (variable)

This variale defines the email address that emails containing agent run output come from.

smtpserver (variable)

This variale defines the smtp server to use when sending agent emails.

acl (variable)

The acl is crucial. This is used by every host, not just the policy server. Make sure you only allow hosts you want to allow.

trustkeysfrom (variable)

trustkeysfrom tells the policy server from which IPs it should accept connections even if the host's key is unknown, trusting it at connect time. This is only useful to be open during for bootstrapping these hosts. As the comments say, empty it after your hosts have been bootstrapped to avoid unpleasant surprises.

services_autorun (class)

Off by default.

When enabled files in services/autorun suffixed with .cf are automatically added to inputs and bundles that are tagged with autorun are automatically actuated (as a policy level feature).

This class needs to be defined from the beginning of policy execution and should be defined using an augments file. Here is an example:

{
  "classes":{
    "services_autorun":[
      "any"
    ]
  }
}

Note: It is not sufficient to define this class within the policy itself in order for files to be discovered and added to inputs appropriately.

To tag a bundle with autorun simply define tags as a meta slist with a value that inclused autorun. Here's a simple example of such a bundle in services/autorun/hello.cf:

bundle agent hello_world_autorun
{
  meta:
      "tags" slist => { "autorun" };

  reports:
    verbose_mode::
      "$(this.bundle): Hello, this is an automatically loaded bundle";
}
cfengine_internal_rotate_logs (class)

On by default.

Rotates CFEngine's own logs. Here is the cfe_internal_log_rotation bundle implementation:

bundle agent cfe_internal_log_rotation
{
  methods:

    cfengine_internal_rotate_logs::
      # CFEngine generates internal log files that need to be rotated.
      # Have a look at def.cf to enable rotation of these files
      "Rotate CFEngine log files"
        handle    => "cfe_internal_log_rotation_rotate_log_files",
        usebundle => logrotate( @(def.cfe_log_files), $(def.mpf_log_file_max_size), $(def.mpf_log_file_retention) ),
        comment => "To keep the disk from getting to full we want to rotate log
                    files when they reach $(def.mpf_log_file_max_size) in size.
                    So that we have some history , we keep
                    $(def.mpf_log_file_retention) versions.";

      "Prune old log files"
        handle    => "cfe_internal_log_rotation_prune_log_dirs",
        usebundle => prunedir( @(def.cfe_log_dirs), $(def.mpf_log_dir_retention) ),
        comment => "Scheduled activities like agent runs and reports can create
                    log files or reports that stack up over time. So that we
                    don't fill the disk, but have some historical information
                    available locally we purge log files older than
                    $(def.mpf_log_dir_retention) days.";

  reports:

    DEBUG|DEBUG_cfe_internal_log_rotation::
      "DEBUG $(this.bundle): Check CFEngine log file for rotation '$(def.cfe_log_files)'";
      "DEBUG $(this.bundle): Check CFEngine log directory for old logs in '$(def.cfe_log_dirs)' older than '$(def.mpf_log_dir_retention)' days";
}
cfengine_internal_agent_email (class)

On by default.

This class enables agent email output from cf-execd.

cfengine_internal_encrypt_transfers (class)

Duplicate of the one in update.cf. They should be set in unison or you may get unexpected behavior.

cfengine_internal_purge_policies (class)

Duplicate of the one in update.cf. They should be set in unison or you may get unexpected behavior.

cfengine_internal_preserve_permissions (class)

Duplicate of the one in update.cf. They should be set in unison or you may get unexpected behavior.

cfengine_internal_sudoers_editing_enable (class)

Off by default. Only used on the CFEngine Enterprise hub.

Turn this on (set to any) to allow the hub to edit sudoers in order for the Apache user to run passwordless sudo cf-runagent (part of Mission Portal troubleshooting).

postgresql_maintenance_supported (class)

On by default only for CFEngine Enterprise Hubs.

This class enables maintaince routines for the database used in CFEngine Enterprise.

postgresql_full_maintenance (class)

On by default only on Sundays at 2am when postgresql_maintenance_supported is defined.

Set this class accordingly if you want to schedule database maintenance operations at a different time.

postgresql_vacuum (class)

On by default at 2am when postgresql_maintenance_supported is defined except for Sundays.

Set this class accordingly if you want to schedule database maintenance operations at a different time.

enable_cfengine_enterprise_hub_ha (class)

Off by default.

Set this class when you want to enable the CFEngine Enterprise HA policies.

enable_cfe_internal_cleanup_agent_reports (class)

Off by default for core. On by default for CFEngine Enteprise clients.

This class enables policy that cleans up report diffs when they exceed def.maxclient_history_size.

@(cfengine_enterprise_hub_ha.classification_bundles) (bundle)

Bundles related to classification for CFEngine Enterprise HA.

services_autorun (bundle)

This bundle loads policies found in services/autorun that are tagged for autorun.

See services_autorun

@(services_autorun.bundles) (bundle)

This activates bundles found by services_autorun.

cfe_internal_management (bundle)

This bundle activates policy related to CFEngine itself. For example rotation of logs generated by the agent.

main (bundle)

This bundle is defined as bundle agent main in services/main.cf, it is the main entry into custom policy.

@(cfengine_enterprise_hub_ha.management_bundles) (bundle)

These bundles activate policy that manages HA in CFEngine Enterprise.

inputs

In order to find bundles, CFEngine needs to know where to look. This list defines what files are needed. Note there are several dynamic entries here, coming from other bundles. CFEngine will keep evaluating the inputs and bundlesequence until all the bundles are found and resolved.

Make sure to add any of your own services files here if you don't use the services_autorun facility, to ensure the bundles in them are found.

failsafe.cf

The failsafe.cf file ensures that your system can survive errors and can upgrade gracefully to new versions even when mistakes are made. It's literally a failsafe if promises.cf and update.cf should fail.

This file is generated during the bootstrapping process, and should normally never be changed. The only job of failsafe.cf is to execute the update bundle in a “standalone” context should there be a syntax error somewhere in the main set of promises. In this way, if a client machine's policies are ever corrupted after downloading erroneous policy from a server, that client will have a failsafe method for downloading a corrected policy once it becomes available on the server. Note that by “corrupted” and “erroneous” we typically mean “broken via administrator error” - mistakes happen, and the failsafe.cf file is CFEngine's way of being prepared for that eventuality.

If you ever change failsafe.cf (or when you upgrade CFEngine), make sure the old and the new CFEngine can successfully parse and execute this file. If not, you risk losing control of your system (that is, if CFEngine cannot successfully execute this policy file, it has no failsafe/fallback mechanism for distributing new policy files).

Some general rules (but again, note you can completely break your CFEngine installation by editing failsafe.cf):

  • Upgrade the software first, then add new features to the configuration.
  • Never use advanced features in the failsafe or update file.
  • Avoid using library code (including any bodies from stdlib.cf or the files it includes). Copy/paste any bodies you need using a unique name that does not collide with a name in library (we recommend simply adding the prefix “u_”). This may mean that you create duplicate functionality, but that is okay in this case to ensure a 100% functioning standalone update process). The promises which manage the update process should not have any dependencies on any other files.

CFEngine will fail-over to the failsafe.cf configuration if it is unable to read or parse the contents successfully. That means that any syntax errors you introduce (or any new features you utilize in a configuration) will cause a fail-over, because the parser will not be able to interpret the policy. If the failover is due to the use of new features, they will not parse until the software itself has been updated (so we recommend that you always update CFEngine before updating policy to use new features). If you accidentally cause a bad (i.e., unparseable) policy to be distributed to client machines, the failsafe.cf policy on those machines will run (and will eventually download a working policy, once you fix it on the policy host).

Further structure

  • cfe_internal: internal CFEngine policies you shouldn't modify or you will get unexpected behavior

  • controls: configuration of components, e.g. the cf-agent or cf-serverd, beyond what def.cf can offer. You'll see 3.6 which is for backwards compatibility and used only by 3.6 agents. 3.7+ agents will use the library files directly inside of this directory.

  • def.cf: defaults you can and should configure, see above

  • inventory: inventory modules (loaded before anything else to discover facts about the system) live here; see above

  • lib: main library directory. You'll see 3.5 and 3.6 and 3.7 under it. These are the supported versions for masterfiles backwards compatibility.

  • promises.cf: main policy, you will need to configure this, see above

  • services: your site's policies go here

  • services_autorun: see above

  • update and update.cf: functionality for updating inputs and CFEngine itself, see above. You only modify files under update if you know the impact of what you are doing or you may get unexpected behavior.

cf_promises_validated

Several CFEngine components that read policy (e.g. cf-agent, cf-execd, cf-serverd) run cf-promises to validate the syntax of their input files before actually running the policy. To illustrate this, if cf-promises runs every 5 minutes then there will be 12 checks occurring every hour, 24 hours a day, 7 days a week -- a total of 2016 possible validation checks. Each of those individual validation sessions can take some number of seconds to perform depending on the system, scale, circumstances and configuration.

Starting with CFEngine 3.1.2, the outcome of every run of cf-promises was cached, which lets agents skip the validation of input files that have not changed since the previous run.

Starting with CFEngine 3.6, outcome on both hosts and hubs is stored in the file $(sys.workdir)/masterfiles/cf_promises_validated (usually sys.workdir is /var/cfengine). The file can be created by cf-agent after it has successfully verified the policy with cf-promises. The file can also be created by a user with cf-promises -T DIRECTORY which is useful for validating an entire directory.

When the hash content of any file under WORKDIR/inputs changes, and validates to be syntactically correct, then a timestamp in cf_promises_validated is updated. If not, the run of cf-promises is skipped and, at the same time, the cf-execd, cf-serverd and cf-monitord daemons will not reload the policy unless cf_promises_validated has an updated timestamp, which cf-agent will normally take care of.

In the default installation, the masterfiles are populated automatically on the policy server and you can even auto-deploy them from a version control system.

You should configure the masterfiles as described above. Leaving them at their default settings may expose your masterfiles or worse, especially the cf-serverd ACL settings. If you are not sure of the terms used below or what it all means, come back to this page after you've learned about writing policy and the CFEngine syntax.

CFEngine 3 Inventory Modules

The CFEngine 3 inventory modules are pieces of CFEngine policy that are loaded and used by the promises.cf mechanism in order to inventory the system.

CFEngine Enterprise has specific functionality to show and use inventory data, but users of the Community Version can use them as well locally on each host.

How It Works

The inventory modules are called in promises.cf:

body common control
{
      bundlesequence => {
                        # Common bundle first (Best Practice)
                          inventory_control,
                          @(inventory.bundles),
                          ...

As you see, this calls the inventory_control bundle, and then each bundle in the list inventory.bundles. That list is built in the top-level common inventory bundle, which will load the right things for some common cases. The any.cf inventory module is always loaded; the rest are loaded if they are appropriate for the platform. For instance, Debian systems will load debian.cf and linux.cf and lsb.cf but may load others as needed.

The effect for users is that the right inventory modules will be loaded and evaluated.

The inventory_control bundle lives in def.cf and defines what inventory modules should be disabled. You can simply set disable_inventory to avoid the whole system, or you can look for the disable_inventory_xyz class to disable module xyz.

Any inventory module works the same way, by doing some discovery work and then tagging its classes and variables with the report or inventory tags. For example:

  vars:
      "ports" slist => { @(mon.listening_ports) },
      meta => { "inventory", "attribute_name=Ports listening" };

This defines a reported attribute "Ports listening" which contains a list of strings representing the listening ports. More on this in a second.

Your Very Own Inventory Module

The good news is, writing an inventory module is incredibly easy.

They are just CFEngine bundles. You can see a simple one that collects the listening ports in any.cf:

bundle agent cfe_autorun_inventory_listening_ports
# @brief Inventory the listening ports
#
# This bundle uses `mon.listening_ports` and is always enabled by
# default, as it runs instantly and has no side effects.
{
  vars:
      "ports" slist => { @(mon.listening_ports) },
      meta => { "inventory", "attribute_name=Ports listening" };
}

Well, the slist copy is a CFEngine detail (we get the listening ports from the monitoring daemon), so just assume that the data is correct. What's important is the second line that starts with meta. That defines metadata for the promise that CFEngine will use to determine that this data is indeed inventory data and should be reported to the CFEngine Enterprise Hub.

That's it. Really. The comments are optional but nice to have. You don't have to put your new bundle in a file under the inventory directory, either. The variables and classes can be declared anywhere as long as they have the right tags. So you can use the services directory or whatever else makes sense to you.

CFEngine Enterprise vs. Community

In CFEngine Enterprise, the reported data is aggregated in the hub and reported across the whole host population.

In CFEngine Community, users can use the classesmatching() and variablesmatching() functions to collect all the inventory variables and classes and report them in other ways.

Implementation Best Practice for CFEngine Enterprise

It is important that inventory variables and classes are continually defined. Only inventory variables and classes defined during the last reported run are available for use by the inventory reporting interface.

Inventory items that change frequently can create a burden on the Enterprise reporting infrastructure. Generally, inventory attributes should change infrequently.

If you wish to inventory attributes that frequently change or are expensive to discover consider implementing a sample interval and caching mechanism.

What Modules Are Available?

As soon as you use the promises.cf provided in the parent directory, quite a few inventory modules will be enabled (if appropriate for your system). Here's the list of modules and what they provide. Note they are all enabled by code in def.cf as explained above.

LSB

  • lives in: lsb.cf
  • applies to: LSB systems (most Linux distributions, basically)
  • runs: lsb_release -a
  • sample data:
Distributor ID: Ubuntu
Description:    Ubuntu 14.04 LTS
Release:    14.04
Codename:   trusty
  • provides:
    • classes lsb_$(os), lsb_$(os)_$(release), lsb_$(os)_$(codename)
    • variables: inventory_lsb.os (Distributor ID), inventory_lsb.codename, inventory_lsb.release, inventory_lsb.flavor, inventory_lsb.description
  • implementation:
bundle agent inventory_lsb
{
  classes:
      "have_lsb" expression => fileexists($(lsb_exec));

      "_inventory_lsb_found" expression => regcmp("^[1-9][0-9]*$", $(dim)),
                                  scope => "namespace";

    _inventory_lsb_found::
      "lsb_$(os)" expression => "any",
      comment => "LSB Distributor ID",
      depends_on => { "inventory_lsb_os" },
      scope => "namespace",
      meta => { "inventory", "attribute_name=none" };

      "lsb_$(os)_$(release)" expression => "any",
      comment => "LSB Distributor ID and Release",
      depends_on => { "inventory_lsb_os", "inventory_lsb_release" },
      scope => "namespace",
      meta => { "inventory", "attribute_name=none" };

      "lsb_$(os)_$(codename)" expression => "any",
      comment => "LSB Distributor ID and Codename",
      depends_on => { "inventory_lsb_os", "inventory_lsb_codename" },
      scope => "namespace",
      meta => { "inventory", "attribute_name=none" };

  vars:
      "lsb_exec" string => "$(inventory_control.lsb_exec)";

    have_lsb::
      "data" string => execresult("$(lsb_exec) -a", "noshell");
      "dim" int => parsestringarray(
                                     "lsb",
                                     $(data),
                                     "\s*#[^\n]*",
                                     "\s*:\s+",
                                     "15",
                                     "4095"
      );

    _inventory_lsb_found::
      "lsb_keys" slist => getindices("lsb");

      "os" string => canonify("$(lsb[Distributor ID][1])"),
      handle => "inventory_lsb_os",
      comment => "LSB-provided OS name",
      meta => { "inventory", "attribute_name=none" };

      "codename" string => canonify("$(lsb[Codename][1])"),
      handle => "inventory_lsb_codename",
      comment => "LSB-provided OS code name",
      meta => { "inventory", "attribute_name=none" };

      "release" string => "$(lsb[Release][1])",
      handle => "inventory_lsb_release",
      comment => "LSB-provided OS release",
      meta => { "inventory", "attribute_name=none" };

      "flavor" string => canonify("$(lsb[Distributor ID][1])_$(lsb[Release][1])"),
      handle => "inventory_lsb_flavor",
      comment => "LSB-provided OS flavor",
      meta => { "inventory", "attribute_name=none" };

      "description" string => "$(lsb[Description][1])",
      handle => "inventory_lsb_description",
      comment => "LSB-provided OS description",
      meta => { "inventory", "attribute_name=none" };

  reports:
    (DEBUG|DEBUG_inventory_lsb)._inventory_lsb_found::
      "DEBUG $(this.bundle): OS = $(os), codename = $(codename), release = $(release), flavor = $(flavor), description = $(description)";
      "DEBUG $(this.bundle): got $(dim) LSB keys";
      "DEBUG $(this.bundle): prepared LSB key $(lsb_keys) = '$(lsb[$(lsb_keys)][1])'";
    (DEBUG|DEBUG_inventory_lsb).!_inventory_lsb_found::
      "DEBUG $(this.bundle): LSB inventory not found";
}
  • sample output:
% cf-agent -KI -binventory_control,inventory_lsb

R: inventory_lsb: OS = Ubuntu, codename = trusty, release = 14.04, flavor = Ubuntu_14_04, description = Ubuntu 14.04 LTS

SUSE

  • lives in: suse.cf
  • applies to: SUSE Linux
  • provides classes: suse_pure and suse_derived
  • implementation:
bundle common inventory_suse
{
  classes:
      "suse_pure" expression => "(sles|sled).!opensuse",
      comment => "pure SUSE",
      meta => { "inventory", "attribute_name=none" };

      "suse_derived" expression => "opensuse.!suse_pure",
      comment => "derived from SUSE",
      meta => { "inventory", "attribute_name=none" };
}

Debian

  • lives in: debian.cf
  • applies to: Debian and its derivatives
  • provides:
    • variables: inventory_debian.mint_release and inventory_debian.mint_codename
    • classes: debian_pure, debian_derived, linuxmint, lmde, linuxmint_$(mint_release), linuxmint_$(mint_codename), $(mint_codename)
  • implementation:
bundle common inventory_debian
{
  vars:
    has_lsb_release::
      "lsb_release_info" string => readfile("/etc/lsb-release","256"),
      comment => "Read more OS info" ;

    has_etc_linuxmint_info::
      "linuxmint_info"  string => readfile("/etc/linuxmint/info","1024"),
      comment => "Read Linux Mint specific info" ;

      "lm_info_count"
      int => parsestringarray("mint_info", # array to populate
                              "$(linuxmint_info)", # data to parse
                              "\s*#[^\n]*",        # comments
                              "=",                 # split
                              100,                 # maxentries
                              2048) ;              # maxbytes

      "mint_release" string  => "$(mint_info[RELEASE][1])" ;
      "mint_codename" string => "$(mint_info[CODENAME][1])" ;

  classes:
    any::
      "debian_derived_evaluated"
      scope => "bundle",
      or => { "has_os_release", "has_lsb_release", "has_etc_linuxmint_info" } ;

      "linuxmint"
      expression => "has_etc_linuxmint_info",
      comment => "this is a Linux Mint system, of some sort",
      meta => { "inventory", "attribute_name=none" } ;

    has_lsb_release::
      "linuxmint"
      expression => regcmp("(?ms).*^DISTRIB_ID=LinuxMint$.*", "$(lsb_release_info)"),
      comment => "this is a Linux Mint system, of some sort",
      meta => { "inventory", "attribute_name=none" } ;

    linuxmint.has_os_release::
      "lmde"
      expression => regcmp('(?ms).*^NAME="Linux Mint LMDE"$.*', "$(inventory_linux.os_release_info)"),
      comment => "this is a Linux Mint Debian Edition",
      meta => { "inventory", "attribute_name=none", "derived-from=inventory_linux.os_release_info" } ;

    linuxmint.has_lsb_release::
      "lmde"
      expression => regcmp('(?ms).*^DISTRIB_DESCRIPTION="LMDE.*', "$(lsb_release_info)"),
      comment => "this is a Linux Mint Debian Edition",
      meta => { "inventory", "attribute_name=none", "derived-from=inventory_debian.lsb_release_info" } ;

    has_etc_linuxmint_info::
      "lmde"
      expression => regcmp('(?ms).*^DESCRIPTION="LMDE.*',"$(linuxmint_info)"),
      comment => "this is a Linux Mint Debian Edition",
      meta => { "inventory", "attribute_name=none", "derived-from=inventory_debian.linuxmint_info" } ;

    debian_derived_evaluated.has_etc_linuxmint_info.!lmde::
      # These need to be evaluated only after debian_derived_evaluated is defined
      # to ensure that the mint_info array has been evaluated as well.
      # Failing to do that will create meaningless classes
      # On non-LMDE Mint systems, this will create classes like, e.g.:
      # linuxmint_14, nadia, linuxmint_nadia
      "linuxmint_$(mint_release)"  expression => "any",
      meta => { "inventory", "attribute_name=none" } ;

      "$(mint_codename)"           expression => "any",
      meta => { "inventory", "attribute_name=none" } ;

      "linuxmint_$(mint_codename)" expression => "any",
      meta => { "inventory", "attribute_name=none" } ;

    debian_derived_evaluated::
      "debian_pure" expression => "debian.!(ubuntu|linuxmint)",
      comment => "pure Debian",
      meta => { "inventory", "attribute_name=none" };

      "debian_derived" expression => "debian.!debian_pure",
      comment => "derived from Debian",
      meta => { "inventory", "attribute_name=none" };

    any::
      "has_lsb_release" expression => fileexists("/etc/lsb-release"),
      comment => "Check if we can get more info from /etc/lsb-release";

      "has_etc_linuxmint_info" expression => fileexists("/etc/linuxmint/info"),
      comment => "If this is a Linux Mint system, this *could* be available";

}

Red Hat

  • lives in: redhat.cf
  • applies to: Red Hat and its derivatives
  • provides classes: redhat_pure, redhat_derived
  • implementation:
bundle common inventory_redhat
{
  classes:
      "redhat_pure" expression => "redhat.!(centos|oracle|fedora)",
      comment => "pure Red Hat",
      meta => { "inventory", "attribute_name=none" };

      "redhat_derived" expression => "redhat.!redhat_pure",
      comment => "derived from Red Hat",
      meta => { "inventory", "attribute_name=none" };

      "inventory_redhat_have_python_symlink" expression => fileexists("$(sys.bindir)/python");

    inventory_redhat_have_python_symlink::
      "cfe_yum_package_module_supported" -> { "CFE-2602" }
        comment => "Here we see if the version of python found is acceptable for
                    the yum package module. We use this guard to prevent errors
                    related to missing python modules.",
        expression => returnszero("$(sys.bindir)/python -V 2>&1 | grep ^Python | cut -d' ' -f 2 | ( IFS=. read v1 v2 v3 ; [ $v1 -ge 3 ] || [ $v1 -eq 2 -a $v2 -ge 4 ] )",
                                  useshell);
}

Windows

  • lives in: windows.cf

Mac OS X

  • lives in: macos.cf

Generic (unknown OS)

  • lives in: generic.cf (see any.cf for generally applicable inventory modules)

LLDP

  • lives in: any.cf
  • runs inventory_control.lldpctl_exec through a Perl filter
  • provides variables: cfe_autorun_inventory_LLDP.K for each K returned by the LLDB executable
  • implementation:
bundle agent cfe_autorun_inventory_LLDP
{
  classes:
      "lldpctl_exec_exists" expression => fileexists($(inventory_control.lldpctl_exec));

  vars:
    !disable_inventory_LLDP.lldpctl_exec_exists::
      # TODO When CFE-3108 is DONE, migrate to capturing only stdout
      "info" -> { "CFE-3109", "CFE-3108" }
        data => parsejson(execresult("$(inventory_control.lldpctl_json) 2>/dev/null", "useshell")),
        if => not(isvariable("def.lldpctl_json")),
        comment => "Not all versions of lldpctl support json, and because an
        absent lldpd will result in an error on stderr resulting noisy logs and
        failure to parse the json we redirect to dev null";

      "info" -> { "CFE-3109" }
        data => parsejson(execresult($(inventory_control.lldpctl_json), "noshell")),
        if => isvariable("def.lldpctl_json"),
        comment => "For safety, we do not run lldpctl in a shell if the path to
        lldpctl is customized via augments";
}

mtab

  • lives in: any.cf
  • parses: /etc/mtab
  • provides classes: have_mount_FSTYPE and have_mount_FSTYPE_MOUNTPOINT
  • implementation:
bundle agent cfe_autorun_inventory_mtab
{
  vars:
    have_mtab::
      "mount_count" int =>  readstringarrayidx("mounts",
                                               $(inventory_control.mtab),
                                               "\s*#[^\n]*",
                                               "\s+",
                                               500,
                                               50000);

      "idx" slist => getindices("mounts");

  classes:
      "have_mtab" expression => fileexists($(inventory_control.mtab));

      # define classes like have_mount_ext4__var for a ext4 /var mount
      "have_mount_$(mounts[$(idx)][2])_$(mounts[$(idx)][1])"
      expression => "any",
      scope => "namespace";

      # define classes like have_mount_ext4 if there is a ext4 mount
      "have_mount_$(mounts[$(idx)][2])"
      expression => "any",
      scope => "namespace";

  reports:
    verbose_mode::
      "$(this.bundle): we have a $(mounts[$(idx)][2]) mount under $(mounts[$(idx)][1])";
}
  • sample output (note this is verbose mode with -v because there's a lot of output):
% cf-agent -Kv -binventory_control,cfe_autorun_inventory_mtab|grep 'cfe_autorun_inventory_mtab: we have'

R: cfe_autorun_inventory_mtab: we have a ext4 mount under /
...
R: cfe_autorun_inventory_mtab: we have a cgroup mount under /sys/fs/cgroup/systemd
R: cfe_autorun_inventory_mtab: we have a tmpfs mount under /run/shm

fstab

  • lives in: any.cf
  • parses: sys.fstab
  • provides classes: have_fs_FSTYPE have_fs_MOUNTPOINT and have_fs_FSTYPE_MOUNTPOINT
  • implementation:
bundle agent cfe_autorun_inventory_fstab
{
  vars:
    have_fstab::
      "mount_count" int =>  readstringarrayidx("mounts",
                                               $(sys.fstab),
                                               "\s*#[^\n]*",
                                               "\s+",
                                               500,
                                               50000);

      "idx" slist => getindices("mounts");

  classes:
      "have_fstab" expression => fileexists($(sys.fstab));

      # define classes like have_fs_ext4__var for a ext4 /var entry
      "have_fs_$(mounts[$(idx)][2])_$(mounts[$(idx)][1])"
      expression => "any",
      scope => "namespace";

      # define classes like have__var for a /var entry
      "have_fs_$(mounts[$(idx)][1])"
      expression => "any",
      scope => "namespace";

      # define classes like have_fs_ext4 if there is a ext4 entry
      "have_fs_$(mounts[$(idx)][2])"
      expression => "any",
      scope => "namespace";

  reports:
    verbose_mode::
      "$(this.bundle): we have a $(mounts[$(idx)][2]) fstab entry under $(mounts[$(idx)][1])";
}
  • sample output (note this is verbose mode with -v because there's a LOT of output):
% cf-agent -Kv -binventory_control,cfe_autorun_inventory_fstab|grep 'cfe_autorun_inventory_fstab: we have'

R: cfe_autorun_inventory_fstab: we have a ext4 fstab entry under /
R: cfe_autorun_inventory_fstab: we have a cifs fstab entry under /backups/load
R: cfe_autorun_inventory_fstab: we have a auto fstab entry under /mnt/cdrom

CMDB

  • lives in: any.cf
  • parses: me.json (which is copied from the policy server; see implementation)
  • provides classes: CLASS for each CLASS found under the classes key in the JSON data
  • provides variables: inventory_cmdb_load.VARNAME for each VARNAME found under the vars key in the JSON data
  • implementation:
bundle agent inventory_cmdb_load(file)
{
  classes:
      "have_cmdb_data" expression => isvariable("cmdb");

      "$(ckeys)" expression => "any", scope => "namespace";

  vars:
      "cmdb"
        data => readjson($(file), "999999"),
        ifvarclass => fileexists( $(file) );

      "cmdb_string"
      string => format("%S", cmdb),
        ifvarclass => isvariable( cmdb );

      "bkeys" slist => getindices("cmdb[vars]");
      "vkeys_$(bkeys)" slist => getindices("cmdb[vars][$(bkeys)]");
      "$(vkeys_$(bkeys))" string => nth("cmdb[vars][$(bkeys)]", $(vkeys));

      "ckeys" slist => getindices("cmdb[classes]");

  reports:
    DEBUG|DEBUG_inventory_cmdb_load::
      "DEBUG $(this.bundle): Got CMDB data from $(file): $(cmdb_string)"
        ifvarclass => "have_cmdb_data";
      "DEBUG $(this.bundle): Got CMDB key = $(vkeys_$(bkeys)), CMDB value = $((vkeys_$(bkeys)))"
        ifvarclass => "have_cmdb_data";
      "DEBUG $(this.bundle): Got CMDB class = $(ckeys)"
        ifvarclass => "have_cmdb_data";
      "DEBUG $(this.bundle): Could not read the CMDB data from $(file)"
        ifvarclass => "!have_cmdb_data";
}

DMI decoding

  • lives in: any.cf
  • runs: dmidecode
  • provides variables: cfe_autorun_inventory_dmidecode.dmi[K] for each key K in the dmidecode output
  • implementation:
bundle agent cfe_autorun_inventory_dmidecode
{
  vars:
      "dmidefs" data => parsejson('
{
  "bios-vendor": "BIOS vendor",
  "bios-version": "BIOS version",
  "system-serial-number": "System serial number",
  "system-manufacturer": "System manufacturer",
  "system-version": "System version",
  "system-product-name": "System product name",
  "system-uuid": "System UUID",
}');
  • sample output (sudo is needed to access the DMI):
% sudo /var/cfengine/bin/cf-agent -KI -binventory_control,cfe_autorun_inventory_dmidecode

R: cfe_autorun_inventory_dmidecode: Obtained BIOS vendor = 'Intel Corp.'
R: cfe_autorun_inventory_dmidecode: Obtained BIOS version = 'BLH6710H.86A.0146.2013.1555.1888'
R: cfe_autorun_inventory_dmidecode: Obtained System serial number = ''
R: cfe_autorun_inventory_dmidecode: Obtained System manufacturer = ''
R: cfe_autorun_inventory_dmidecode: Obtained System version = ''
R: cfe_autorun_inventory_dmidecode: Obtained CPU model = 'Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz'

Listening ports

  • lives in: any.cf
  • provides variables: cfe_autorun_inventory_listening_ports.ports as a copy of mon.listening_ports
  • implementation:
bundle agent cfe_autorun_inventory_listening_ports
{
  vars:
      "ports" -> { "ENT-150" }
        slist => sort( "mon.listening_ports", "int"),
        meta => { "inventory", "attribute_name=Ports listening" },
        ifvarclass => some("[0-9]+", "mon.listening_ports"),
        comment => "We only want to inventory the listening ports if we have
                    values that make sense.";
}

Disk space

  • lives in: any.cf
  • provides variables: cfe_autorun_inventory_disk.free as a copy of mon.value_diskfree
  • implementation:
bundle agent cfe_autorun_inventory_disk
{
  vars:
    enterprise::
      "free" -> { "ENT-5190" }
        string => "$(mon.value_diskfree)",
        meta => { "inventory", "attribute_name=Disk free (%)" },
        if => isvariable( "mon.value_diskfree" );
}

Available memory

  • lives in: any.cf
  • provides variables: cfe_autorun_inventory_memory.free as a copy of mon.value_mem_free and cfe_autorun_inventory_memory.total as a copy of mon.value_mem_total
  • implementation:
bundle agent cfe_autorun_inventory_memory
{
  vars:
@if minimum_version(3.11)
      # The `with` attribute is necessary for this to work in a single promise.

    enterprise_edition.windows::

      # wmic returns "TotalVisibleMemorySize=10760224" so split on = and take
      # the second item (0-based with nth())

      "total" -> { "ENT-4188" }
        meta => { "inventory", "attribute_name=Memory size (MB)" },
        string => format( "%d", eval("$(with)/1024", "math", "infix" )),
        if => not( isvariable( "total" ) ),
        with => nth( string_split( execresult("wmic OS get TotalVisibleMemorySize /format:list", useshell ),
                                   "=", 2), 1);

      "totalPhysical" -> { "CFE-2896" }
        meta => { "inventory", "attribute_name=Physical memory (MB)" },
        string => format( "%d", eval("$(with)/1024", "math", "infix" )),
        if => not( isvariable( "total" ) ),
        with => nth( string_split( execresult("wmic ComputerSystem get TotalPhysicalMemory /format:list", useshell ),
                                   "=", 2), 1);

      # This is a volatile metric, perhaps  not well suited for inventory
      "free"
        meta => { "report" },
        string => format( "%d", eval("$(with)/1024", "math", "infix" )),
        if => not( isvariable( "free" ) ),
        with => nth( string_split( execresult("wmic OS get FreePhysicalMemory /format:list", useshell ),
                                   "=", 2), 1);
@endif
    enterprise_edition.aix::
      "total" -> { "CFE-2797", "CFE-2803" }
        string => execresult("/usr/bin/lparstat -i | awk '/Online Memory/ { print $4 }'", "useshell"),
        meta => { "inventory", "attribute_name=Memory size (MB)" };

    enterprise_edition.hpux::
       "total" -> { "ENT-4188" }
         string => execresult( "machinfo | awk '/^Memory =/ {print $3}'", useshell ),
         meta => { "inventory", "attribute_name=Memory size (MB)" };

    enterprise_edition.!(aix|windows|hpux)::
      "total" string => "$(mon.value_mem_total)",
        meta => { "inventory", "attribute_name=Memory size (MB)" },
        if => isvariable( "mon.value_mem_total" );

      "free" string => "$(mon.value_mem_free)",
        if => and( not( isvariable( "free" ) ),
                   isvariable( "mon.value_mem_free" )),
        meta => { "report" };

}

Load average

  • lives in: any.cf
  • provides variables: cfe_autorun_inventory_loadaverage.value as a copy of mon.value_loadavg
  • implementation:
bundle agent cfe_autorun_inventory_loadaverage
{
  vars:
    enterprise::
      "value" -> { "ENT-5190" }
        string => "$(mon.value_loadavg)",
        meta => { "report" },
        if => isvariable( "mon.value_loadavg" );
}

procfs

  • lives in: any.cf
  • parses: consoles, cpuinfo, modules, partitions, version
  • provides variables: cfe_autorun_inventory_proc.console_count, cfe_autorun_inventory_proc.cpuinfo[K] for each CPU info key, cfe_autorun_inventory_proc.paritions[K] for each partition key
  • provides classes: _have_console_CONSOLENAME, have_module_MODULENAME
  • implementation:
bundle agent cfe_autorun_inventory_proc
{
  vars:
      "basefiles" slist => { "consoles", "cpuinfo", "modules", "partitions", "version" };
      "files[$(basefiles)]" string => "$(inventory_control.proc)/$(basefiles)";

    _have_proc_consoles::
      "console_count" int =>  readstringarrayidx("consoles",
                                                 "$(files[consoles])",
                                                 "\s*#[^\n]*",
                                                 "\s+",
                                                 500,
                                                 50000);

      "console_idx" slist => getindices("consoles");

    _have_proc_modules::
      "module_count" int =>  readstringarrayidx("modules",
                                                "$(files[modules])",
                                                "\s*#[^\n]*",
                                                "\s+",
                                                2500,
                                                250000);

      "module_idx" slist => getindices("modules");

    _have_proc_cpuinfo::
      # this will extract all the keys in one bunch, so you won't get
      # detailed info for processor 0 for example
      "cpuinfo_count" int =>  readstringarrayidx("cpuinfo_array",
                                                 "$(files[cpuinfo])",
                                                 "\s*#[^\n]*",
                                                 "\s*:\s*",
                                                 500,
                                                 50000);

      "cpuinfo_idx" slist => getindices("cpuinfo_array");
      "cpuinfo[$(cpuinfo_array[$(cpuinfo_idx)][0])]" string => "$(cpuinfo_array[$(cpuinfo_idx)][1])";
      "cpuinfo_keys" slist => getindices("cpuinfo");

    _have_proc_partitions::
      "partitions_count" int =>  readstringarrayidx("partitions_array",
                                                    "$(files[partitions])",
                                                    "major[^\n]*",
                                                    "\s+",
                                                    500,
                                                    50000);

      "partitions_idx" slist => getindices("partitions_array");
      "partitions[$(partitions_array[$(partitions_idx)][4])]" string => "$(partitions_array[$(partitions_idx)][3])";
      "partitions_keys" slist => getindices("partitions");

    _have_proc_version::
      "version" string => readfile("$(files[version])", 2048);

  classes:
      "have_proc" expression => isdir($(inventory_control.proc));

    have_proc::
      "_have_proc_$(basefiles)"
      expression => fileexists("$(files[$(basefiles)])");

    _have_proc_consoles::
      "have_console_$(consoles[$(console_idx)][0])"
      expression => "any",
      scope => "namespace";

    _have_proc_modules::
      "have_module_$(modules[$(module_idx)][0])"
      expression => "any",
      scope => "namespace";

  reports:
    _have_proc_consoles.verbose_mode::
      "$(this.bundle): we have console $(consoles[$(console_idx)][0])";
    _have_proc_modules.verbose_mode::
      "$(this.bundle): we have module $(modules[$(module_idx)][0])";
    _have_proc_cpuinfo.verbose_mode::
      "$(this.bundle): we have cpuinfo $(cpuinfo_keys) = $(cpuinfo[$(cpuinfo_keys)])";
    _have_proc_partitions.verbose_mode::
      "$(this.bundle): we have partitions $(partitions_keys) with $(partitions[$(partitions_keys)]) blocks";
    _have_proc_version.verbose_mode::
      "$(this.bundle): we have kernel version '$(version)'";
}
  • sample output (note this is verbose mode with -v because there's a LOT of output):
% cf-agent -Kv -binventory_control,cfe_autorun_inventory_proc|grep 'cfe_autorun_inventory_proc: we have'

R: cfe_autorun_inventory_proc: we have console tty0

R: cfe_autorun_inventory_proc: we have module snd_seq_midi
...
R: cfe_autorun_inventory_proc: we have module ghash_clmulni_intel

R: cfe_autorun_inventory_proc: we have cpuinfo flags = fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
...
R: cfe_autorun_inventory_proc: we have cpuinfo model name = Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz

R: cfe_autorun_inventory_proc: we have partitions sr0 with 1048575 blocks
...
R: cfe_autorun_inventory_proc: we have partitions sda with 468851544 blocks

R: cfe_autorun_inventory_proc: we have kernel version 'Linux version 3.11.0-15-generic (buildd@roseapple) (gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu8) ) #25-Ubuntu SMP Thu Jan 30 17:22:01 UTC 2014'