Show notes: The agent is in - Episode 43 - Nick & Cody's Thanksgiving Day Special

Nick and Cody celebrate Thanksgiving with a holiday special reviewing some policy related questions Nick recently received.

Get the list of all network interfaces present

default:sys.interfaces only contains configured interfaces, not configured interfaces.

default:sys.interfaces_data at least for Linux systems, this variable contains much more information.

 
            /tmp/getindices-sys.interfaces_data.cf
            
bundle agent __main__
{
     vars:
        "sys_interfaces_data_keys"
          slist => getindices( "sys.interfaces_data" );

      reports:
          "$(sys_interfaces_data_keys)";
}

command
cf-agent --no-lock --log-level info --file /tmp/getindices-sys.interfaces_data.cf

output
R: lo
R: enx0892048803e7
R: enx5cff35c6864b
R: wlp0s20f3
R: virbr0
R: docker0
R: br-a7d465b9949b
R: vboxnet0
R: tun0

cf-promises can be very useful for reviewing the first order variables that are defined using the --show-vars option.

command
cf-promises --show-vars=sys\.interfaces

output
Variable name                            Variable value                                               Meta tags                                Comment
default:sys.interfaces                    {"enx0892048803e7","wlp0s20f3","virbr0","docker0","br-a7d465b9949b","vboxnet0","tun0"} inventory,source=agent,attribute_name=Interfaces
default:sys.interfaces_data              {"br-a7d465b9949b":{"device":"br-a7d465b9949b","receive_bytes":"0","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"0","transmit_bytes":"0","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"0"},"docker0":{"device":"docker0","receive_bytes":"0","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"0","transmit_bytes":"0","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"0"},"enx0892048803e7":{"device":"enx0892048803e7","receive_bytes":"6362565862","receive_compressed":"0","receive_drop":"10728","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"8119215","transmit_bytes":"2704360457","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"5210158"},"enx5cff35c6864b":{"device":"enx5cff35c6864b","receive_bytes":"0","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"0","transmit_bytes":"0","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"0"},"lo":{"device":"lo","receive_bytes":"30236494","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"331169","transmit_bytes":"30236494","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"331169"},"tun0":{"device":"tun0","receive_bytes":"1224387","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"6628","transmit_bytes":"1478565","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"8042"},"vboxnet0":{"device":"vboxnet0","receive_bytes":"0","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"0","transmit_bytes":"2473935","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"3790"},"virbr0":{"device":"virbr0","receive_bytes":"0","receive_compressed":"0","receive_drop":"0","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"0","transmit_bytes":"0","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"0"},"wlp0s20f3":{"device":"wlp0s20f3","receive_bytes":"29822471","receive_compressed":"0","receive_drop":"9015","receive_errors":"0","receive_fifo":"0","receive_frame":"0","receive_multicast":"0","receive_packets":"150090","transmit_bytes":"3287961","transmit_compressed":"0","transmit_drop":"0","transmit_errors":"0","transmit_fifo":"0","transmit_frame":"0","transmit_multicast":"0","transmit_packets":"9598"}} networking,/proc,source=agent,procfs

cf-agent can show the state of variables defined at the end of execution using the --show-evaluated-vars option.

Matching an IP address with a regular expression

Why was the regular expression 172\.16\.1\.* not matching 172.16.1.10?

 
            /tmp/regex-ip.cf
            
bundle agent __main__
{

  vars:
      "ipregex" string => "172\.16\.1\.*";

  classes:
      "found" expression => regcmp("$(ipregex)","172.16.1.10");

  reports:
    found::
      "Found the ip";

    !found::
      "Did not find the ip";
}

command
cf-agent --no-lock --log-level info --file /tmp/regex-ip.cf

output
R: Did not find the ip

Because there was a bug in the regex, a missing . after the escaped one:

 
            /tmp/regex-ip-fixed.cf
            
bundle agent __main__
{

  vars:
      "ipregex" string => "172\.16\.1\..*";

  classes:
      "found" expression => regcmp("$(ipregex)","172.16.1.10");

  reports:
    found::
      "Found the ip";

    !found::
      "Did not find the ip";
}

command
cf-agent --no-lock --log-level info --file /tmp/regex-ip-fixed.cf

output
R: Found the ip

Use the builtin functions for iprange() and isipinsubnet() for improved readability.

Checking IP validity

Regular expressions can still be useful for validating input from external data sources to avoid application mis-configuration.

For example, every() can be used to validate a whole list of ip addresses. Remember, regular expressions can be multi-line which can improve readability.

 
            /tmp/ipv4-validity.cf
            
bundle agent __main__
{

  vars:
      "ip_list1" slist => { "192.168.56.2", "1.1.1.1" };
      "ip_list2" slist => { "2.1.1.356", "192.168.1.1" };

      "ipv4_regex"
      # https://watson-wilson.ca/blog/2015/08/20/build-better-regular-expressions-in-cfengine/
        string => "(?xms)
            # First octet
            (?:
               25[0-5]            # Remember ips go as high as 255.
               |                  # or
               2[0-4][0-9]        # Between 200 and 249.
               |                  # or
               [0-1]?[0-9]{1,2}   # Less than 200.
            )
            [.]                   # Decimal point

            # Second octet
            (?:
               25[0-5]            # Remember ips go as high as 255.
               |                  # or
               2[0-4][0-9]        # Between 200 and 249.
               |                  # or
               [0-1]?[0-9]{1,2}   # Less than 200.
            )
            [.]                   # Decimal point
            # Third octet
            (?:
               25[0-5]            # Remember ips go as high as 255.
               |                  # or
               2[0-4][0-9]        # Between 200 and 249.
               |                  # or
               [0-1]?[0-9]{1,2}   # Less than 200.
            )
            [.]                   # Decimal point
            # Forth octet
            (?:
               25[0-5]            # Remember ips go as high as 255.
               |                  # or
               2[0-4][0-9]        # Between 200 and 249.
               |                  # or
               [0-1]?[0-9]{1,2}   # Less than 200.
            )
         ";

  classes:
      "ip_list1_all_valid_ipv4" expression => every( "$(ipv4_regex)", @(ip_list1) );
      "ip_list2_all_valid_ipv4" expression => every( "$(ipv4_regex)", @(ip_list2) );

  reports:
    ip_list1_all_valid_ipv4::
      "All elements of ip_list1 are valid IPv4 addresses";

    ip_list2_all_valid_ipv4::
      "All elements of ip_list2 are valid IPv4 addresses";

    !ip_list1_all_valid_ipv4::
      "Not all elements of ip_list1 are valid IPv4 addresses";

    !ip_list2_all_valid_ipv4::
      "Not all elements of ip_list2 are valid IPv4 addresses";
}

command
cf-agent --no-lock --log-level info --file /tmp/ipv4-validity.cf

output
R: All elements of ip_list1 are valid IPv4 addresses
R: Not all elements of ip_list2 are valid IPv4 addresses

Validating IPv4 addresses with isipinsubnet()

You can use isipinsubnet() to check that an IP address is valid (Nick was trying to use iprange() incorrectly for this during the recording):

 
            /tmp/validate-ip-isipinsubnet.cf
            
bundle agent __main__
{
   vars:
      "ip" string => "172.16.1.10";
      "range" string => "0.0.0.0/0";

  classes:
      "valid_ipv4" expression => isipinsubnet( "$(range)", "$(ip)" );

  reports:
    valid_ipv4::
      "The IP $(ip) is part of the range $(range)";

    !valid_ipv4::
      "The IP $(ip) is NOT part of the range $(range)";
}

command
cf-agent --no-lock --log-level info --file /tmp/validate-ip-isipinsubnet.cf

output
: R: The IP 172.16.1.10 is part of the range 0.0.0.0/0

Limiting messages to the system log

system_log_level in body common control control controls the minimum log level for messages to be sent to the system log.

Newly exposed (master/3.25.0) to be configurable via Augments.

{
  "variables": {
    "default:def.control_common_system_log_level": {
      "value": "info",
      "comment": "We want syslog to recieve messages tha are level info and above."
    }
  }
}

Is it possible to increase log verbosity for bundles called via usebundle?

Yes, it’s possible to increase the verbosity of specific promises, but not for all promises in a specific bundle. log_level in action body allows you to increase the verbosity of a specific promise, but it does not apply to all promises of a bundle at once (nor would it be inherited by subsequent methods promises).

Is it possible to change the order of promises

Normal ordering (vars before classes, files before packages) can not be changed but promise execution can be influenced using classes and/or more weakly with the depends_on attribute.

 
            /tmp/depends_on.cf
            
bundle agent __main__
{
  reports:
      "one" handle => "one", depends_on => { "two"};
      "two" handle => "two";
}

command
cf-agent --no-lock --log-level info --file /tmp/depends_on.cf

output
R: two
R: one

Maybe someday, an option to resolve using written order vs normal order.

Video

The video recording is available on YouTube:

At the end of every webinar, we stop the recording for a nice and relaxed, off-the-record chat with attendees. Join the next webinar to not miss this discussion.