Show notes: The agent is in - Episode 39 - Editing /etc/default/grub and similar files

Posted by Nick Anderson
July 25, 2024

Ever want to manage subvalues in a configuration file? In episode 39 we talk about using field_edits in edit_line bundles.

Cody and Craig join Nick as he prototypes and iterates on some policy showing how to manage subvalues in KEY = quoted values. A question raised during one of our recent post show discussions.

Video

The video recording is available on YouTube:

At the end of every webinar, we stop the recording for a nice and relaxed, off-the-record chat with attendees. Join the next webinar to not miss this discussion.

Developing policy to edit /etc/default/grub and similar files

Promise the initial content for a test file

To prototype the policy we first lay down a file with content that is representative of what we are trying to edit. This is very helpful in prototyping policy as it re-sets the initial state of the test file during each execution.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
    info: Updated file '/tmp/etc-default-grub' with content 'GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      '
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
R: GRUB_CMDLINE_LINUX=""
R:

Testing against GRUB_CMDLINE_LINUX_DEFAULT

When we went to make a promise about GRUB_CMDLINE_LINUX_DEFAULT we encountered an error. The field_separator in body edit_field quoted_var is configured to look only for double quotes.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub;

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}

bundle edit_line etc_default_grub
{
  field_edits:
      "GRUB_CMDLINE_LINUX=.*"
        edit_field => quoted_var( "ipv6.disable=1", "append" );

      "GRUB_CMDLINE_LINUX_DEFAULT=.*"
        edit_field => quoted_var( "init_on_alloc=1", "append" );
}

body edit_field single_or_double_quoted_subvalue( entry, method)
{
        inherit_from => quoted_var( "$(entry)", "$(method)" );
        field_separator => "(\"|')";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
   error: Field edit, no fields found by promised pattern '"' in '/tmp/etc-default-grub'
   error: Errors encountered when actuating fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*'
    info: Appended field sub-value 'ipv6.disable=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
R: GRUB_CMDLINE_LINUX="ipv6.disable=1"
R:

Refactor to handle single or double quotes

Since body edit_field quoted_var from the standard library does not handle single quotes we define our own edit_field body, single_or_double_quoted_subvalue (based on quoted_var) which supports single or double quoted values.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub;

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}

bundle edit_line etc_default_grub
{
  field_edits:
      "GRUB_CMDLINE_LINUX=.*"
        edit_field => quoted_var( "ipv6.disable=1", "append" );

      "GRUB_CMDLINE_LINUX_DEFAULT=.*"
        edit_field => single_or_double_quoted_subvalue( "init_on_alloc=1", "append" );
}

body edit_field single_or_double_quoted_subvalue( entry, method)
{
        inherit_from => quoted_var( "$(entry)", "$(method)" );
        field_separator => "(\"|')";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
    info: Updated file '/tmp/etc-default-grub' with content 'GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      '
    info: Appended field sub-value 'ipv6.disable=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX=.*' repaired
    info: Appended field sub-value 'init_on_alloc=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash init_on_alloc=1'
R: GRUB_CMDLINE_LINUX="ipv6.disable=1"
R:

Make more re-usable (but less efficient)

Next we made the policy more re-usable by exposing parameters to the edit_line bundle. This change however is less efficient as each files promise results in a separate file operation.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub("GRUB_CMDLINE_LINUX",
                                      "ipv6.disable=1",
                                      "append" );

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub("GRUB_CMDLINE_LINUX_DEFAULT",
                                      "init_on_alloc=1",
                                      "append" );

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub("GRUB_CMDLINE_LINUX_DEFAULT",
                                      "quiet",
                                      "delete" );

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}

bundle edit_line etc_default_grub(key, entry, method)
{
  field_edits:

      "$(key)=.*"
        edit_field => single_or_double_quoted_subvalue( "$(entry)", "$(method)" );
}

body edit_field single_or_double_quoted_subvalue( entry, method)
{
        inherit_from => quoted_var( "$(entry)", "$(method)" );
        field_separator => "(\"|')";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
    info: Updated file '/tmp/etc-default-grub' with content 'GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      '
    info: Appended field sub-value 'ipv6.disable=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
    info: Appended field sub-value 'init_on_alloc=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
    info: Deleted column field sub-value 'quiet' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash init_on_alloc=1'
R: GRUB_CMDLINE_LINUX="ipv6.disable=1"
R:

From the execution output above we can clearly see that the file is actually modified three separate times.

Making it a single edit – broken – need coffee

During the show we started to reign in the file operations while maintaining the ability to reuse the policy in different contexts but Nick hadn’t yet had his coffee and chose poorly when he defined the data structure.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  vars:
     "conf[GRUB_CMDLINE_LINUX][ipv6.disable=1]"         string => "append";
     "conf[GRUB_CMDLINE_LINUX_DEFAULT][init_on_alloc=1]" string => "append";
     "conf[GRUB_CMDLINE_LINUX_DEFAULT][quiet]" string => "delete";

  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub( "$(this.namespace):$(this.bundle).conf" );

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}

bundle edit_line etc_default_grub(d)
{

  vars:
     "s" data => mergedata( d );
     "keys" slist => getindices( s );

  field_edits:

      "$(s[$(keys)])=.*"
        edit_field => single_or_double_quoted_subvalue( "$(s[$(keys)][1])",
                                                        "$(s[$(keys)][2])" );
}

body edit_field single_or_double_quoted_subvalue( entry, method)
{
        inherit_from => quoted_var( "$(entry)", "$(method)" );
        field_separator => "(\"|')";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
    info: Updated file '/tmp/etc-default-grub' with content 'GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      '
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
R: GRUB_CMDLINE_LINUX=""
R:

The fix for a single edit

The solution came to Nick after the post show discussion when he walked away to get coffee.

He needed to change the way he was structuring the data. Here we see he switched to a data type variable and changed the way that the data was passed to the edit_line bundle.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  vars:
     "conf" data => '
{
  "GRUB_CMDLINE_LINUX": {
    "append": [ "ipv6.disable=1" ],
  },
  "GRUB_CMDLINE_LINUX_DEFAULT": {
    "prepend": [ "init_on_alloc=1" ],
    "delete": [ "quiet" ],
  }
}';

  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub( "@($(this.namespace):$(this.bundle).conf)" );

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}

bundle edit_line etc_default_grub(conf_map)
{
  vars:
     "keys" slist => getindices( conf_map );
     "methods[$(keys)]" slist => getindices( "conf_map[$(keys)]" );

  field_edits:

      "$(keys)=.*"
        edit_field => single_or_double_quoted_subvalue( "$(conf_map[$(keys)][$(methods[$(keys)])])",
                                                        "$(methods[$(keys)])" );
}

body edit_field single_or_double_quoted_subvalue( entry, method)
{
        inherit_from => quoted_var( "$(entry)", "$(method)" );
        field_separator => "(\"|')";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
    info: Updated file '/tmp/etc-default-grub' with content 'GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      '
    info: Appended field sub-value 'ipv6.disable=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX=.*' repaired
    info: Prepended field sub-value 'init_on_alloc=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Deleted column field sub-value 'quiet' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='init_on_alloc=1 cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
R: GRUB_CMDLINE_LINUX="ipv6.disable=1"
R:

From the above output we can see that all the changes were made as expected and only a single file change was made.

How do I use the same data structure with associative arrays?

In addressing the issue Nick switched to using a data variable instead of an associative array which is a collection of individual variables in CFEngine.

Here is an iteration showing use with associative arrays. Note that in addition to changing the way the data structure is defined, we also changed how it was passed back to passing the name of the variable rather than the data structure as a whole.

/tmp/example-edit-etc-grub-default.cf
bundle agent __main__
{
  vars:
      "conf[GRUB_CMDLINE_LINUX][append]" slist => { "ipv6.disable=1" };
      "conf[GRUB_CMDLINE_LINUX_DEFAULT][prepend]" slist => { "init_on_alloc=1" };
      "conf[GRUB_CMDLINE_LINUX_DEFAULT][delete]" slist => { "quiet" };

  files:
      "/tmp/etc-default-grub"
        content => `GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      `;

      "/tmp/etc-default-grub"
        edit_line => etc_default_grub( "$(this.namespace):$(this.bundle).conf" );

  reports:
      "/tmp/etc-default-grub"
        printfile => cat( "$(this.promiser)");
}

bundle edit_line etc_default_grub(conf_var)
{
  vars:
     "conf_map" data => mergedata( "$(conf_var)" );
     "keys" slist => getindices( conf_map );
     "methods[$(keys)]" slist => getindices( "conf_map[$(keys)]" );

  field_edits:

      "$(keys)=.*"
        edit_field => single_or_double_quoted_subvalue( "$(conf_map[$(keys)][$(methods[$(keys)])])",
                                                        "$(methods[$(keys)])" );
}

body edit_field single_or_double_quoted_subvalue( entry, method)
{
        inherit_from => quoted_var( "$(entry)", "$(method)" );
        field_separator => "(\"|')";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-edit-etc-grub-default.cf
output
    info: Updated file '/tmp/etc-default-grub' with content 'GRUB_CMDLINE_LINUX_DEFAULT='quiet cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
GRUB_CMDLINE_LINUX=""
      '
    info: Deleted column field sub-value 'quiet' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Prepended field sub-value 'init_on_alloc=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX_DEFAULT=.*' repaired
    info: Appended field sub-value 'ipv6.disable=1' in '/tmp/etc-default-grub'
    info: fields_edit promise 'GRUB_CMDLINE_LINUX=.*' repaired
    info: Edited file '/tmp/etc-default-grub'
R: /tmp/etc-default-grub
R: GRUB_CMDLINE_LINUX_DEFAULT='init_on_alloc=1 cryptdevice=UUID=4c56337e-878a-48c7-bca3-ed6fa50cf017:luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 root=/dev/mapper/luks-4c56337e-878a-48c7-bca3-ed6fa50cf017 splash'
R: GRUB_CMDLINE_LINUX="ipv6.disable=1"
R:

Post show discussion

  • Did you know that CFEngine has 3 quote characters? Single quote ( ' ), double quote ( " ) and back tick ( ` ). This can allow you to avoid having to escape inner quotes.

Questions

How to influence order of operations (Normal Order)

Using handle and depends_on
bundle agent __main__
[file=/tmp/example-depends-on.cf]
{
  commands:
      "/bin/echo RED"
        handle => "red",
        depends_on => { "yellow" };

      "/bin/echo YELLOW"
        handle => "yellow";

  methods:
      "help"
        depends_on => { "red" };
}
bundle agent help
{
    commands: "/bin/echo Byeee";
}
command
cf-agent --no-lock --log-level info --file /tmp/example-depends-on.cf
output
    info: Executing 'no timeout' ... '/bin/echo YELLOW'
  notice: Q: ".../bin/echo YELLO": YELLOW
    info: Last 1 quoted lines were generated by promiser '/bin/echo YELLOW'
    info: Completed execution of '/bin/echo YELLOW'
    info: Executing 'no timeout' ... '/bin/echo RED'
  notice: Q: ".../bin/echo RED": RED
    info: Last 1 quoted lines were generated by promiser '/bin/echo RED'
    info: Completed execution of '/bin/echo RED'
    info: Executing 'no timeout' ... '/bin/echo Byeee'
  notice: Q: ".../bin/echo Byeee": Byeee
    info: Last 1 quoted lines were generated by promiser '/bin/echo Byeee'
    info: Completed execution of '/bin/echo Byeee'
Use methods to get a more strict order
bundle agent __main__
{
    methods:
      "stage1";
      "stage2";
      "stage3";
}
Verify configuration before restarting the service

You may want to verify a configuration file has valid syntax before restarting a service. Consider rendering your configuration to a staging location that is verified and copied to final destination after syntax check. This way you can avoid issues that could be cause by some random reboot where you have a bad config in place but the service wasn’t restarted since it wasn’t valid.

How can I access another bundles classes?

You can not. Classes are either namespace scope (visible by any other bundle), or they are bundle scoped.

  • Classes defined from classes promises in common bundles are namespace scoped by default.
  • Classes defined from classes promises in agent bundles are bundle scoped by default.
  • You can specify override the default scope using the scope attribute.
  • Bundle scoped classes can be inherited by another bundle
  • Information from a bundle can be passed to an associative array in the parent bundle via the bundle_return_value_index attribute of reports promises if the parent called the bundle with useresult.