Function Caching in CFEngine 3.6

Posted by Mahesh Kumar
December 11, 2013

Many users have been asking for ways to limit the amount of some function invocations in CFEngine, inparticular functions such as execresult and returnszero. First, let me try to explain why functions were called so many times to begin with, and how we have approached this for version 3.6.

Functions may be executed during checking with cf-promises, or during normal evaluation.

When cf-agent executes a policy, it first runs it through checking with cf-promises. Many policy checks are static in nature (types, keywords, etc.), but since CFEngine is a fairly dynamic language, certain problems may only arise at run-time. Therefore, cf-promises has traditionally attempted to find these problems by executing all functions. For 3.6, we have turned off function evaluation for cf-promises, but retained the old behavior optionally using the flag –eval-functions.

After checking a policy, the agent proceeds with evaluation. Here, naturally all functions called from active promises should be called at least once. But why should each call be evaluated more than once for each agent run? CFEngine will only decide to not execute a promise iteration once the evaluation context for it has been set up. Suppose we have the following promise, where ‘a’ is a list varibale.

bundle agent test { reports: “Hello $(a)” ifvarclass => canonify("$(a)"); }

To set up the evaluation context for a promise iteration, $(a) is dereferenced both in the promiser and the function call. Once this is done, the ‘report’ promise handler decided whether to actuate the given context, which it does by hashing the evaluation context and match it against the promise lock database. The last part prevents promises from being executed too often. In general it is not possible to know whether a promise should not be executed before its whole evaluation context is set up, including calling associated functions.

Nevertheless, we decided for version 3.6 to group function calls into two categories: those that can have their results cached, and those that have to execute each time. We label the former category ‘system’ function, and presently they are execresult, returnszero as well as various network related functions. All calls to system function will be cached by default, meaning they will be executed at most once per run. Other function calls may be evaluated many times. This should hopefully bring performance benefits in the normal case. However, we have added an option to keep the old behavior, simply set ‘cache_system_functions => false’ in body common control.