Show notes: The agent is in - Episode 61 - Use our lightweight C library libntech to make a handy json tool

We thought we would mix things up a bit here and highlight our open source C library: libntech.

I wanted to use libntech’s JSON code to write a minimal tool to parse, check and query JSON data. Using the various library functions related to files, json, strings, sequences and regexes helps make writing a C program a breeze and the result easy to read.

A beginning (of sorts)

This library was created in August 2019 with this commit:

Author: CFEngine Contributors <>
Date:   Wed Aug 14 12:41:41 2019 +0200

    Forked cfengine/core
    
    libntech was forked from CFEngine community at this commit SHA:
    
    https://github.com/cfengine/core/commit/3b6af9850007fe9339db7d7fcc69f490acea5867
    
    To see git history, blame, etc. before this point, checkout the commit in
    cfengine/core above. This was done because the cfengine repository already had
    15 000 commits, takes quite some time to clone and most of the history is not
    related to the contents of libntech (only a small part of the CFEngine codebase).
    This will also give the repo a "fresh" start in git and github statistics.
    Over time these will show contributors to the new libntech library, not to the
    rest of the cfengine codebase.
    
    Thank you to everyone who contributed to CFEngine in the past:
    
    https://github.com/cfengine/core/blob/3b6af9850007fe9339db7d7fcc69f490acea5867/AUTHORS
    
    Signed-off-by: Ole Herman Schumacher Elgesem <ole@northern.tech>

How to use libntech

There is a template repository in github libntech-example which can be used to create a new repository. The advantage over cloning libntech is that you get a nice clean commit history and a pre-canned auto tools setup with libntech as a git submodule.

First, visit libntech-example and click on Use this template and Create a new repository.

Once that is done simply follow the directions in the README.md for cloning, initialization and building.

Make a new command

Adding a new binary program to build in automake is fairly straight forward and easier still because we already have an example to follow in the sample command argv_printer.

Here we add a new json program to the top-level Makefile.am:

--- a/Makefile.am
+++ b/Makefile.am
@@ -17,9 +17,16 @@ ACLOCAL_AMFLAGS = -I m4 --install
 EXTRA_DIST = m4/NOTES
 
 bin_PROGRAMS = \
+       json \
        argv_printer_zero \
        argv_printer_libs # Equal length file name for fair comparison
 
+json_SOURCES = json.c
+json_LDADD = libntech/libutils/.libs/libutils.la
+json_CPPFLAGS = -I libntech/libutils/ -I libntech/libcompat/
+json_CFLAGS = -Os -fdata-sections -ffunction-sections
+json_LDFLAGS = $(STRIP_LDFLAGS)

To test that this works we can add a minimal C file: json.c:

int main(int argc, char *argv[])
{
  return 0;
}

And indeed, making this program and running it works.

$ make json
gcc -DHAVE_CONFIG_H -I.  -I libntech/libutils/ -I libntech/libcompat/   -g -O2 -MT json-json.o -MD -MP -MF .deps/json-json.Tpo -c -o json-json.o `test -f 'json.c' || echo './'`json.c
mv -f .deps/json-json.Tpo .deps/json-json.Po
/bin/sh ./libtool  --tag=CC   --mode=link gcc  -g -O2   -o json json-json.o libntech/libutils/.libs/libutils.la 
libtool: link: gcc -g -O2 -o json json-json.o  libntech/libutils/.libs/libutils.a -lpcre2-8 -lssl -lcrypto -lyaml -ldl -lrt -lm
$ ./json
$ echo $?
0

Speak JSON? Yes.

Next up, let’s use libntech libraries to read a JSON file or stdin and parse it.

From file_lib.h we use safe_open(), PathExists(), FileReadFromFd() to read from stdin or a path. These functions definitely save us from having to do far more low-level programming and error condition checking.

From json.h we use JsonParse() to parse the content and return a non-zero exit code if that fails.

$ ./json <corrupt.json 
json parse failed
$ echo $?
2
$ ./json object.json 
$ echo $?
0

Notice that the first example uses pipe redirection to send corrupt.json to stdin of json and the second example uses a command line argument.

I wanted the tool to fit in with the UNIX philosophy but didn’t want to use either -f [file] or the common - for filename means stdin idiom as option parsing can get complicated.

Subcommands

Some tools use options, flags, arguments and some use sub-commands. I happen to prefer those which use sub-commands and as few options and flags as possible.

So let’s add a pretty print sub-command and migrate our existing parse check to a new check sub-command.

To do this we can use libntech’s StringStartsWith() function from string_lib.h to be sneaky about checking the sub-command and enable shortcuts like c for check and p for print or pretty.

Note: p is a command I don’t use much in ed(1) as I usually like to see line numbers with the n command.

Here I show how this looks with some sample files I have added for testing: object.json and array.json. The json.h header provides the JsonWrite() function which pretty prints a JSON element.

$ ./json p object.json 
{
  "one": {
    "Id": "one yo yo"
  },
  "two": {
    "Id": "two ho ho"
  }
}
$ ./json pr <array.json 
[
  {
    "Id": "one"
  },
  {
    "Id": "two"
  }
]

Find those needles!

Now things are getting exciting, let’s examine a JSON structure and find the bits that we are interested in.

In this case I was interestged in picking out some specific key/value pairs in container engine ps and inspect outpout from docker and podman.

There is already a very flexible JSON query language in jq that I don’t need to reproduce so let’s keep things simple.

I query for a path that is like a path glob in the UNIX shell where the components are separated by forward slashes / and the asterisk * character matches any item at that level.

To find the identifier for each container instance in the JSON output of the engine ps commands I do a query like these:

./json q '*/ID' <docker.ps.json
./json q '*/Id' <podman.ps.json

Noting that the names of the keys differ in terms of case: ID versus Id.

Looking at the code for this step you can learn how easy it is to use our JSON library is to navigate your way around the structure.

++ ./json q '*/ID' docker.ps.json 2>/dev/null
ID      "fcc93df9746b831206140f1c4a3fa0bd25cbc389a6aa1de40353024c7240eae1"
ID      "3552cd1635e85bc278358b12050887f5b28893d956f87744ef47a731ef0c32e8"
++ ./json q '*/Id' podman.ps.json 2>/dev/null
Id      "ece583449836c061d7f97b75f142a6a0e38a4ea4583227222921cb378859df83"
++ ./json q '*/Names' podman.ps.json 2>/dev/null
Names   ["agitated_aryabhata"]

Notice above that the Names value is an array of strings. I am going to assume that generally a container only has one name so not sure why this is an array but let’s give our tool the ability to grab the first item in the array and show that instead for ease of use later.

I thought using the subscript notation from C using [n] and also from the Plan 9 shell rc using (n) would be a nice combination as both seem reasonable to me.

To bring this in we can use regex.h from libntech and the fantastically helpful and code-reducing StringMatchCaptures() function. Wow! I don’t really want to learn PCRE2 today, maybe another day.

And with this addition feature we can get the name of the container out in a much more usable form without the surrounding square brackets:

++ ./json q '*/Names[0)' podman.ps.json 2>/dev/null
Names   "agitated_aryabhata"

UNIX philosophy

One final note: I tried to make this tool usable with other tools by outputing the results of the query command with a tab delimiter so our quoted values work with the cut command :).

$ ./json  q docker.ps.json '*/Status' 2>/dev/null
Status  "Up 13 days"
Status  "Up 3 weeks"
[libre - cfengine-agent-is-in-2026-05-28-libntech-and-json-command]
$ ./json  q docker.ps.json '*/Status' 2>/dev/null | cut -f2
"Up 13 days"
"Up 3 weeks"

Video

The video recording is available on YouTube:

At the end of every webinar, we stop the recording for a nice and relaxed, off-the-record chat with attendees. Join the next webinar to not miss this discussion.