libntech 1.0: now available to more projects

The license of our in-house C utility and compatibility library libntech was recently changed from GPLv3 to Apache License Version 2.0 which makes the library suitable for more projects thanks to the more permissive license. While GPLv3 practically required any project using libntech to be licensed under GPLv3 as well, the Apache License v2.0 allows any open source as well as proprietary software to utilize our utility library, keeping the copyright attributions.

With this important change we also decided to release the current version of libntech as version 1.0. Maybe a big jump from 0.1 which could suggest an early development stage, but we believe that the code is very mature and want to reflect that in the version number.

(Hi)Story

CFEngine, being written in C and being a portable/multi-platform project started in 1993, has required implementations of many things that are usually provided by standard libraries in other languages and environments. Be it data structures like linked lists, sequences (automatically expanding arrays), hash maps and many others or utility functions for various string operations, locking and threading primitives, etc. Plus implementations of commonly, but not strictly generally available things like getline(), getopt(), poll(),…

Around 2019 it was quite obvious that this code could be usable for other projects in our company and also for anybody else in need of a simple, fast and small C utility library. Building CFEngine and extracting the library from it was difficult, challenging and cumbersome and thus libntech as a separate project and repository was created as a fork of the code from the CFEngine core repository. And that's how it inherited CFEngine's license – GPLv3. Or, to be more precise, the dual license GPLv3 + COSL (our in-house Commercial Open Source License). This posed no licensing problems for us because both use cases for the library were compatible.

The situation changed this year, in 2023, when we started rewriting parts of our other product – Mender – in C++. With Mender using the Apache License, it was impossible to use libntech unless we would choose GPLv3 for at least some of its parts. Using different and incompatible FOSS licenses for parts of one project didn't sound like a good idea and we also quickly realized that other potential users of our utility library will likely have the same problem. Thus, since we had the option thanks to being copyright owners of the libntech code and having the right to relicense it, we decided to change the license to Apache License, Version 2.0, the same license that the open-source parts of Mender use.

What it has to offer

libntech has two major parts – libutils and libcompat – that contain the implementations of utility functions and data structures, and compatibility functions, respectively. libcompat is much smaller and provides implementations of around 50 functions. Much of its code was taken from various places and is under various authors' copyrights and custom licenses. All of it now being compatible with Apache License v2.0.

libutils is a much bigger part of the codebase and provides simple, fast and small implementations of the following data structures and some more:

• Sequences (aka expandable arrays)
• Hash maps and sets
• Expandable buffers
• Double-linked lists
• Queues (FIFO) and stacks (LIFO)
• Red-black trees
• Ring buffers
• Thread-safe queues, deqeues and stacks

It also provides a simple logging framework and memory allocation functions that check for OOM and kill the process accordingly. Then many helper and utility functions for working with strings, files, directories, file system paths, regular expressions, IP addresses and others. Last but not least, actually, some would say that this is the most valuable part, it also provides simple implementations for JSON and CSV file formats, for rendering Mustache templates and a custom very straightforward format for storing and loading sequences of strings.

There are still a couple places where the origin of the code is more obvious than what would be ideal (e.g. the ManPageWrite() function writes a hard-coded CFEngine header and footer and the known_dirs.h API is CFEngine specific), but we are actively working on either (re)moving the code or, wherever possible, making it more generic. In the meantime, these few functions should either be avoided or patched with project-specific snippets.

All of the code is truly open-source, not only from the licensing perspective, but also by being very easy to read and understand and thus to modify and tailor for the needs of the project it's being used by. Sometimes it even prefers simplicity and readability over optimization because we believe easier to read and thus easier to debug code is more useful than super-optimized code nobody understands and can work with. We’d like to thank everyone¹ in our community who has contributed to the CFEngine codebase (and libntech) over the years, and made it what it is today.