<The CFEngine Blog

How to properly check if files are readable

Posted by Lars Erik Wik
March 28, 2023

Opening and reading files may cause your program to block indefinitely, which is a major problem for configuration management tools like CFEngine. What if we make a program that can check if these file operations would block indefinitely, before using them? Unfortunately our good friend Alan Turing proved that such programs are not theoretically possible. But if we let theory be theory; we can in practice make some compromises by changing the definition of an unreadable file, to be a file that could not be read in N number of seconds.

The first possible solution that comes to mind, is creating a program that opens a file using the O_NONBLOCK flag. Followed by using poll(2) to wait a limited amount of time for the file descriptor to become ready for reading. But the man page states that the file will be opened in a “non-blocking” manner whenever possible, and that applications should not depend upon the blocking behavior when specifying this flag for regular files and block devices. Thus this option is not ideal.

Instead, we can try performing the file operations in a separate thread, or in a forked process, waiting a limited amount of time for the thread/process to terminate. Both suggestions would work; but since fork creation is more expensive than thread creation, and since the sharing of information between processes is both more difficult and expensive, we chose to solve this problem using threads.

Solution

We’ll start off by declaring a function that takes two parameters. A filename for specifying the file that we want to check. And a timeout interval for specifying how long we’re willing to wait. The function will return an integer, indicating whether the read was successful, failed, or an unexpected error occurred.

enum {
  READABLE_SUCCESS = 0, // Successfully read file.
  READABLE_FAILURE,     // Failed to read file.
  READABLE_ERROR,       // Unexpected error occurred.
};

/**
 * @brief Determine if a file is readable.
 * @param[in] filename path to file.
 * @param[in] timeout number of milliseconds to wait.
 * @return READABLE_SUCCESS on successful read, READABLE_FAILUIRE on failed
 *         read, and READABLE_ERROR in case of unexpected error.
 */
int Readable(const char *filename, long timeout);

Our plan is to use the pthread_cond_timedwait(3) function in order to provide the timeout functionality. To do this, we’ll need to create a mutex for synchronizing the threads, and a condition to signal a state change. We’ll also need to calculate an expiration time for when to give up waiting.

static pthread_mutex_t MUTEX = PTHREAD_MUTEX_INITIALIZER;
static pthread_cond_t CONDITION = PTHREAD_COND_INITIALIZER;

int Readable(const char *const filename, const long timeout) {
  // Calculate absolute expiration time from timeout interval in milliseconds.
  struct timespec ts = {0};
  if (clock_gettime(CLOCK_REALTIME, &ts) != 0) {
    perror("Failed to get system's real time");
    return READABLE_ERROR;
  }
  ts.tv_sec += ((timeout * 1000000) + ts.tv_nsec) / 1000000000;
  ts.tv_nsec = ((timeout * 1000000) + ts.tv_nsec) % 1000000000;
}

Both the mutex and condition variables are statically initialized using the PTHREAD_MUTEX_INITIALIZER- and PTHREAD_COND_INITIALIZER macros respectively. To initialize them dynamically, you’ll need to use the pthread_mutex_init(3) and pthread_cond_init(3) functions from the pthread library, passing them to the thread routine in a struct along with the filename.

Our program takes a timeout interval (in milliseconds) as an argument. But since pthread_cond_timedwait(3) expects an absolute expiration time, we’ll need to do some calculations. We’ll get the system’s real time using clock_gettime(3). Then we’ll convert our timeout interval argument into seconds and nanoseconds, before adding it to the timespec struct.

Next we’ll lock the mutex with pthread_mutex_lock(3) before spawning the thread with pthread_create(3), giving the main thread time to call pthread_cond_timedwait(3) before the spawned thread goes about its business.

int Readable(const char *const filename, const long timeout) {
  ...

  int ret = pthread_mutex_lock(&MUTEX);
  if (ret != 0) {
    fprintf(stderr, "Failed to lock mutex: %s\n", strerror(ret));
    return READABLE_ERROR;
  }

  pthread_t thread;
  ret = pthread_create(&thread, NULL, &TryRead, (void *)filename);
  if (ret != 0) {
    fprintf(stderr, "Failed to create thread: %s\n", strerror(ret));
    return READABLE_ERROR;
  }
}

The spawned thread will execute the - soon to be defined - TryRead() routine, where it will try to acquire the mutex we just locked. The call to pthread_cond_timedwait(3) causes the main thread to release the mutex, allowing the spawned thread to acquire it and carry on.

int Readable(const char *const filename, const long timeout) {
  ...

  ret = pthread_cond_timedwait(&CONDITION, &MUTEX, &ts);
  const bool time_expired = (ret == ETIMEDOUT);

  if (time_expired) {
    ret = pthread_cancel(thread);
    if (ret != 0) {
      fprintf(stderr, "Failed to cancel thread: %s\n", strerror(ret));
      return READABLE_ERROR;
    }
  } else if (ret != 0) {
    fprintf(stderr, "Failed to wait for condition: %s\n", strerror(ret));
    return READABLE_ERROR;
  }

  void *status;
  ret = pthread_join(thread, &status);
  if (ret != 0) {
    fprintf(stderr, "Failed to join thread: %s\n", strerror(ret));
    return READABLE_ERROR;
  }
}

Once we return from pthread_cond_timedwait(3) we determine whether or not the spawned thread timed out. If the thread did not finish in time, we’ll need to cancel its execution before joining with it; preventing the main thread from getting blocked.

There is a gotcha when it comes to using pthread_cancel(3); i.e. precisely what happens to a thread when it receives a cancellation request depends on the thread’s cancellation state and type. If the cancellation state is PTHREAD_CANCEL_DISABLE; any cancellation request will remain pending until its state is changed to PTHREAD_CANCEL_ENABLE. If cancellation type is PTHREAD_CANCEL_DEFFERED, any cancellation request will remain pending until a cancellation point is reached. This is opposed to the cancellation type PTHREAD_CANCEL_ASYNCHRONOUS, in which the thread can be canceled at any time.

Cancellation state and type can be set using pthread_setcancelstate(3) and pthread_setcanceltype(3) respectively. We’ll use cancellation state PTHREAD_CANCEL_ENABLE to make sure the thread can be canceled. And we’ll use the cancellation type PTHREAD_CANCEL_DEFFERED since open(2), read(2) and close(2) are all cancellation points. Luckily these are the default cancellation state and type, so there is no need to change them. A list of functions that are cancellation points can be found in pthreads(7) man page.

Please note that setting the cancellation type to PTHREAD_CANCEL_ASYNCHRONOUS is rarely useful, and can lead to some serious challenges when it comes to managing resources. Since a thread can be canceled at any time; the cancellation can happen before, after, or even during the locking of a mutex, or a memory allocation. Thus there is no way to know the exact state when the cancellation takes place; making it impossible to safely release these resources in a cleanup handler. So if you’re planning to use this cancellation type, you better think twice.

With the spawned thread safely joined, we need to clean up after ourselves by releasing the mutex that was reacquired when returning from pthread_cond_timedwait(3), before we can return the code indicating whether or not we were able to successfully read the file.

int Readable(const char *const filename, const long timeout) {
  ...

  ret = pthread_mutex_unlock(&MUTEX);
  if (ret != 0) {
    fprintf(stderr, "Failed to unlock mutex: %s\n", strerror(ret));
    return READABLE_ERROR;
  }

  return (time_expired) ? READABLE_FAILURE : (int)status;
}

We’ll program the TryRead() function - to be executed by the spawned thread - to return whether or not the file was readable through its exit status, which we’ll obtain in the status variable when joining the thread. Please note that we don’t return the value of the status variable in case of timeout. As it may contain the value PTHREAD_CANCELED, if the thread was indeed canceled. Instead we return READABLE_FAILURE as we consider the file unreadable anyway, in this case.

The TryRead() function will be defined to take a void pointer, which will point to the name of the respective file. The function will return a void pointer indicating success, failure or error.

static void *TryRead(void *data) {
  int ret = pthread_mutex_lock(&MUTEX);
  if (ret != 0) {
    fprintf(stderr, "Failed to lock mutex: %s\n", strerror(ret));
    return (void *)READABLE_ERROR;
  }

  int fd = -1;
  bool success;
  pthread_cleanup_push(&CleanupHandler, &fd);

  ret = pthread_mutex_unlock(&MUTEX);
  if (ret != 0) {
    fprintf(stderr, "Failed to unlock mutex: %s\n", strerror(ret));
    return (void *)READABLE_ERROR;
  }
}

We’ll synchronize with the main thread by acquiring the mutex using pthread_mutex_lock(3). Once synchronized, we’ll push a cleanup handler to the thread’s cleanup handler stack using pthread_cleanup_push(3). This way we can assume that the file we’re about to open will be closed in the case of receiving a cancellation request. With the cleanup handler pushed to the stack, we release the mutex, so that the main thread is allowed to timeout.

Some UNIX implementations - including Linux - implement pthread_cleanup_push(3) and pthread_cleanup_pop(3) as macros that expand into statement sequences including the opening ({) and closing (}) of a scope. Thus every use of pthread_cleanup_push(3) must be paired with a corresponding pthread_cleanup_pop(3), within the same lexical scope. This also means that any variables declared within this scope, cannot be accessed outside it. Thus we’ll have to declare our success variable just before pthread_cleanup_push(3) so that we can still use it after the call to pthread_cleanup_pop(3).

The cleanup handler passed to pthread_cleanup_push(3) is simply defined to close the file descriptor using close(2) if it has a valid value (i.e. greater or equal to zero).

static void CleanupHandler(void *data) {
  const int fd = *((int *)data);
  if (fd >= 0) {
    close(fd);
  }
}

Continuing with our thread routine, we’ll attempt to open the respective file with open(2), and we’ll try to read 1 Byte with read(2). If the opening of the file succeeds, and if we’re able to read 1 Byte or reach end-of-file, we’ll consider the file readable. Last, but not least, we’ll pop off the cleanup handler passing 1 as the argument to pthread_cleanup_pop(3) in order to also execute the cleanup handler.

static void *TryRead(void *data) {
  ...

  char buffer[1];
  const char *const filename = data;
  fd = open(filename, O_RDONLY);
  success = ((fd > 0) && (read(fd, buffer, sizeof(buffer)) >= 0));
  pthread_cleanup_pop(1);
}

The last remaining piece of the thread routine is to signal the main thread using pthread_cond_signal(3), before returning code indicating success, failure or error.

static void *TryRead(void *data) {
  ...

  ret = pthread_cond_signal(&CONDITION);
  if (ret != 0) {
    fprintf(stderr, "Failed to signal waiting thread: %s\n", strerror(ret));
    return (void *)READABLE_ERROR;
  }

  return (void *)((success) ? READABLE_SUCCESS : READABLE_FAILURE);
}

In order for our program to execute, we’ll also create a main function that parses the command line arguments, calls the Readable() function, and outputs the results.


int main(int argc, char *argv[]) {
  time_t timeout = 10;

  int opt;
  char *endptr;
  while ((opt = getopt(argc, argv, "+ht:")) != -1) {
    switch (opt) {
    case 'h':
      printf("%s [-h] [-t TIMEOUT] [FILENAME ...]\n", argv[0]);
      return EXIT_SUCCESS;

    case 't':
      errno = 0;
      timeout = strtol(optarg, &endptr, 10);
      if (errno != 0 || endptr == optarg) {
        fprintf(stderr, "Failed to parse integer '%s'\n", optarg);
        return EXIT_FAILURE;
      }
      break;

    default:
      return EXIT_FAILURE;
    }
  }

  for (int i = optind; i < argc; i++) {
    const char *const filename = argv[i];
    switch (Readable(filename, timeout)) {
    case READABLE_SUCCESS:
      printf("File '%s' is readable.\n", filename);
      break;

    case READABLE_FAILURE:
      printf("File '%s' is not readable.\n", filename);
      break;

    default:
      fprintf(stderr, "An unexpected error occured while checking file '%s'\n",
              filename);
      return EXIT_FAILURE;
    }
  }

  return EXIT_SUCCESS;
}

Testing

The complete source code is available in the GitHub repository:

https://github.com/larsewi/isreadable

Feel free to clone it and test it out.

$ git clone git@github.com:larsewi/isreadable.git
Cloning into 'isreadable'...
remote: Enumerating objects: 34, done.
remote: Counting objects: 100% (34/34), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 34 (delta 15), reused 18 (delta 8), pack-reused 0
Receiving objects: 100% (34/34), 12.23 KiB | 6.11 MiB/s, done.
Resolving deltas: 100% (15/15), done.
$ make
gcc -g -Wall -Wextra -c main.c -o main.o
gcc -g -Wall -Wextra -Wno-int-to-void-pointer-cast -Wno-void-pointer-to-int-cast -c readable.c -o readable.o
gcc main.o readable.o -o readable -pthread
$ ./readable -t100 *.c *.h not-a-file
File 'main.c' is readable.
File 'readable.c' is readable.
File 'readable.h' is readable.
File 'not-a-file' is not readable.

We can also check that the timeout functionality works, by adding a sleep() just before or after the file operations, as sleep(3) is also a cancellation point according to pthreads(7) man page.

static void *TryRead(void *data) {
  ...
  fd = open(filename, O_RDONLY);
  success = ((fd > 0) && (read(fd, buffer, sizeof(buffer)) >= 0));
  sleep(1);
  ...
}

As you can see from the output below, the program times out after approximately 900 milliseconds, deeming the file unreadable. Upping the timeout interval deems the file readable.

$ time ./readable -t900 main.c
File 'main.c' is not readable.
./readable -t900 main.c  0.00s user 0.00s system 0% cpu 0.907 total
$ time ./readable -t1100 main.c
File 'main.c' is readable.
./readable -t1100 main.c  0.00s user 0.00s system 0% cpu 1.007 total

Conclusion

We’ve discussed how to properly check if a file is readable, without potentially blocking indefinitely. Although this program may not be perfect, in that other system calls used in the implementation may block, it definitely reduces the probability. Another point that can be made, is that even though this program correctly concludes that the file is readable, it may not be readable immediately after this decision is made. Thus checking the file before reading it, gives no definitive guarantees.

If you have any questions in regards to this blogpost, or any feedback on how this program could be improved. Please feel free to hit us up on GitHub Discussions, we are eager to hear from you.

Sources

  • Kerrisk, M. (1961). The Linux Programming Interface.
  • Kerrisk, M. et al. (2023). Linux man pages online.

Try CFEngine Enterprise for free

Sign up for CFEngine Enterprise and connect up to 25 hosts for free. No credit card required. Get started in minutes.

Try enterprise for free