The future of Catch2

I am planning to create a branch for the next major version of Catch2 soon, and doing so brings some questions about Catch2's future.

Distribution model

Our current distribution model that is providing just a single header file for the users to include in their project has been good to us. It is, without doubt, one of the reasons for Catch2's immense popularity, because it lets users get started with Catch2 quickly, without any headache with different build systems, setting up proper linking steps and so on. This ability to "quick start" the use of Catch2 also makes it a valuable tool for teaching, because it takes just a single file for your students to have access to a capable unit testing framework.

However, distributing just a single header file, that your user compiles as part of their project, has many disadvantages. First, it complicates using Catch2 via package managers, such as vcpkg and Conan. When you use a classical library via a package manager, you have to tell your package manager to install it, tell your build system to use it (e.g. with target_link_libraries in CMake) and that is it, you can now use the library from your own code. When you use a Catch-style single-header library via a package manager, you add an extra step, because you need to add another .cpp file to your project, where the implementation of the single-header library will live.

Another problem of the single-header distribution model for Catch2 is the compilation time. Every feature we add increases the compilation times, even for people who do not use this feature. Recently, I finalized generator support in Catch2, and I am planning to add some utility generators in the next release so that our users won't have to implement them themselves. However, the increase in compilation times will be paid by all of our users, even those who don't use the generator feature. And it is not just our code, implementing a RandomIntGenerator will drag in the <random> header and everything it transitively depends on.

We try to avoid this problem by providing configuration macros that let you disable/enable parts of Catch2, such as the CATCH_CONFIG_ENABLE_*_STRINGMAKER configuration macros, but this is annoying, doesn't scale well and is less discoverable than just providing these features in separate headers. I've already rejected some utility additions (e.g. specific Matchers), because they were unlikely to be used by most people, but would impose compilation costs on everyone.

The future

Recently I made a twitter poll to see what people think about going forward with a classical library model for Catch2 and the answers were mostly positive. I also talked with some people in different venues, and they pointed out that the single header version is valuable for beginners and teaching^[1]. To this end, I am thinking about going forward with a hybrid approach, where the single-include version will still exist, but it will be cut down^[2], and our documentation and examples will primarily cover using the full Catch2 library distribution.

Maintainership

Keeping up with Catch2 needs a lot of time. I took a 2-week break from Catch2 to finish up the Winter semester^[3], and when I checked yesterday, GitHub told me that that I have more than 70 notifications waiting for me.

For some extra numbers, when I started working on Catch2 in 2017, there were some 300 open issues, with new issues getting numbers <800. After these 2 years, there are ~150 open issues, and the latest issue has number 1543. In other words, we addressed ~800 issues and PRs^[4] over the last 2 years, where "we" means, for the most part^[5], Phil Nash and I. However, Phil hasn't been really active much during the last year and the work of keeping Catch2 updated, issues handled and PRs responded-to, takes a ton of time, especially for a free-time activity.

Over the last 2 years, I also lost a lot of my motivation for updating Catch2. Most of the problems I've had with it in the past have been fixed and the features I wanted the most have been implemented. Nowadays, Catch2 has to compete for my free time with these two

The future

My current plan is to keep Catch2 going until the next major version is released. The current code contains many little annoying problems that cannot be fixed without breaking backwards compatibility, which means that they had to wait until a major release. After these are fixed, ¯\_(ツ)_/¯.^[6]

Planned changes

Apart from the various fixes to current behaviour that you can find in our documentation, there are 3 largish changes that I would like to get into the next major release:

1. Large scale rewrite of internal data structures to improve performance

The current internals of Catch2 were optimized for simplicity and correctness, and carry a lot of evolutionary cruft that accumulated over time -- as an example, registering a test case is likely to allocate 10 or more times. While I don't think Catch2's internals should be savagely optimized, I want them to be at least "kinda optimized".

2. Dropping support for older compilers (and C++11)

We currently have workarounds for VS2015, GCC4.8 and similar oldish compilers, that I would like to drop support for. C++14 does not bring that many improvements over C++11, but it will still let us perform some internal cleanups, and it brings us 1 step closer to C++17, which would actually provide us with significant benefits.

3. Thread safe assertions

This does not mean running the tests in multiple threads^[7], but rather that multiple threads can run through an assertion like REQUIRE at the same time. The problem with this feature is that having it on by default would pessimize Catch2's performance for users who do not use multiple threads in that way, which are most of them. On the other hand, having a compile-time configuration that modifies whether assertions are thread-safe would increase the implementation complexity significantly.

The future

The first two significant changes will definitely happen. The third one might.

If you want to discuss Catch2's future with others, you should come to our Discord.