Adding Tests and Benchmarks

Out of the box, Pamplejuce builds a Tests target and a Benchmarks CMake target, powered by Catch2.

It also provides you some examples and helpers.

Adding your first Tests

To add new tests:

• Create a new .cpp file in the /tests directory. It will automatically be included next build.
• Include whatever Catch2 headers you need in your new test file. The most basic is #include <catch2/catch_test_macros.hpp>.
• Go to town!

Pamplejuce has a custom Catch2 main function that starts JUCE’s ScopedJuceInitialiser_GUI before running tests. This greatly simplifies and facilitates running tests that touch the message thread. Without it, you’ll get memory leaks when interacting with message thread methods.

Catch2 Tips

• Using REQUIRE will halt execution for the SECTION if it fails. I tend to use CHECK anytime I have multiple assertions in a row. Best practices are to have only 1 “assertion” (CHECK or REQUIRE) per SECTION but sometimes it makes sense to have a couple together.
• You can (and should!) nest SECTION blocks. In Catch, this how you specify “test setup” for a series of sub-tests (or you can have setup for all assertions inside the TEST_CASE before the SECTION blocks)
• If you want to use “or” conditions in assertions, you’ll need an extra pair of parenthesis inside the CHECK macro. For example, to check something is a true square wave (all values are 0 or 1) it’ll look like this: CHECK ((sample == Catch::Approx (0) || sample == Catch::Approx (1)));
• You can happily add plain C++ functions for test helpers at the top of the file. For example, the following helper lets you fill an AudioBlock concisely with fill_block (wavetable, { 10, 100, 50, 0, -50 }):

void fill_block (juce::dsp::AudioBlock<float>& block, const std::vector<float> values)
{
    jassert (block.getNumSamples() >= values.size());
    for (int i = 0; i < values.size(); ++i)
    {
        block.setSample (0, i, values[i]);
    }
}

Testing Floating Point values

Floating point values are a… thing. It’s really easy to write brittle tests that will fail cross platform or fail because of float inaccuracies.

In JUCE, you can always use juce::approximatelyEqualTo.

I lazily use Catch::Approx for easy stuff like CHECK (myValue == Catch::Approx (1.0f)).

You can specify explicit margins like so for more complex dsp stuff: Catch::Approx (0.16667f).margin (0.0001f)).

Read the Catch2 floating point docs, the Catch2 maintainer’s deep dive, or check out Julia’s amazing PDF for more.

Debugging Catch2 tests in LLDB

This can be noisy unless you put the following into you ~/.lldbinit file:

settings set target.process.thread.step-avoid-regexp Catch|_catch_sr
settings set target.process.thread.step-out-avoid-regexp "Catch|_catch_sr"

You can create the file if it doesn’t yet exist.

Examples & guidance on writing tests and matchers

• I have some very easy to understand Catch2 tests around JUCE NormalisableRange’s here.
• Check out my custom JUCE AudioBlock matchers for an example of how to write matchers.

I also have a blog article about when to test dsp code.