Pluginval is a cross-platform open source app from Tracktion.
Think of it as quality control for your plugin. It loads your audio plugin as a DAW would, runs myriad tests against it, and reports any problems.
Whether you are having problems with your plugin failing DAW validation or just want to catch issues before your users do, it’s worth learning how to use!
Here are a few tips on working with pluginval:
Pluginval Tip 1: Compile locally
You could download and run the latest release from GitHub — but in order to efficiently debug and resolve issues that pluginval unearths, you’ll want to run it in a debugger.
Pluginval Tip 2: Run it in CI
You’ll probably want to run pluginval continually while developing a plugin to proactively catch potential problems you are introducing. Otherwise, problems will bunch up and you won’t be quite sure when they were introduced, making resolution more troublesome.
There are some straightforward docs about running in CI, which boils down to sucking down the binary and running it against your built plugin.
You can see a real life config example my Pamplejuce repo which runs pluginval on GitHub Actions.
Pluginval Tip 3: “Validate in process” to catch asserts and set breakpoints
It can feel a bit nerve wracking the first few times you run into your first
FAIL. What exactly failed? How should you resolve it?
The process to resolve a failure is:
- Run pluginval from your IDE with a debugger
- Make sure “Validate in process” is checked so that the debugger can “see through” to your code from pluginval. (If you are on the command line, specify
- Place breakpoints in pluginval itself. For example, search the source for
testRunner.setAssertOnFailure (false)and set it to
- Inspect pluginval’s code and the state of your plugin to resolve the issue.
Pluginval Tip 4: Specify a seed to reproduce issues
If you want to reproduce a problematic run, grab the random seed that’s output at the start of the log:
Started validating: AudioUnit-MyPlugin-1682755d-61203125 Random seed: 0x374115a
You can then feed this seed to pluginval as an option, ensuring the exact same set of tests are run. This is especially useful when reproducing an intermittent problem on CI.
Common Problem: No types found
Starting test: pluginval / Scan for known types: /home/runner/work/myplugin/Builds/MyPlugin_artefacts/Release/VST3/MyPlugin.vst3... Num types found: 0 !!! Test 1 failed: No types found
This translates to “We couldn’t find your plugin.”
Double check the following:
- The plugin is actually built and your plugin exists.
- The path is correct.
- Does your plugin .vst/.component have spaces in it? If you are using JUCE, be aware that the name of the plugin is set by
PRODUCT_NAMEwhich allows spaces.
I’ve been bitten by all 3 of these problems!
Common FAIL: Not destroying listeners
Pluginval catches all sorts of issues. Here’s a popular one in JUCE: parameter listeners being referenced after they are destroyed.
This usually boils down to “you forgot to call
removeListener in a Component’s destructor”.
It will show up as a memory-related complaint where listeners are called, for example in AudioProcessorValueTreeState.cpp‘s implementation of
Is Pluginval enough?
Occasionally, a plugin will fail validation in a DAW like logic, but pass a strictness level 10 with pluginval. Ideally, those test cases are then identified and contributed back to pluginval itself.
On MacOS, you can use apple’s
auval tool on the command line to first get more detail on the failure.
To get more detail than what logic shows you, run
auval -v in Terminal.
auval -v TYPE SUBT MANU
aufx for an audio effect or
aumu for an audio instrument.
SUBT is what JUCE refers to as your 4 character
MANU is what JUCE refers to as your 4 character `PLUGIN_MANUFACTURER_CODE`
Here are a couple examples:
aumu Vita Tyte - Matt Tytel: Vital aufx TP3X GDHZ - Goodhertz: Ghz Tupe 3
I’ve never had to deal with this. If you get stuck, here’s a JUCE forum post for you.