August 2022 update✨: I teamed up with Dave Rowland from Tracktion and The Audio Programmer to release ValidateMyPlugin.com — the easiest way to get started!
Pluginval is a cross-platform open source tool by Tracktion.
Think of it as quality control for your audio plugin. It loads your plugin as a DAW would, runs myriad tests against it, and reports any problems.
Whether you are having problems with your plugin failing DAW validation or just want to catch issues before your users do, it’s worth learning how to use!
Here are a few tips on working with pluginval:
Pluginval Tip 1: Compile locally
To get started, try validating your plugin on ValidateMyPlugin.com or download and run the latest release from GitHub.
However, to efficiently debug and resolve issues that pluginval unearths, you’ll need to start to running it locally in a debugger.
So build and run the project locally.
Pluginval Tip 2: Run it in CI
You’ll probably want to run pluginval continually while developing a plugin to proactively catch potential problems you are introducing. Otherwise, problems will bunch up and you won’t be quite sure when they were introduced, making resolution more troublesome.
There are some straightforward docs about running in CI, which boils down to sucking down the binary and running it against your built plugin.
You can see a real life config example my Pamplejuce repo which runs pluginval on GitHub Actions.
Pluginval Tip 3: Catch asserts and set breakpoints
It can feel a bit nerve wracking the first few times you run into your first
FAIL. What exactly failed? How should you resolve it?
The process to resolve a failure is:
- Run pluginval from your IDE with a debugger
Make sure “Validate in process” is checked so that the debugger can “see through” to your code from pluginval(No longer needed in pluginval 1.0 and higher)
- Place breakpoints in pluginval itself. For example, search the source for
testRunner.setAssertOnFailure (false)and set it to
- Inspect pluginval’s code and the state of your plugin to resolve the issue.
Pluginval Tip 4: Specify a seed to reproduce issues
Pluginval uses randomness throughout the code base (and optionally the test order). This means that you might see intermittent failures and slightly different results on each run.
If you want to reproduce a problematic run, grab the random seed that’s output at the start of the log:
Started validating: AudioUnit-MyPlugin-1682755d-61203125 Random seed: 0x374115a
You can then feed this seed to pluginval as an option, ensuring the exact same set of tests are run. This is especially useful when reproducing an intermittent problem on CI.
Troubleshooting Pluginval: No types found
!!! Test 1 failed: No types found. This usually means the plugin binary is missing or damaged, an incompatible format or that it is an AU that isn't found by macOS so can't be created.
Unfortunately, this error can mean a lot of things.
It boils down to”We couldn’t find your plugin” but there are many reasons for this, especially on macOS.
Verify the following:
- The plugin is actually built.
- The path to your plugin is correct.
- Does your
.componenthave spaces in it? (If you are using JUCE, be aware that the name of the plugin is set by
PRODUCT_NAMEwhich does allow spaces).
- Are you trying to validate a
.componenton macOS? You’ll need to make sure the plugin is picked up by the Audio Unit registry by moving or copying it to
~/Libraryversion). You might want to run
killall -9 AudioComponentRegistrarto kick the registrar to pick it up. Why? Like all JUCE AU hosts, pluginval opens the
.plistand gets the component ID out of that to load it up.
- Are you on macOS? Is it built as a universal binary? Be aware that you can see this error if you have fed pluginval a binary that is on a different architecture than the system you are running on.
I’ve been bitten by all of these problems!
That last issue is more esoteric, but you might encounter it if the plugin you are validating came from a different machine, for example if you are validating plugins in CI. You can check what architectures your plugins support with the
➜ ~ file /Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate /Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit bundle x86_64 - Mach-O 64-bit bundle x86_64] [arm64] /Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate (for architecture x86_64): Mach-O 64-bit bundle x86_64 /Library/Audio/Plug-Ins/Components/ValhallaPlate.component/Contents/MacOS/ValhallaPlate (for architecture arm64): Mach-O 64-bit bundle arm64
Common Validation FAIL: Not destroying listeners
Pluginval catches all sorts of common pitfalls. Here’s a popular one in JUCE: parameter listeners being referenced after they are destroyed.
This usually boils down to “you forgot to call
removeListener in a Component’s destructor”.
It will show up as a memory-related complaint where listeners are called, for example in AudioProcessorValueTreeState.cpp‘s implementation of
Is Pluginval enough?
Occasionally, a plugin will fail validation in a DAW like logic, but pass a strictness level 10 with pluginval. Ideally, those test cases are then identified and contributed back to pluginval itself.
Since version 1.0,
pluginval will now run auval for you at strictness levels greater than 5.
On MacOS, you can also apple’s
auval tool on the command line to first get more detail on failures.
To get more detail than what logic shows you, run
auval -v in Terminal.
auval -v TYPE SUBT MANU
aufx for an audio effect or
aumu for an audio instrument.
SUBT is what JUCE refers to as your 4 character
MANU is what JUCE refers to as your 4 character
Here are a couple examples:
aumu Vita Tyte - Matt Tytel: Vital aufx TP3X GDHZ - Goodhertz: Ghz Tupe 3
I’ve never had to deal with this. If you get stuck, here’s a JUCE forum post for you.
Leave a Reply