As a package maintainer, I am overwhelmed by the idea of making sure all plugins work after they and/or their dependencies are updated.
The plugins' own unit tests (when present, which is not often) aren't 100% reliable for testing: the plugins can always crash when loaded into a host.
Some days ago I came across a fuzzer for the Godot libre game engine (Qarminer) and that might have influenced me to come up with this idea: why not use a fuzzer for the major plugins technologies (VST, VST3 and LV2) at the very least to discover internal crashes?
After all plugins should be an ideal setting for fuzzing: they are data-oriented, they have a bunch of discoverable parameters which are supposed to be automated, they are written using 3 major standardized interfaces.
Call this a research thread, mostly to put on (totally not) paper ideas from more skilled developers than myself.
- Are there any libre and open source plugin fuzzers around?
- How would you go about implementing a fuzzer targeting all major plugin technologies (LV2/VST/VST3)? (I thought about leveraging carla-devel)
- What do you think would be the most problematic aspects of implementing a fuzzer?