Image-based sound synthesizer?
Posted: Tue Jun 27, 2017 8:42 am
I have an idea for an LV2 plugin, that'll synthesize sounds based on input images, treating them as spectrograms.
Basically, the reverse of generating a spectrogram, in a DAW-friendly way, that'd also allow the users to play musical notes (pitching the synthesized result according to MIDI input).
That'd be an image-based synthesizer. I'd rather thing of this a s a tool for sound effects design, but coudl als obe used in music.
Right now the only tool I've found in the Linux audio world that can synthesize sound from images is ARSS, and it's a bit of a pain in the ARSS to use it
To synthesize a sound from an image you have to:
1. Download ARSS binary (https://sourceforge.net/projects/arss/)
2. Prepare the image
3. Save your image it as a specific BMP format, becasue it's the only one supported by ARSS
4. Compose a shell command to synthesize a WAV file from your BMP file
5. Run ARSS
6. Listen
7. Tweak the commandline parameters and repeat from 4. , or tweak the image and repeat from 2.
8. You now have to use a sampler plugin or another way of putting that WAV file into your DAW session.
I wish there was a DAW-friendly way of doing this, like
using an LV2 plugin that can just load an image file, have some knobs I can tweak live. It'd be fantastic, if the sound synthesis could happen in realtime, because that'd make it possible to automate some parameters, and have that actually reflected in the sound.
ZynAddSubFX's PADsynth generates samples under the hood (and it works as an LV2 plugin) so I guess that's one approach. I think that Image-Line Harmor synthesizes sound from images on the fly (and does a lot of other addtive synthesis awesomness, but that's not what I'm looking for).
What do you think?
Basically, the reverse of generating a spectrogram, in a DAW-friendly way, that'd also allow the users to play musical notes (pitching the synthesized result according to MIDI input).
That'd be an image-based synthesizer. I'd rather thing of this a s a tool for sound effects design, but coudl als obe used in music.
Right now the only tool I've found in the Linux audio world that can synthesize sound from images is ARSS, and it's a bit of a pain in the ARSS to use it
To synthesize a sound from an image you have to:
1. Download ARSS binary (https://sourceforge.net/projects/arss/)
2. Prepare the image
3. Save your image it as a specific BMP format, becasue it's the only one supported by ARSS
4. Compose a shell command to synthesize a WAV file from your BMP file
5. Run ARSS
6. Listen
7. Tweak the commandline parameters and repeat from 4. , or tweak the image and repeat from 2.
8. You now have to use a sampler plugin or another way of putting that WAV file into your DAW session.
I wish there was a DAW-friendly way of doing this, like
using an LV2 plugin that can just load an image file, have some knobs I can tweak live. It'd be fantastic, if the sound synthesis could happen in realtime, because that'd make it possible to automate some parameters, and have that actually reflected in the sound.
ZynAddSubFX's PADsynth generates samples under the hood (and it works as an LV2 plugin) so I guess that's one approach. I think that Image-Line Harmor synthesizes sound from images on the fly (and does a lot of other addtive synthesis awesomness, but that's not what I'm looking for).
What do you think?