Audio to midi

All your LV2 and LADSPA goodness and more.

Moderators: MattKingUSA, khz

thebutant
Established Member
Posts: 169
Joined: Sun Nov 18, 2012 12:59 pm
Has thanked: 50 times
Been thanked: 8 times

Audio to midi

Post by thebutant »

This is a way of working I do quite often and really enjoy:

Playing the guitar -> connecting it to Rakarrack -> muting everything guitar sounding, only using the midi converter -> connecting these midi signals to a synth or something else midi -> voila! my guitar is a synth!

But I meet a couple of issues along the way.

1/ When using Rakarack this way, I'm not able to make the midi signals release, they get stuck. This means I cannot use long lasting synths like a pad or something, as each tone will just be playing forever. It also means recording the midi gets really messy - every new note goes into a giant cluster.
2/ The Rakarrack audio to midi converter is (as far as I understand) not sensitive for velocity. So it's difficult to play with dynamics, and easily gets a bit flat sounding.

So my questions are:
Is there a better way for converting audio to midi? Maybe even velocity sensitive?
In Rakarrack - why won't the midi signals release, is there a way to fix this?
User avatar
autostatic
Established Member
Posts: 1994
Joined: Wed Dec 09, 2009 5:26 pm
Location: Beverwijk, The Netherlands
Has thanked: 32 times
Been thanked: 104 times
Contact:

Re: Audio to midi

Post by autostatic »

You might want to try the GxTuner LV2 plugin from the Guitarix suite, it also has audio to MIDI functionality.
thebutant
Established Member
Posts: 169
Joined: Sun Nov 18, 2012 12:59 pm
Has thanked: 50 times
Been thanked: 8 times

Re: Audio to midi

Post by thebutant »

AutoStatic wrote:You might want to try the GxTuner LV2 plugin from the Guitarix suite, it also has audio to MIDI functionality.
Thanks so much for that tip! I didn't know.
Nevertheless, after playing around with it, I do find Rakarrack to work better: Quicker, more precise, easier to play with and more configurable.

Ok, so now I know of 2 programs converting audio to midi. But they're both guitar applications, they're both mainly doing something else. The converting part is more of a bonus than a main feature.
Still the code is written, the task is achievable.
So isn't it almost strange if there's no program focusing mainly on the task of converting audio to midi - a small program doing it even better and smoother (preferably even sensitive for velocity) than two programs focusing on guitar sound?

I don't know of any such program, but if someone else does - I'd be delighted to hear about it!
Or if somebody got inspired and even wants to make it - ooh la la!
gimmeapill
Established Member
Posts: 564
Joined: Thu Mar 12, 2015 8:41 am
Has thanked: 44 times
Been thanked: 8 times

Re: Audio to midi

Post by gimmeapill »

Adding a third one:
https://github.com/geraldmwangi/GuitarSynth-DPF

I'm also interested in the matter, but I don't think a software only solution can currently provide good enough performance and accuracy (at least in the current state of technology). You will probably need dedicated hardware like a Roland GR-33 to get something really playable.

The question was already discussed a few times in the past:
viewtopic.php?f=48&t=16223&p=74804&hilit=gxtuner#p74804

https://www.linuxmusicians.com/viewtopic.php?t=11561

User tavasti would probably tell you far more - he's got both a Roland GR-33 and a You Rock Guitar ;-)
ssj71
Established Member
Posts: 1294
Joined: Tue Sep 25, 2012 6:36 pm
Has thanked: 1 time

Re: Audio to midi

Post by ssj71 »

this is a hard problem to solve using general computing processors. But with that said, you can also look at Aubio. https://aubio.org/ It is a dedicated audio processing library including audio to midi conversion.
I've never played with it myself, but its on my list.
_ssj71

music: https://soundcloud.com/ssj71
My plugins are Infamous! http://ssj71.github.io/infamousPlugins
I just want to get back to making music!
thebutant
Established Member
Posts: 169
Joined: Sun Nov 18, 2012 12:59 pm
Has thanked: 50 times
Been thanked: 8 times

Re: Audio to midi

Post by thebutant »

gimmeapill wrote:Adding a third one:
https://github.com/geraldmwangi/GuitarSynth-DPF
Lovely!
Thanks. Look forward to trying that one out.
ssj71 wrote:you can also look at Aubio. https://aubio.org/ It is a dedicated audio processing library including audio to midi conversion.
Also great, I've never tried Aubio.

And yes, a hardware tool will probably do the job better than software does. But to be honest, I'm not really looking for the midi guitar experience, I really enjoy the glitches and small accidents that occur using for example Rakarrack. Unpredictable is often quite interesting to play with, in my opinion. So the Rakararrack experience, but with releasing midi notes + velocity - that's my dream.

Thanks for great help! And also for these interesting discussions:
gimmeapill wrote:The question was already discussed a few times in the past:
viewtopic.php?f=48&t=16223&p=74804&hilit=gxtuner#p74804
https://www.linuxmusicians.com/viewtopic.php?t=11561
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: Audio to midi

Post by khz »

@gxtuner
Image
Play with the knobs :wink:
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
glowrak guy
Established Member
Posts: 2315
Joined: Sat Jun 21, 2014 8:37 pm
Been thanked: 251 times

Re: Audio to midi

Post by glowrak guy »

I used lin-vst for wrapping an older version
of Jam Origens Midi Guitar plugin, and it's
working pretty well using MDA JX10 synth.
I can play 1/8th notes, without changing
the default settings.

The JX10 plugin controls can be accessed,
and presets selected, from within Jam Origen gui.
I'm using a Fender Mustang usb amp for input,
but didn't turn off the cab-sim for testing,
which would yield a cleaner tone for conversion.

There is a free trial you can download,
but it's the standalone version:

https://www.jamorigin.com/download/

It defaults to a piano tone, but has a selector for
synths on your system.

Make a set of strings that minimizes string gauge differences.
and tune it a note or two higher than E, which should help
the accuracy of the conversion.
Cheers
glowrak guy
Established Member
Posts: 2315
Joined: Sat Jun 21, 2014 8:37 pm
Been thanked: 251 times

Re: Audio to midi

Post by glowrak guy »

The other MDA instruments also work in Jam Origen. Mallet sounds from he DX 10
are fun to use with a Rakarrak or three, to make each pluck do a little dance routine :wink:
Cheers
tavasti
Established Member
Posts: 2047
Joined: Tue Feb 16, 2016 6:56 am
Location: Kangasala, Finland
Has thanked: 369 times
Been thanked: 208 times
Contact:

Re: Audio to midi

Post by tavasti »

gimmeapill wrote: I'm also interested in the matter, but I don't think a software only solution can currently provide good enough performance and accuracy (at least in the current state of technology). You will probably need dedicated hardware like a Roland GR-33 to get something really playable.

The question was already discussed a few times in the past:
viewtopic.php?f=48&t=16223&p=74804&hilit=gxtuner#p74804

User tavasti would probably tell you far more - he's got both a Roland GR-33 and a You Rock Guitar ;-)
You Rock Guitar has been unused for me. I have seen videos where it is played by proper player, and it sounds nice, but at least my hammer-on & strumming timing accuracy is too bad for playing it live. Recording midi from it, and fixing timings on DAW should work. I need to test it bit more.

Roland guitar synths are fine for creating something to backup your guitar playing, or playing something slow & smooth. For faster playing (or with lousy player like me) there will be sometimes wrong notes added. For example, this https://www.youtube.com/watch?v=fjxy_NKsHec is live recording, GR-33 playing strings, and there is some wrong notes here and there, for example in 13s. Player with perfect playing skills might not get those. I have handled it with adjusting patch to have longer attack, so that fast wrong note will not be that loud.

I've used it for recording something in midi, and after it, removing extra notes in DAW. (And fixinging timing caused by bad player)

I tend to think that you can't replace keyboards with guitar for live playing, but for recording midi it can be usable solution.

Edit: You Rock Guitar has some settings which can make it more usable, I need to experiment more with it
Last edited by tavasti on Sat Oct 07, 2017 7:36 am, edited 2 times in total.

Linux veteran & Novice musician

Latest track: https://www.youtube.com/watch?v=ycVrgGtrBmM

glowrak guy
Established Member
Posts: 2315
Joined: Sat Jun 21, 2014 8:37 pm
Been thanked: 251 times

Re: Audio to midi

Post by glowrak guy »

tavasti wrote: I tend to think that you can't replace keyboards with guitar
I agree. The strength of midi guitar will be the guitar players knowledge and use of
familiar note layouts and progressions, with wonderful synth sounds,
embellished by great effects chains. There are so many delay and modulation combos
that one can easily extend 1/8th and 1/4 notes, creating music that is delightful,
and utterly impossible with guitar tones, (as varied and enjoyable as they may be).
Midi guitar can be a great tool to add to one's arsenal.

Cheers
User avatar
English Guy
Established Member
Posts: 525
Joined: Wed Oct 17, 2012 7:28 pm
Location: England
Has thanked: 8 times
Been thanked: 7 times

Re: Audio to midi

Post by English Guy »

I play synth with guitar using aubio tools and have the same problem with the midi notes not ending. The workaround is to produce a 'click' on a muted string that ends the note without starting a new one.
Luc
Established Member
Posts: 741
Joined: Fri Mar 27, 2015 1:04 pm
Been thanked: 1 time

Re: Audio to midi

Post by Luc »

thebutant wrote:1/ When using Rakarack this way, I'm not able to make the midi signals release, they get stuck.
English Guy wrote:I play synth with guitar using aubio tools and have the same problem with the midi notes not ending. The workaround is to produce a 'click' on a muted string that ends the note without starting a new one.
Perhaps you can hack your midi signals with one of these plugins:
http://thepiz.org/plugins/?p=pizmidi
User avatar
English Guy
Established Member
Posts: 525
Joined: Wed Oct 17, 2012 7:28 pm
Location: England
Has thanked: 8 times
Been thanked: 7 times

Re: Audio to midi

Post by English Guy »

Luc wrote:
thebutant wrote:1/ When using Rakarack this way, I'm not able to make the midi signals release, they get stuck.
English Guy wrote:I play synth with guitar using aubio tools and have the same problem with the midi notes not ending. The workaround is to produce a 'click' on a muted string that ends the note without starting a new one.
Perhaps you can hack your midi signals with one of these plugins:
http://thepiz.org/plugins/?p=pizmidi
& already on my system, must be thanks to KX Studio :D
oddy.o.lynx
Established Member
Posts: 115
Joined: Sat Jun 30, 2018 2:54 am
Has thanked: 22 times
Been thanked: 22 times

Re: Audio to midi

Post by oddy.o.lynx »

I am able to get decent results using Sonic Visualiser and combinations of Vamp Aubio plugins, MELODIA - Melody Extraction and Silvet Note Transcription.

I have a bunch of old recordings and musical musings that I wanted to explore and it's not always easy to figure out what I was playing back then. Using these tools I have had much success, even figuring out complex/diatonic chords that have up until now eluded me.

The best extractor for the final result is Silvet Note Transcription. Sonic Visualiser can accurately convert audio to midi using this plug in. You have to tinker with the settings and analysis to get best results. Adding a few decibels to the audio helps significantly.
Post Reply