RT MIDI

Optimize your system for ultimate performance.

Moderators: MattKingUSA, khz

j_e_f_f_g
Established Member
Posts: 2032
Joined: Fri Aug 10, 2012 10:48 pm
Been thanked: 357 times

Re: RT MIDI

Post by j_e_f_f_g »

raboof wrote:Isn't taking into account timestamps needed to prevent jitter?
Yes, but that's an irrelevant point. Jitter is a problem only if/when a human detects it. Roland did studies that revealed that, even among musicians, humans can't detect timing discrepancies of less than 3 milliseconds. So if your jitter doesn't repeatedly and frequently introduce bigger delays, it's irrelevant in practice. The same is true of latency. So the ideal solution is to get that buffer size as small as possible, and reduce any "DSP processing overhead" to a minimum. And of course that goal is made harder if you're using a I/O layer that, by design, separates the user-space i/o code into another process from the dsp user-space code. Of course, I mean things like jack-midi (people, it's bad enough that Linux has 2 different midi api's, including one -- Alsa Seq -- which suffers from the aforementioned design. Now you want to proliferate another one that does the same? I'll say it right out. This jack-midi shite is ridiculous, and will only make developers regard linux audio/midi as even more fragmented and inconsistent than it already is, thanks to the ridiculous redundancy of too many audio servers. Kill this jack-midi nonsense NOW. Kill it. If you need jack-midi to add midi support to your app, then you don't understand how midi works, much less know how to add support for it. KILL JACK-MIDI! In fact, kill all of jack.)
raboof wrote:If you don't timestamp and just trigger the MIDI audio at the beginning of the next buffer, then MIDI messages that arrive close to the beginning of the previous buffer have a higher latency than MIDI messages that arrive close to the end of the previous buffer.
And in the context of BackupBand, I say "So?". I'm not dissatisfied with my system's MIDI-to-audio performance. I've been using BB on weekly gigs for 8 years now. No one ever complained about timing. Haven't heard that from endusers either. And I hope never to hear someone say:
When i click on BB's chord chart, things play fine. But my midi keyboard is another story. I connect it to my computer via a real 31khz baud interface because I don't trust usb's vastly, vastly, V-A-S-T-L-Y faster transfer rate due to hearing some urban tales about what a huge problem midi jitter supposedly is. And to avoid this great plague, I set my software to use Alsa Seq api (instead of raw midi), which inflicts an unnecessary process context. Then I run it through some "bridge" (daemon that introduces another unnecessary process context) to repackage my midi data into "jack-midi" format. All so I can use software written by some misguided dude who used jack-midi instead of raw midi because he too thinks the former is the right way to deal with jitter and latency. But now my midi gear is giving me problems.
My response will be "Really?".

Silly me. BB doesn't have all those fun jack things to play with like periods, and number of buffers, and bridges. It has only a "buffer size". I set it one day, and I guess I just forgot to worry about jitter and latency during the past 8 years of gigging.
How do you deal with this trade-off?
I don't. Seems I'm missing out on life's great adventures.

Author of BackupBand at https://sourceforge.net/projects/backupband/files/
My fans show their support by mentioning my name in their signature.

merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: RT MIDI

Post by merlyn »

Well ...

I'm guessing you're not keen on JACK MIDI.

So in a practical context, how would I use Ardour and soft synths with a better MIDI system?
User avatar
raboof
Established Member
Posts: 1855
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Has thanked: 50 times
Been thanked: 74 times
Contact:

Re: RT MIDI

Post by raboof »

j_e_f_f_g wrote:
raboof wrote:Isn't taking into account timestamps needed to prevent jitter?
Yes, but that's an irrelevant point. Jitter is a problem only if/when a human detects it.
That is obviously true.
j_e_f_f_g wrote:Roland did studies that revealed that, even among musicians, humans can't detect timing discrepancies of less than 3 milliseconds.
That's interesting, do you have a reference to that research? A quick google (scholar) didn't turn it up for me. When I played with a Yamaha WX-11 though Linux softsynths, I certainly did notice it when I misconfigured my machine and introduced jitter, but I don't remember the numbers.
j_e_f_f_g wrote:So if your jitter doesn't repeatedly and frequently introduce bigger delays, it's irrelevant in practice.
Well, depending on your buffer size it might indeed introduce bigger delays.
j_e_f_f_g wrote:of course that goal is made harder if you're using a I/O layer that, by design, separates the user-space i/o code into another process from the dsp user-space code. Of course, I mean things like jack
Those context switches can indeed in theory be a problem. On the other hand: like jitter, context switches are a problem only if/when a human detects it. While obviously Jack introduces complexity, and for that reason when you don't need it, by all means don't use it, but I have not yet seen a practical instance where a pure-ALSA program outperformed a Jack-based setup.
j_e_f_f_g wrote:Now you want to proliferate another one that does the same?
I never said I wanted to 'proliferate' anything, I just came here for a grown-up discussion on trade-offs in audio software design.
j_e_f_f_g wrote:
raboof wrote:If you don't timestamp and just trigger the MIDI audio at the beginning of the next buffer, then MIDI messages that arrive close to the beginning of the previous buffer have a higher latency than MIDI messages that arrive close to the end of the previous buffer.
And in the context of BackupBand, I say "So?". I'm not dissatisfied with my system's MIDI-to-audio performance.
That's great! For other contexts, things might be different, though, right?
j_e_f_f_g wrote:
raboof wrote:How do you deal with this trade-off?
I don't.
So for your use use of MIDI the jitter introduced by not taking into account the time of arrival of the MIDI message is irrelevant in practice. That's good to know. You could have made that point with a lot less snark, though ;).
j_e_f_f_g
Established Member
Posts: 2032
Joined: Fri Aug 10, 2012 10:48 pm
Been thanked: 357 times

Re: RT MIDI

Post by j_e_f_f_g »

how would I use Ardour and soft synths with a better MIDI system?
In realtime?

Use a midi controller that connects via usb (directly -- no hub). Don't go anywhere near a 5-pin midi DIN connector. If you've got usb ports you never use, see if you can disable them in your BIOS. Ditto with any parallel/serial ports, IEEE 1394 (firewire), internal audio if you're not using it, extra ethernet.

Get rid of any software that describes itself as a "bridge". If it has a "2" in the middle of its filename, eye it suspiciously and snarl "I'm looking at you".

Set your (music) software to use alsa directly (MMAP mode) for audio. Not jack. Not pulse-audio. Not anything else. Set your software to use alsa "raw midi" for midi. Not alsa sequencer. Not jack-midi. Not anything else. If an app doesn't support those, ask its author if his app is designed for realtime input. If he says "no" (or gives a really wishy-washy answer with lots of caveats), try other apps. If he says "yes" tell him he's using the wrong API's. If he starts arguing about it, run like he's Freddy Kruger from "Nightmare on Elm street".

Load all add-ons as plugins (that run in the same process context and address space). Always plugins. Not standalone (unless it directly supports alsa mmap and raw midi).

Author of BackupBand at https://sourceforge.net/projects/backupband/files/
My fans show their support by mentioning my name in their signature.

j_e_f_f_g
Established Member
Posts: 2032
Joined: Fri Aug 10, 2012 10:48 pm
Been thanked: 357 times

Re: RT MIDI

Post by j_e_f_f_g »

raboof wrote:have a reference to that research?
No, but it was done in the 1970's so never published on the internet. In fact, it was internal research Roland R+D did while developing products like the tr-808 and later mpu-401, in order to determine the minimum useable clock rate for the sequencing features of these products. Their engineers did openly discuss it around the time they were trying to get other music manufacturers to adopt their new music "communication protocol" going to used in their mpu-401. The mpu would have been a useless product without widespread music industry support. The mpu was the first IBM PC peripheral of its kind. The "Roland Music Processing Unit 401" was the first computer interface to use this protocol known as MIDI. (But the Yamaha DX-7 was the first MIDI product to market. Everyone and his dog bought one of those. And the rest is midi history). Imagine mpu sales if there hadn't been many midi-capable music instruments? Roland knew they needed widespread adoption of midi. When shopping it around, some engineers being as haughty as they are looked down their noses at midi rate/resolution. (cough...cough... yamaha). They felt it was too low. So roland had to defend their design citing their r+d showing "our music customers are good with this". After a ceremonial delay, yamaha signed on as the last major holdout for midi... Then implemented it incorrectly on the widely sold dx-7, which is why the otherwise useless MIDI Mode 2 had to be added to the spec. But that's another story, and I already hear the snoring.
made that point with a lot less snark
My lawyer tells me that, if I don't actively use my trademark, I could lose it. Blame the lawyers.

Pretend you're brit/aussie, and the humor won't seem so annoying. If you're canadian, wear a blindfold.

Author of BackupBand at https://sourceforge.net/projects/backupband/files/
My fans show their support by mentioning my name in their signature.

Kott
Established Member
Posts: 816
Joined: Thu Mar 21, 2013 12:55 am
Location: Vladivostok
Has thanked: 65 times
Been thanked: 120 times

Re: RT MIDI

Post by Kott »

j_e_f_f_g wrote: Don't go anywhere near a 5-pin midi DIN connector.
What's wrong with them? And I don't want to throw away my BCF1010 :)
j_e_f_f_g
Established Member
Posts: 2032
Joined: Fri Aug 10, 2012 10:48 pm
Been thanked: 357 times

Re: RT MIDI

Post by j_e_f_f_g »

Kott wrote:What's wrong with them?
1980's modem speed.

Author of BackupBand at https://sourceforge.net/projects/backupband/files/
My fans show their support by mentioning my name in their signature.

User avatar
raboof
Established Member
Posts: 1855
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Has thanked: 50 times
Been thanked: 74 times
Contact:

Re: RT MIDI

Post by raboof »

j_e_f_f_g wrote:Roland did studies that revealed that, even among musicians, humans can't detect timing discrepancies of less than 3 milliseconds. (...) it was internal research Roland R+D did while developing products like the tr-808 and later mpu-401, in order to determine the minimum useable clock rate for the sequencing features of these products.
Are you talking about the DIN Sync (https://en.wikipedia.org/wiki/DIN_sync) here? Looks like the MPU-401 had a Pulses-Per-Quarter-Note (PPQN) of 24, so for example at 120 BPM there would be 20 milliseconds between ticks. Are MIDI notes quantized to those ticks, or is it possible for notes to arrive 'in between' the ticks? Are you saying a jitter caused by quantization to 20ms ticks should be undetectable to humans?
Jack Winter
Established Member
Posts: 381
Joined: Sun May 28, 2017 3:52 pm

Re: RT MIDI

Post by Jack Winter »

Just for the context, IIRC it takes approximately 1ms to send a midi on msg (30 bits at 31.25 Kbaud).
Reaper/KDE/Archlinux. i7-2600k/16GB + i7-4700HQ/16GB, RME Multiface/Babyface, Behringer X32, WA273-EQ, 2 x WA-412, ADL-600, Tegeler TRC, etc 8) For REAPER on Linux information: https://wiki.cockos.com/wiki/index.php/REAPER_for_Linux
CrocoDuck
Established Member
Posts: 1133
Joined: Sat May 05, 2012 6:12 pm
Been thanked: 17 times

Re: RT MIDI

Post by CrocoDuck »

Not sure whether it is relevant, but I reviewed a 2007 article about latency perception here on my blog. I don't know if perception is at all similar for jitter, but as far as latency goes latency can be detected by humans starting from 2 - 3 ms. The study finds lower thresholds, but data might be unreliable due to small sample size.
User avatar
raboof
Established Member
Posts: 1855
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Has thanked: 50 times
Been thanked: 74 times
Contact:

Re: RT MIDI

Post by raboof »

CrocoDuck wrote:Not sure whether it is relevant, but I reviewed a 2007 article about latency perception here on my blog. I don't know if perception is at all similar for jitter, but as far as latency goes latency can be detected by humans starting from 2 - 3 ms. The study finds lower thresholds, but data might be unreliable due to small sample size.
My subjective experience is that playing a softsynth I was much more sensitive to jitter than to constant latencies.
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: RT MIDI

Post by merlyn »

j_e_f_f_g wrote:Set your (music) software to use alsa directly (MMAP mode) for audio. Not jack. Not pulse-audio. Not anything else. Set your software to use alsa "raw midi" for midi. Not alsa sequencer. Not jack-midi.
That could be called 'bare metal MIDI'. If I were to set that up, it would involve hosing half my system.

The word 'JACK' in a post is usually followed by fuming and screaming about how difficult it is to set up, and how it doesn't work, and how it should be obliterated from the face of the Earth.

I like JACK. So far, I have needed it because I use different applications at the same time.

I haven't done anything that used a lot of MIDI so I don't know if JACK MIDI is an issue or not. This thread has made me want to look at MIDI on my system in more detail.

j_e_f_f_g wrote that it's a pain to create applications using JACK. That's less than ideal, but does the end user see any improvement in performance by using bare metal MIDI?
tramp
Established Member
Posts: 2328
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 450 times

Re: RT MIDI

Post by tramp »

merlyn wrote:j_e_f_f_g wrote that it's a pain to create applications using JACK.
I ain't have read this thread fully, so I don't know were he wrote that, but, beside that jeff is known here to be a jack hater, that isn't true.
From the developers pov, it's much more easy to implement jack-midi then raw alsa.

However, well known is as well, that jack-midi never was done for interact with "hardware synthesizers". For hardware midi handling you indeed better use raw alsa.
That changed, when interaction with software midi devices steps up in the game.
jack-midi is handled in the realtime-audio thread of jack. Developers needs to be aware that anything not real-time save shouldn't be done in the jack_process_callback, regardless if it is midi or audio. You may imagine that a couple of VSTi plugs ain't have a clue about this.

jack-midi does exactly what jeff proposed, no timestamp, plain messages handled as they comes in.
I must admit that I use midi only as a controlling interface, so jitter ain't matter for me, but, for my use case, jack-midi is the best, and painless choice for me and the users.
So if you would ask me if I would support plain raw alsa, I would answer "No".

regards
hermann
On the road again.
User avatar
raboof
Established Member
Posts: 1855
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Has thanked: 50 times
Been thanked: 74 times
Contact:

Re: RT MIDI

Post by raboof »

tramp wrote:jack-midi does exactly what jeff proposed, no timestamp, plain messages handled as they comes in.
Actually the timestamp/frame of the incoming MIDI message is recorded (http://jackaudio.org/api/struct__jack__midi__event.html, https://github.com/jackaudio/jack2/blob ... r.cpp#L187), but indeed it is up to you as the library user to decide whether you want to ignore it (more jitter) or take it into account (more latency).
tramp
Established Member
Posts: 2328
Joined: Mon Jul 01, 2013 8:13 am
Has thanked: 9 times
Been thanked: 450 times

Re: RT MIDI

Post by tramp »

raboof wrote:Actually the timestamp/frame of the incoming MIDI message is recorded (http://jackaudio.org/api/struct__jack__midi__event.html, https://github.com/jackaudio/jack2/blob ... r.cpp#L187), but indeed it is up to you as the library user to decide whether you want to ignore it (more jitter) or take it into account (more latency).
Your interpretation is wrong here, this isn't a timestamp, this is the "stamp" in relation to the audio sample (the single float position in a audio sample frame). So, all you've to do is as for audio or for midi, work on the incoming data on sample base. eg:

Code: Select all

for (i = 0; i < event_count; i++) {
              process_midi_event[i]
}
to process incoming events sample accurate, without jitter. The main issue is indeed the "conversation", or "bridge" from alsa to jack. That's were jitter may be introduced.
On the road again.
Post Reply