New! Comprehensive latency correction / compensation

MusE is a MIDI/Audio sequencer with recording and editing capabilities, aiming to be a complete multitrack virtual studio for Linux. http://muse-sequencer.org/

Moderators: khz, MattKingUSA, spamatica

Tim E. Real
Established Member
Posts: 277
Joined: Sat Sep 15, 2012 12:36 am

New! Comprehensive latency correction / compensation

Postby Tim E. Real » Sat Jun 29, 2019 8:25 am

Psst! I've merged my latency_fixes branch into master!

Currently you need to enable the feature in Global Settings > Latency tab.
Should be OK to start usage trials. Bear with me, it needs a full README etc.
Still a few more things to do. I'll try to include a test wav file for input/output and send/return, and a test song file
so that you can be confident that your setup works and everything is aligned.

Note: There is usually a further small unaccounted-for latency in any real internal or external hardware.
The program jack_iodelay is your friend.
Take it's reported value and enter it into QJackCtl or on the command line.
It would be good to add this functionality in MusE somehow... And... for midi-to-midi and midi-to-audio (intelligent with visual assistance)

Note: As mentioned here:
https://github.com/muse-sequencer/muse/issues/648
when you load a song and press play you may see terminal messages like this:

Code: Select all

FIFO 0xda8d600 underrun
WaveTrack::getData(Playback track) (A) fifo underrun

It's an old bug. Follow the very easy workaround 'till I fix it.
[EDIT] Fixed in git master now! Yay. :D
Caveat: If you are 'looping' the messages might still occur. Working on it, know the problem...

That's all I can type for now. Cheers.
Tim.

From ChangeLog:

Code: Select all

      * New! Comprehensive full midi and audio latency correction / compensation:
      - This was a very tough problem to solve.
      - Currently DISABLED by default until certain it is OK.
        You must enable it. See the new Global Settings > Latency tab.
      - Features: 'Corrects' (looks ahead) or 'compensates' (delays) ALL audio and midi
         signals throughout. Recorded or played audio files should be TRUSTWORTHY now,
          regardless of audio driver period size.
      - Tested extensively with many configurations. Should be OK for usage trials.
[EDIT] - Works OK with plugins that report latency (properly). It accounts for their latency.
      - Works OK with Audio Output track sends or Midi output sends and Audio Input returns.
      - Works OK with 'Bounce to file/track' function, with or without Jack Freewheel enabled.
      - TODO: Support Aux(illiary) tracks. Not handled at all ATM.
              Fix small snippet of missing sound at start of all playback material. Should be
               played BEFORE transport rolls. Use 'slow-sync' feature to hold up the transport.
              Add user-adjustment boxes on hardware ports. (Enter results from jack_iodelay for ex.)

oscillator
Established Member
Posts: 30
Joined: Sat Jan 17, 2015 6:07 pm

Re: New! Comprehensive latency correction / compensation

Postby oscillator » Fri Oct 11, 2019 8:05 am

I am curious of this fix. Mostly because I don't have any problems. :)

When would one experience latency problems in Muse? And find use of the fix?

Cheers,

Staffan

Tim E. Real
Established Member
Posts: 277
Joined: Sat Sep 15, 2012 12:36 am

Re: New! Comprehensive latency correction / compensation

Postby Tim E. Real » Fri Oct 11, 2019 4:50 pm

It is a very important fix, perhaps the most fix important ever.

Without latency correction/compensation, when you record a wave or midi track, there is a delay (called latency)
in the recorded track caused by the period size of the audio system, in this case Jack.
Suppose Jack is running with a period size of 1024 samples.
When a track is recorded, there is a delay in the recorded signal by that amount, sometimes more.
When the track is played back, the alignment is not correct and it will be slightly out of sync,
and it will be noticeable with higher period sizes.

Traditionally, the solution has been to run Jack at the lowest possible period size that works
for you without xruns or hiccups. Typically one can get the size down to say, 256 or 128 samples or even lower.
The delay is there, but small enough to be not really noticeable. You can make a song reasonably this way.
But that's still not 'zero' period size and still creates a delay.
Any delay is bad. You don't want that.
All recorded tracks should be aligned properly or else they'll be out of sync on playback.

There are times when it is not possible to run Jack at such a low period size.
For example if you are using CPU-intensive plugins, often you must set Jack's period size higher to avoid xruns and glitches.

Also, plugins, such as audio rack plugins, sometimes introduce latency of their own.
Some are quite high due to their processing nature. Look-ahead filters, pitch shifters etc. do this.
When this is coupled with recorded tracks that are already out of sync, and mixed together with all the other tracks
and plugins and so on that are also out of sync, this makes a big mess of unsynchronized misaligned signals.

The solution is latency correction/compensation.

The fix presented here is comprehensive, not just a quick fix. The entire system is affected.
Where possible, MusE completely corrects this latency.
Otherwise it compensates for the latency, which at least guarantees that signals everywhere are in sync,
although still with an overall delay.

The most important thing is that signals everywhere, at every node, are in sync.
Even if there must be an unavoidable delay, at least the signals will actually all be in sync.

Note that this fix affects both midi and audio tracks.

Ardour has latency correction/compensation as well, and it was also a very difficult thing to perfect, as I have read.
QTractor also has it. Although I read a post by the author that he was so fed up with plugins not reporting their latency correctly
that he did not correct for them at all !
[ Correction ] QTractor does in fact have a checkbox for each plugin for whether it participates or not. Seems to be disabled by default.

The feature is enabled in the Global Settings dialog.
There are a few more fixes and features to do (Aux tracks are not handled, and looping may cause errors),
but overall it has been tested extensively and should at least be ready for testing and everyday use.

HTH.
Tim.
Last edited by Tim E. Real on Wed Oct 30, 2019 6:15 pm, edited 1 time in total.

artofmusic
Established Member
Posts: 89
Joined: Mon Mar 17, 2014 8:01 pm

Re: New! Comprehensive latency correction / compensation

Postby artofmusic » Fri Oct 11, 2019 9:42 pm

One thing I don't like about latency compensation right now, has to do with aux sends. If you say have a plugin capable of 32 outs and connect only 2 of 32 of its audio outputs to the audio output track of your choice then you'll get this annoying feedback that just stays held. The only way to stop this is to disable latency compensation or to disconnect the outputs and connect the same outputs again. I used Yoshimi (Multi) LV2 for this exercise and the audio gets stuck. Here is my session file to use and Yoshimi is in the Debian and Ubuntu repos if you want to give my session a try. All you need to do is connect a MIDI Device to it and play some notes and here some :( yuck! Also, this same behavior is happening when I connect my other multi-out VST plugins in the same fashion. On a final note, aux tracks work with latency compensation as long as they only have 2 audio outs right now. This is worth investigating because its really annoying and Muse is coming along quite nicely for 3.1 now otherwise.
You do not have the required permissions to view the files attached to this post.

oscillator
Established Member
Posts: 30
Joined: Sat Jan 17, 2015 6:07 pm

Re: New! Comprehensive latency correction / compensation

Postby oscillator » Sun Oct 13, 2019 11:07 am

Thank you Tim for your throrough explanation. I record voice with an external soundcard which gives me really low latency, so I guess that is why I haven't noticed it. I will try it out!

Tim E. Real
Established Member
Posts: 277
Joined: Sat Sep 15, 2012 12:36 am

Re: New! Comprehensive latency correction / compensation

Postby Tim E. Real » Mon Oct 14, 2019 5:02 am

@artofmusic
Verified. Thanks for the file!
This may be related to another post
https://linuxmusicians.com/viewtopic.php?f=61&t=20527

Yeah that's pretty messed up, man.

It happens with other tracks such as wave tracks. :(
And only if using individual channel routes, not 'Omni' routing.
And only if latency correction is enabled.

Thanks for spotting it. I will try to investigate.

oscillator
Established Member
Posts: 30
Joined: Sat Jan 17, 2015 6:07 pm

Re: New! Comprehensive latency correction / compensation

Postby oscillator » Mon Oct 14, 2019 8:37 am

Re viewtopic.php?f=61&t=20527 (which I posted) I cannot replicate this error (feedback), with latency correction on or off. Has something changed? Soloing and muting works fine, and I think I am running with the exact same plugins.

artofmusic
Established Member
Posts: 89
Joined: Mon Mar 17, 2014 8:01 pm

Re: New! Comprehensive latency correction / compensation

Postby artofmusic » Sat Oct 19, 2019 10:02 pm

I was using latest git build from a week ago, you might be using pre1 still maybe.

oscillator
Established Member
Posts: 30
Joined: Sat Jan 17, 2015 6:07 pm

Re: New! Comprehensive latency correction / compensation

Postby oscillator » Sun Oct 20, 2019 8:23 am

No, I am using the latest git too...

Tim E. Real
Established Member
Posts: 277
Joined: Sat Sep 15, 2012 12:36 am

Re: New! Comprehensive latency correction / compensation

Postby Tim E. Real » Tue Oct 29, 2019 9:38 pm

OK The feedback/noise when using channel routing (vs. omni routing) has been fixed :)
It was two TODO sections I neglected to get around to, regarding per-channel latency compensator delay unit mixing.

Tested OK with several configurations.
This should fix artofmusic's problem.
We'll see if oscillator's is fixed as well... maybe something different.

Try 'er out. In github master now.

Note: As mentioned already, when playing/recording with latency correction enabled, there may be a small bit of noise
and/or missing sound at the beginning of a playback or recording. Just a fraction of a second :wink:
This is a known problem and is being worked on. It has to do with queuing the waves up and playing
a small part of them before play is actually started.
For now, ehm... try not to use sounds so very close to the beginning of a recording or wave file.

Thanks.
Tim.

oscillator
Established Member
Posts: 30
Joined: Sat Jan 17, 2015 6:07 pm

Re: New! Comprehensive latency correction / compensation

Postby oscillator » Wed Oct 30, 2019 6:05 pm

Sorry, the problem with mute/unmuting giving feedback noise still exists in my projects with the latest git.

As soon as I have the time I will try to nail it down in a minimum song.


Return to “MusE Sequencer”

Who is online

Users browsing this forum: No registered users and 3 guests