Options to add latency in order to sync with video better?

All your LV2 and LADSPA goodness and more.

Moderators: MattKingUSA, khz

Post Reply
wolftune
Established Member
Posts: 1350
Joined: Fri Feb 24, 2012 7:40 pm
Location: Portland, OR
Been thanked: 2 times
Contact:

Options to add latency in order to sync with video better?

Post by wolftune »

I'm wanting to use a higher-quality camera as a webcam source in WebRTC like with Jitsi Meet. I'm having trouble with particular hardware that seems to end up with the video latency long enough to be out of sync with the audio.

Since the web browsers only use Pulse, I can send audio by just choosing what goes into Jack's Pulse Audio source.

The only trouble is, how could I set up a plugin to *increase* the latency of the audio, just add a delay, before it goes out to pulse and to Jitsi?

I tried the Steve Harris "Artificial Latency" plugin in Carla, but it didn't seem to do anything. From what I can tell, it's a misnomer. What it does is simply push the DSP Load up really high. But I wasn't getting any shift in the audio at all.

Update: as I was writing this, I guess "Simple Delay Line" set to just some simple delay should do it. I also noticed Steve Harris "Fractionally Addressed Delay Line" not sure what that is, seems to use more DSP (not much, but the Simple Delay Line shows no indication of use). I also note LSP Delay Compensator and Calf Compensation Delay Line.

Are these all about the same? The LSP and Calf plugins have a lot more controls, not sure what they are all for.

Any suggestions about the best approach here?

Any other thoughts about the best way to measure the video stream latency that I need to compensate for?
Aaron Wolf
Music teacher, scholar
http://wolftune.com
User avatar
LAM
Established Member
Posts: 992
Joined: Thu Oct 08, 2020 3:16 pm
Has thanked: 140 times
Been thanked: 348 times

Re: Options to add latency in order to sync with video better?

Post by LAM »

Not sure, but you could try to use OBS Virtual Camera feature, so the sync will be done in OBS, and you can use it as a webcam source for Jitsi.

EDIT: Look here for latency settings in OBS https://streamshark.io/obs-guide/audio-delay

in mix, nobody can hear your screen

wolftune
Established Member
Posts: 1350
Joined: Fri Feb 24, 2012 7:40 pm
Location: Portland, OR
Been thanked: 2 times
Contact:

Re: Options to add latency in order to sync with video better?

Post by wolftune »

Of course that OBS approach seems like it should work, but OBS doesn't send audio *output* in any way I can use! If OBS supported Jack and had a jack output, everything could route through OBS fine. But instead, it only sends audio output when it streams to a specific streaming service like YouTube (or when recording locally of course).

It can *monitor* audio but that sends from pulse *to* my pulse client (i.e. Jack), but the only way to send that to the pulse bridge to Jitsi would be to connect pulse output to pulse input. And I could do that, but it that would create echo because it would send the other Jitsi sounds (and whatever other pulse-based sounds I hear) back out to Jitsi. It actually might work given echo cancelation and such, but it would give me no control to choose which pulse-based sounds to share in Jitsi or not, everything that plays through pulse on my computer would go to Jitsi.

The main reason OBS throughput is no-go is because it would force me to monitor the OBS output with its latency! I would be forced to *hear* the delayed audio of myself! Because OBS has no separate Jack output, I can't send audio out from it without *hearing* it myself unless I hear none of the pulse sounds at all (which would mean I can't hear anything from others on Jitsi).

So, I'm stuck with separate audio-latency via Jack. I'd love to get any opinions on the plugins I listed above, but I think they'll work fine.
Aaron Wolf
Music teacher, scholar
http://wolftune.com
User avatar
LAM
Established Member
Posts: 992
Joined: Thu Oct 08, 2020 3:16 pm
Has thanked: 140 times
Been thanked: 348 times

Re: Options to add latency in order to sync with video better?

Post by LAM »

Another option is to create a virtual sink in PulseAudio and set latency there:
https://askubuntu.com/questions/421014/ ... on-like-sk

Basically this is the method I use when i livestream, you should want to change the latency-msec value to match the latency of your video.

in mix, nobody can hear your screen

wolftune
Established Member
Posts: 1350
Joined: Fri Feb 24, 2012 7:40 pm
Location: Portland, OR
Been thanked: 2 times
Contact:

Re: Options to add latency in order to sync with video better?

Post by wolftune »

I'm really glad to hear about that option with pulse. It looks complicated but seems to be the ideal once I figure it out!

Meanwhile, I did preliminary plugin tests, and here's some notes:

- Simple Delay Line is awkward because after a delay is set, it can't reset, it only adds. So, there's no way to try a larger amount and then go back to smaller. I mean, it's glitchy in a sense. It doesn't start over with the current input, it continues playing the already-delayed input even if the delay is turned off. But if I started it just correctly, it would work. It has ms level resolution

- Fractionally Addressed Delay Line works, but the minimum setting is a tenth of a second and it can't be turned off except by bypassing. It otherwise has tenth of ms resolution, so it is usable for anything that needs more than a tenth of a second delay

- Calf Compensation Delay Line only does some distance-based approach to setting the delay

- LSP Delay Compensator works and has a stereo option, and it can be set to samples, distance, or time. It has a clear readout, it does everything, what a winner

LSP is the best by far. In this case, I'd almost want to remove Calf from being installed, LSP is superior in every way!

So, LSP plugin will certainly work, and I just have to decide whether to use it or go with the Pulse virtual sink.

Is there any reason to do the extra complex set up with Pulse virtual sinks rather than just use the LSP plugin and feed everything into it that I want to be going with the video?
Aaron Wolf
Music teacher, scholar
http://wolftune.com
Post Reply