[Solved]: Drifting hardware latency leads to XRun

Support & discussion regarding DAWs and MIDI sequencers.

Moderators: MattKingUSA, khz

Post Reply
finetuned
Established Member
Posts: 6
Joined: Sat May 18, 2019 12:30 pm
Location: Ede, NL
Has thanked: 1 time
Been thanked: 1 time

[Solved]: Drifting hardware latency leads to XRun

Post by finetuned »

Hi,

I've been using Ardour with my Echo Audiofire 12 on Elementary OS. I've noticed that the latency seems to 'drift' by about 10 ms and then snap back every 10 minutes or so. I decided to get a bit more analytical and compare the behavior to an RME HDSPe AIO sound card and a Behringer Xenyx UB1202 USB mixer. The results are below.

The Setup
  • Intel Core i7 9700K 8-core non-hyperthreaded CPU
  • AsRock Z390-ITX motherboard
  • StarTech PCI Express Firewire card with TI XIO2213A/B/XIO2221 firewire chipset. This card contains a PCIe to PCI brigde as well, from the info of lspci.
  • Linux kernel: 4.15.0-50-lowlatency
  • Echo Audiofire 12 Firewire interface
  • RME HDSPe AIO PCIe interface
  • Behringer Xenyx UB1202 USB mixer
All tests are run at 44.1kHz and 512 samples with 2 periods per sample. The charts below show the hardware latency, which is the total roundtrip latency minus the expected buffer latency (3 x 512/44.1 milliseconds).

Visual inspection

This is a recording of the click track in ardour which I then re-recorded multiple times on new tracks (each time recording only the original one again). You can see the behaviour of the drift-and-snap-back here in an informal way. The top track is the original, the ones below are re-recordings of it. I zoomed in to show a single click from the track:

Image

I ran jack_delay on the interface for more than an hour, and recorded the measurements. I visualized them in this chart. You can clearly see the latency increasing and snapping back each time:

Image

Zooming in a bit, we can see the step-wise increases in latency:

Image

Displayed as a number of samples, the jumps are pretty constant at 8 samples each jump:

Image

Comparisons to other sound cards on the same computer

Compared to the RME sound card, which is completely solid (this was for about 30 minutes):

Image

And the Behringer USB mixer, pretty solid too (notice the sub-millisecond differences in the scale on the left):

Image

Wondering what could be the problem, I ran jack_delay again and looked at the QJackCtl message log. Every time the "snap back" occurs, there is an XRun at exactly that moment.

Theory

I can think of the following theory of what might be going on:
  • The clock source of the FireWire card and my PC are not in sync, causing the samples to slowly drift, eventually leading to an empty buffer and requiring a refill. If the PCs clock is slightly faster than the Interface's clock, the samples might not be entering the buffer fast enough, eventually leading to a buffer underrun and the system waiting for new data, and the process starting over again. Does this make any sense?
  • The LED on the front of my Audiofire indicate that the clock source is INT. There is also an LED for "1394", which is not lit. I remember this LED being on when connected to my Mac which I was using before my Linux PC. Could this indicate that, indeed, the clock is not properly synced with the PC?
Is there anything I can do to sync the clock properly, if this is the problem?

Thanks for your any insight you might be able to provide.
Last edited by finetuned on Wed Mar 09, 2022 9:27 am, edited 3 times in total.
finetuned
Established Member
Posts: 6
Joined: Sat May 18, 2019 12:30 pm
Location: Ede, NL
Has thanked: 1 time
Been thanked: 1 time

Re: Analysis: drifting hardware latency leads to XRun

Post by finetuned »

I can confirm that when connected to my old Mac, both the "Internal" and "1394" sync lights are lit on the Audiofire 12. When connected to the Linux computer, only the "Internal" light is on.

Does anyone have experience with correct clock syncing over Firewire from the host computer? Perhaps any other theories as to why this drift-to-Xrun might occur?
Last edited by finetuned on Sat May 18, 2019 6:19 pm, edited 1 time in total.
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: Analysis: drifting hardware latency leads to XRun

Post by merlyn »

I like the graphs. :)

I think you're right about this being caused by the soundcard using its internal clock.

You haven't said anything about your JACK setup.

I don't know much about firewire, but the key would seem to be getting the card to use the computer's clock, which may be a setting in JACK.
finetuned
Established Member
Posts: 6
Joined: Sat May 18, 2019 12:30 pm
Location: Ede, NL
Has thanked: 1 time
Been thanked: 1 time

Re: Analysis: drifting hardware latency leads to XRun

Post by finetuned »

(Moderators: I realize that this thread might be better suited to the "Computer Related Hardware" subforum. Feel free to move it there)

I've been looking through the Jack settings (at least those exposed by QJackCtl) but could not find anything related to the clock source in there:

Image

Image

I see that there is a --clocksource attribute to the Jack startup parameters, but I'm not sure how to use it correctly. Does anyone know?
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: Analysis: drifting hardware latency leads to XRun

Post by merlyn »

Is there a control in

Code: Select all

alsamixer
to switch the card so the 1394 light comes on?
finetuned
Established Member
Posts: 6
Joined: Sat May 18, 2019 12:30 pm
Location: Ede, NL
Has thanked: 1 time
Been thanked: 1 time

Re: Analysis: drifting hardware latency leads to XRun

Post by finetuned »

Sadly, alsamixer only shows "This sound device does not have any controls". I can control the RME sound card just fine with it, just not the Audiofire.

Update: I made some progress. When I launch ffado-dbus-server, the "1394" light comes on. However, the output from that command shows errors related to the clock source:

Code: Select all

00721051555: Debug (devicemanager.cpp)[1279] showDeviceInfo: Clock sync sources:
00721068182: Warning (efc_avc_cmd.cpp)[  90] deserialize: Deserialization failed
00721068203: Error (fireworks_device.cpp)[ 208] doEfcOverAVC: EfcOverAVCCmd command failed
And later:

Code: Select all

00721084084: Debug (efc_cmds_hardware.cpp)[ 242] showEfcCmd: EFC POLLED info:
00721084086: Debug (efc_cmds_hardware.cpp)[ 243] showEfcCmd:  Status          : 0x909EB5AD
00721084086: Debug (efc_cmds_hardware.cpp)[ 244] showEfcCmd:  Detect SPDIF    : 0xA07F0000
00721084087: Debug (efc_cmds_hardware.cpp)[ 245] showEfcCmd:  Detect ADAT     : 0xE0A6B5AD
00721084089: Error (fireworks_device.cpp)[ 524] isClockValid: Could not update polled values
00721084129: Debug (devicemanager.cpp)[1288] showDeviceInfo:  Type: Internal          , Id:  0, Valid: 1, Active: 0, Locked 1, Slipping: 0, Description: Internal sync
00721084131: Debug (devicemanager.cpp)[1288] showDeviceInfo:  Type: WordClock         , Id:  2, Valid: 0, Active: 0, Locked 1, Slipping: 0, Description: Word Clock
00721086042:  (ffado-dbus-server.cpp)[ 329] main: DBUS service running
00721086046:  (ffado-dbus-server.cpp)[ 330] main: press ctrl-c to stop it & exit
00721086048: Debug (ffado-dbus-server.cpp)[ 333] main: dispatching...
I also cannot connect Jack using the Firewire driver (Cannot start server).
finetuned
Established Member
Posts: 6
Joined: Sat May 18, 2019 12:30 pm
Location: Ede, NL
Has thanked: 1 time
Been thanked: 1 time

Re: Analysis: drifting hardware latency leads to XRun

Post by finetuned »

Another update. I compiled and installed the latest ffado version (2.4.1) from source and managed to get it to run with Jack and the jack_delay tools. I also reverted the firmware version of the Audiofire back to 4.8, which seems to be the version that the FFADO drivers were developed against originally.

The ffado-dbus-server now shows no errors pertaining to the Clock of the unit (good), and shows:

Code: Select all

10521529366: Debug (devicemanager.cpp)[1279] showDeviceInfo: Clock sync sources:
10521561238: Debug (devicemanager.cpp)[1288] showDeviceInfo:  Type: Internal          , Id:  0, Valid: 1, Active: 1, Locked 1, Slipping: 0, Description: Internal sync
10521561245: Debug (devicemanager.cpp)[1288] showDeviceInfo:  Type: WordClock         , Id:  2, Valid: 0, Active: 0, Locked 1, Slipping: 0, Description: Word Clock
I now have the "Internal" and "1394" leds lit, and I can use the "firewire" driver in Jack. However, the drift problem has not been solved; the behavior is mostly the same as with the Alsa driver.

I think I have to learn more about how clocking is communicated between the audio device and the computer; I'm still not certain about how that works and if that is indeed a factor here.
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: Analysis: drifting hardware latency leads to XRun

Post by merlyn »

fine tuned wrote:I now have the "Internal" and "1394" leds lit
It may not be a clock or sync problem then. Have you tried different combinations of sample rate and buffer size? Try 48000 Hz.
finetuned
Established Member
Posts: 6
Joined: Sat May 18, 2019 12:30 pm
Location: Ede, NL
Has thanked: 1 time
Been thanked: 1 time

Re: Analysis: drifting hardware latency leads to XRun

Post by finetuned »

I've been able to solve the drifting problem with the Audiofire.

Things worked out once I set up my udev rules to allow access to the firewire device from the current user. This allowed Jack to properly start the firewire (ffado) driver, which brought up the "1394" clock light, and solved the drift problems.

Whereas I had previously been able to get the 1394 light on when starting ffado-dbus-server using sudo, that light would remain on even after I had quit the ffado-dbus-server, which I believe to be false. When I would start Jack after that, the device would actually not be in sync.

The steps to solve the problem were:
  • Revert the firmware of the Audiofire to version 4.8
  • Compile FFado 2.4.1 from source
  • Install jack2-firewire so that jack will be able to use the ffado driver. This will also install an older version of libffado2, but I can see that it actually uses 2.4.1 which I compiled from source
  • Install the udev rules file /etc/udev/rules.d/60-ffado.rules with the following content:

Code: Select all

SUBSYSTEM!="firewire", GOTO="ffado_end"

# TC GROUP A/S
ATTR{vendor}=="0x000166", GROUP="audio", ENV{ID_FFADO}="1"
# Mark of the Unicorn, Inc. (aka MOTU)
ATTR{vendor}=="0x0001f2", GROUP="audio", ENV{ID_FFADO}="1"
# Apogee Electronics Corp.
ATTR{vendor}=="0x0003db", GROUP="audio", ENV{ID_FFADO}="1"
# Alesis Corporation
ATTR{vendor}=="0x000595", GROUP="audio", ENV{ID_FFADO}="1"
# Bridgeco Co AG
ATTR{vendor}=="0x0007f5", GROUP="audio", ENV{ID_FFADO}="1"
# Presonus Corporation
ATTR{vendor}=="0x000a92", GROUP="audio", ENV{ID_FFADO}="1"
# TerraTec Electronic GmbH
ATTR{vendor}=="0x000aac", GROUP="audio", ENV{ID_FFADO}="1"
# M-Audio
ATTR{vendor}=="0x000d6c", GROUP="audio", ENV{ID_FFADO}="1"
# Ego Systems Inc.
ATTR{vendor}=="0x000f1b", GROUP="audio", ENV{ID_FFADO}="1"
# Loud Technologies Inc.
ATTR{vendor}=="0x000ff2", GROUP="audio", ENV{ID_FFADO}="1"
# Stanton Magnetics,inc.
ATTR{vendor}=="0x001260", GROUP="audio", ENV{ID_FFADO}="1"
# Focusrite Audio Engineering Limited
ATTR{vendor}=="0x00130e", GROUP="audio", ENV{ID_FFADO}="1"
# Echo Digital Audio Corporation
ATTR{vendor}=="0x001486", GROUP="audio", ENV{ID_FFADO}="1"
# Phonic Corporation
ATTR{vendor}=="0x001496", GROUP="audio", ENV{ID_FFADO}="1"
# BEHRINGER Spezielle Studiotechnik GmbH
ATTR{vendor}=="0x001564", GROUP="audio", ENV{ID_FFADO}="1"
# FlexRadio Systems
ATTR{vendor}=="0x001c2d", GROUP="audio", ENV{ID_FFADO}="1"
# Weiss Engineering Ltd.
ATTR{vendor}=="0x001c6a", GROUP="audio", ENV{ID_FFADO}="1"
# ROLAND DG CORPORATION
ATTR{vendor}=="0x0040ab", GROUP="audio", ENV{ID_FFADO}="1"
# DnR
ATTR{vendor}=="0x000f64", GROUP="audio", ENV{ID_FFADO}="1"
# Avid (for Mbox 3 Pro)
ATTR{vendor}=="0x00a07e", GROUP="audio", ENV{ID_FFAOD}="1"
# Yamaha (for GO4x devices)
ATTR{vendor}=="0x00a0de", GROUP="audio", ENV{ID_FFADO}="1"
# Lexicon (from Onix-FW810S)
ATTR{vendor}=="0x000fd7", GROUP="audio", ENV{ID_FFADO}="1"
# Allen and Heath
ATTR{vendor}=="0x0004c4", GROUP="audio", ENV{ID_FFADO}="1"
# Midas
ATTR{vendor}=="0x10c73f", GROUP="audio", ENV{ID_FFADO}="1"

# The devices below are by vendors who make other firewire devices in
# addition to their audio interfaces.  They need more specific rules to
# ensure only audio interfaces are covered here.

# Tascam, a subsiduary of TEAC (the OUI is TEAC's)
ATTR{vendor}=="0x00022e", ATTR{model}=="0x010067", GROUP="audio", ENV{ID_FFADO}="1"

# The devices below abuse another Vendor's ID, and therefore we need more advanced rules for those.

# CME, Matrix K FW
ATTR{vendor}=="0x00000a", ATTR{model}=="0x030000", ATTR{units}=="*0x00a02d:0x010001*", GROUP="audio", ENV{ID_FFADO}="1"
# Mackie, Onyx Firewire 
ATTR{vendor}=="0x00000f", ATTR{model}=="0x01006?", ATTR{units}=="*0x00a02d:0x010001*", GROUP="audio", ENV{ID_FFADO}="1"
# RME
ATTR{vendor}=="0x000a35", ATTR{units}=="0x000a35:0x00000[1234]", GROUP="audio", ENV{ID_FFADO}="1"

LABEL="ffado_end"
I can now use Jack with FFado and get the device all the way down to 32 samples with no XRuns and complete stability. This is the best this interface has ever worked, even better than on the (2009) Mac I was using before. I'm happy that I persevered and I thank all the Linux community for their endless amounts of work that went into the projects that I'm using: ffado, jack, ardour, and all the other supporting bits. I'm grateful to be part of the community and I hope to be able to contribute someday.

Also thanks to merlyn for thinking along with me.
User avatar
raboof
Established Member
Posts: 1855
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Has thanked: 50 times
Been thanked: 74 times
Contact:

Re: Analysis: drifting hardware latency leads to XRun

Post by raboof »

finetuned wrote:I've been able to solve the drifting problem with the Audiofire.
Nice!
finetuned wrote:I had previously been able to get the 1394 light on when starting ffado-dbus-server using sudo, that light would remain on even after I had quit the ffado-dbus-server, which I believe to be false.
Aahh tricky
finetuned wrote:I'm grateful to be part of the community and I hope to be able to contribute someday.

Also thanks to merlyn for thinking along with me.
I think this detailed and interesting thread is a nice contribution already :D
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: [Solved]: Drifting hardware latency leads to XRun

Post by merlyn »

@finetuned: I'm glad you got it working.

When it wasn't working you got this:

Code: Select all

10521561238: Debug (devicemanager.cpp)[1288] showDeviceInfo:  Type: Internal          , Id:  0, Valid: 1, Active: 1, Locked 1, Slipping: 0, Description: Internal sync
I'm wondering what that says now. Is the sync now '1394' or similiar?
Post Reply