Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Moderators: MattKingUSA, khz, muldjord, deva

Post Reply
XRunner
Established Member
Posts: 6
Joined: Sun Dec 22, 2019 9:26 pm

Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by XRunner »

Hi:

I recently switched from Hydrogen to DrumGizmo. Using the stock DrumGizmo install in Ubuntu18 (two years and a half old) I downloaded both the Crocell and Muldjord kits and did the neccesary changes to my MIDI files so I would trigger the correct samples and loaded DrumGizmo as an Ardour plugin. When using the Crocell kit (in a 44k session, which I guess needs resampling) I got so many XRuns that the Ardour GUI froze when adding markers. Using the Muldjord kit I got better results, but still a lot of XRuns. These XRuns are accompanied by crackling noise, noise artifacts and plain noise when my computer cannot just keep up.

Following advice from an IRC member, I compiled the latest DrumGizmo from source and got marginally better results, but still a lot of Xruns on playback. I may be misinformed but it is my understanding that no XRuns should happen in a stable system (in fact, I got no XRuns in my previous workflow, which would route the MIDI data to Hydrogen and some other synths). The beat I am playing consists on 4 measures of snare count and a steady blast of 16th notes, three notes at a time, at 260BPM. The XRuns start with the blast.

A couple of interesting bits:
- Setting the buffer size to 2048 (from 256) made the XRuns decrease just a little bit.
- had DrumGizmo directed to just two Ardour buses. When I used 16 buses the XRuns decreased, but are still there.
- Setting the buffer size to the original 256 value with 16 buses still cause a lot of XRuns.

My current specs are:

Code: Select all

Processor:  Intel(R) Core(TM) i3-7100 CPU @ 3.90GHz
RAM (free -m): 3831
External audio card: FocusRite Scarlett 2i4. (there's a crappy onboard card too).
Ubuntu Studio: Ubuntu 18.04 LTS.
DrumGizmo v: drumgizmo-0.9.18.1
Jack settings: 44100 - 256 frame - 3 periods, 95 priority, realtime
.

I tried generic XRun stuff (to no avail) like:

- Removing the wireless USB stick.
- Using a single display
- Setting "realtime audio" and "CPU governor in performance mode".

My question: is this the expected behaviour of the plugin? Is there something I can do to get better performance? Of course, if you need any other data I would be more than happy to oblige.

Thanks a lot in advance.
User avatar
English Guy
Established Member
Posts: 525
Joined: Wed Oct 17, 2012 7:28 pm
Location: England
Has thanked: 8 times
Been thanked: 7 times

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by English Guy »

I found pulse audio was still impacting performance. I changed the settings to stop it respawning and start and stop it as needed with commands. (I put the commands into qjackctl 's settings)
XRunner
Established Member
Posts: 6
Joined: Sun Dec 22, 2019 9:26 pm

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by XRunner »

I tried preventing pulseaudio from spawning and stopping it through a stop script:

Code: Select all

#!/bin/bash
echo autospawn = no > $HOME/.config/pulse/client.conf
pulseaudio --kill
rm $HOME/.config/pulse/client.conf
and a start script:

Code: Select all

#!/bin/bash
#This is redundant, I know :).
pulseaudio --start
After running the stop script I can do:

Code: Select all

pulseaudio --check; echo $?
Will give me "1" until I run QJackCtl (which I use to start jackd), which seems to start it over again, causing "0" to be returned. Actually, running

Code: Select all

jackd -d alsa --help
seems to start it again (makes it return 0)... The only audio source I have connected is the Scarlett USB interface, so I can't be sure that pulseaudio is, indeed, dead... In any case, still got XRuns, the same performance as before, I'd say.

Did you notice this performance problem only with DrumGizmo or did it affect other applications?. I never experienced any other problem in either Hydrogen or any of the soft synths I ran.

Thanks for the input.
User avatar
deva
Established Member
Posts: 281
Joined: Sun Oct 23, 2016 10:15 am
Has thanked: 3 times
Been thanked: 30 times
Contact:

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by deva »

You can try a few things within the drumgizmo ui itself:
- Increase the diskstream cache size to ease the lod on the sample loader if you harddrive is the reson for the produced hick-ups.
- Disable the resampler (there's a checkbox for that) which will change the timbre of drums slightly if you run a 44k1Hz kit in a 48kHz session (or vice versa).

In general, running drumgizmo with resampling with very low jack buffer sizes can give xruns. I usually try to avoid resampling altogether and usually run with 2048 samples buffer size in jack.

Not mapping some of the output channels will have zero effect on the performance as drumgizmo renders all the channels anyway.
XRunner
Established Member
Posts: 6
Joined: Sun Dec 22, 2019 9:26 pm

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by XRunner »

Thanks for reaching out Deva.

I tried both approaches yesterday and didn't get good results.

First, disabling resampling (or switching to a new 48k session, or using the Muldjord kit at 44k) got marginally better results, but still was unusable. Using a very large buffer size still gave me Xruns and artifacts (a fraction of what I had before, tough).

Second I tried with unlimited cache size and it made most GUIs on the computer freeze. Some other sizes like 2GB gave me mostly the sams results. A very large Jack buffer was the better (still faulty) option.

One thing I also tried was to lower the BPM from 260 to 140, in case it was too much for my computer. Unfortunately I ran into Xruns too.

I guess I can try a smaller kit but perhaps I should direct my inquiries towards the kind of workflow that other users have. Do you use DrumGizmo "live" as you draw your MIDI events and play them back? Perhaps the plugin is too CPU intensive and recording to audio tracks (freeing the plugin afterwards) is the way to go?

I have devised an alternative (light kits on Hydrogen while setting up the beats and recording the live instruments, then using DrumGizmo for the "real" take) but I can't shake the feeling that something is off.

Thanks for the input. Very much appreciated.

PS: given than I got the attention of a developer and being a developer myself, perhaps I could help with some development tasks, you know, give back to the community a bit. Just in case you want to PM, my experience is mostly in C++, Linux, SDL2 and systems programming.
User avatar
deva
Established Member
Posts: 281
Joined: Sun Oct 23, 2016 10:15 am
Has thanked: 3 times
Been thanked: 30 times
Contact:

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by deva »

I think I have a few more ideas for what you can try, and even thought of a new feature which might be able to help in this use-case.

I'm not using DG myself for live playing, but I know a lot of people do use it like that, so it should be possible.
Have you tried running it in stand-alone jack mode instead of running it through Ardour? I'm aware that will remove the option to use plugins on the output channels but it should cut down heavily on the system resource requirements.

I think the reason why you experienced UI freeze with the cache option set to unlimited is because of swapping being activated. You should try to find a sweet-spot, where the swap is not utilized and the memory is maxed out.

You can try the Shitty-kit from the website, just to see if that changes anything. It is by far the smallest kit we have available. CrocellKit on the other hand is by far the biggest one...

IIRC the Intel i3 CPU does not have multiple cores and no hyper-threads either (ie. no real multi processing capabilities). DrumGizmo uses one thread (the one from the DAW) for audio processing and a second one for loading the samples into the cache. If there is only one core available it might give better results with different kernel schedulers. Also using real-time patches for the kernel and making it preemptive might make a difference.

The actual problem is that playing a blast-beat at 260 bpm on the ride cymbal will at one time be playing ~10.000 samples simultaneously, increasing with the length of the samples and the length of the blast-beat.
This is because DrumGizmo has no notion of "voices" and will happilly play as many simultanious samples as you instruct it to - until it starts producing xruns.
The feature I am thinking of it to add a per instrument max number of "voices", where you can instruct it to play at most, for example, 4 ride cymbals at once and the moment you activate number 5 it will choke (ie. gradually mute) the "oldest" one, which probably cannot heard anyway because of the 4 more recent ones already playing.

If you jump on IRC we can talk further about it, as well as your interesting PS proposal :-)
XRunner
Established Member
Posts: 6
Joined: Sun Dec 22, 2019 9:26 pm

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by XRunner »

Hi Deva:

I am at work (break time) at the moment, so I won't be able to jump in an try your suggestions, but will definitely try both the stand alone mode and the "shitty kit", as well as a more relaxed beat, since they are my most immediate options. Of course, if beat and voices themselves turn out to be the problem, such a feature as the one you mention would be great to reduce CPU load, not only in my case, but also for all users.

As for the hardware itself, if it helps, I will check when I get home. The product page at Intel says that this thing has 2 processors and 4 threads per processor, but hardware is hardly my field of expertise.

In any case I will try to investigate the matter further as soon as my schedule clears (holidays and children are a very bad recipe to sit in front of the computer). As for jumping into IRC, I will give it a try, at least to setup further chats.

Cheers.
XRunner
Established Member
Posts: 6
Joined: Sun Dec 22, 2019 9:26 pm

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by XRunner »

Good people of DrumGizmo:

I took some time to run more tests and got very reasonable results. The key seemed to be the CPU governor set to performance mode (I don't know why Ubuntu Studio resets this value when I restart my machine).

I ran all tests using the VST plugin, two displays, showing the main Ardour and mixer screens plus QJackCtrl. I ran them separatedly in the Muldjord and Crocell kits, with the default velocity and timing humanizers and sample selection. For the Crocell kit, I turned off resampling too.

For the Muldjord kit:

- Cache: 2200MB, buffer size: 256, 120BPM, got 4(7) Xruns, no glitches. This is the only test I did at 120BPM.
- Cache: 2200MB, buffer size: 256, 259BPM, 1(3) Xruns, ever so slight glitches. Seems that having loaded the samples in the previous test helped.
- Cache: 2200MB, buffer size: 512, 259BPM 0(1) Xruns, good quality.
- Cache: 2200MB, buffer size: 1024, 259BPM 0(0), good quality.
- Cache: 2200MB, buffer size: 2048, 259BPM 0(0), good quality.
- Cache: 1200MB, buffer size: 256, 259BPM, 2(5) Xruns, small glitches.
- Cache: 1200MB, buffer size: 512, 259BPM 0(1) good quality,
- Cache: 1200MB, buffer size: 1024, 259BPM 0(0), good quality.
- Cache: 1200MB, buffer size: 2048, 259BPM 0(0), good quality.

So, buffer size seems to be the most important factor here.

For the Crocell kit:

- Cache: 1200MB, buffer size: 512, 259BPM 19(205), lots of glitches that become noise. A very large change from the Muldjord.
- Cache: 1200MB, buffer size: 1024, 259BPM 10(33), glitches that become noise.
- Cache: 1200MB, buffer size: 2048, 259BPM 2(9), some glitches, specially in fast fills.

However, as soon as I turned all "sample selection" values to 0 I got 0 Xruns in the Crocell at 2048 buffer size, so I guess we can blame my disk drive or the huge number of samples in the Kit. In any case, I am sure I can work with the Muldjord kit without hassles. I just wonder now if a "Freeze track" (offline rendering of plugin audio to a static waveform) will be added to Ardour in a future release, so I can make use of the 8GB beast kits :).

In any case, I am willing to give a try to the voice limiting feature. I hope my experience can help other users of DrumGizmo.

Cheers.
User avatar
bluebell
Established Member
Posts: 1903
Joined: Sat Sep 15, 2012 11:44 am
Location: Saarland, Germany
Has thanked: 111 times
Been thanked: 114 times

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by bluebell »

XRunner wrote: I took some time to run more tests and got very reasonable results. The key seemed to be the CPU governor set to performance mode (I don't know why Ubuntu Studio resets this value when I restart my machine).
/etc/init.d/ondemand from the package "initscripts" (can't be removed safely) waits 60 seconds and then screws up your governor settings.

I had to tweak it on all my Ubuntu machines used for audio production.

Linux – MOTU UltraLite AVB – Qtractor – http://suedwestlicht.saar.de/

XRunner
Established Member
Posts: 6
Joined: Sun Dec 22, 2019 9:26 pm

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by XRunner »

bluebell wrote:
XRunner wrote: /etc/init.d/ondemand from the package "initscripts" (can't be removed safely) waits 60 seconds and then screws up your governor settings.

I had to tweak it on all my Ubuntu machines used for audio production.
Didn't know that. I will look into it :).
JamesPeters
Established Member
Posts: 188
Joined: Fri Jun 29, 2018 6:35 pm
Has thanked: 8 times
Been thanked: 15 times

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by JamesPeters »

The voicing limiting feature is a great idea. I know several people who have struggled to get solid performance with DrumGizmo, and I know from my testing that having multiple samples of the same kit piece continually playing ("stacked", as the decay is still happening) affects it a lot. This is compounded with the fact that the larger kits on the DrumGizmo site are all playing bleed in every audio channel all the time. I'm working on a kit which has between 4 and 7 channels per kit piece depending on what it is (room and overhead buses, but no bleed); its performance makes a lot less demand on the CPU.
theatomicdog
Established Member
Posts: 9
Joined: Mon May 27, 2019 10:00 pm
Been thanked: 1 time

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by theatomicdog »

Hi There,

I experienced same issues, which forced my to stick to AVLDrums https://github.com/x42/avldrums.lv2 for a long time. Despite that setup working fine to practice here and there, the sound quality of Crocell (which has a real, decent OpenHiHat and great ride sound) made me give it another try, and this time I succeeded to my goal of keeping my 48khz 256 buffer (which gives a processing latency of max 10ms).

Background:
So, I'll describe in a bit of detail how I made it, but first, a note about my hardware. I have a refurbished HP notebook with an SSD drive. The cpu is an i5-8250U and has 8GB of RAM. I also have Behringer UMC 404HD external sound card. This is more or less, a pretty "stock" system. If you want you can compare it to a Toyota Corolla, made to get you here and there. In comparison, most people doing audio have way more powerful systems, so quite a lot of tuning is required to get mine to perform at a decent level.

First Steps:
Tuning your system, kill all things you don't use. Having processes and subsystems consuming cpu and blocking i/o during your realtime audio session is like texting while racing the nascar. In my case I took down things like wifi, and daemons that don't have any value.

First, I installed the realtime kernel. In my system it was

Code: Select all

sudo apt install linux-image-5.2.0-0.bpo.2-rt-amd64
Go and find the equivalent for your system.

Then, time to turn off anything you don't need for your audio session.

Code: Select all

# I turned off every service I could find with systemctl status and htop that I didn't needed
for service in wpa_supplicant.service ModemManager.service colord.service NetworkManager.service cups-browsed.service cups.service gvfs-daemon.service accounts-daemon.service udisks2.service dnsmasq.service; do
    sudo systemctl stop $service
done

# And removed all modules that kept things working
for module in ppdev lp uvcvideo videodev ath9k iwlmvm iwlwifi; do
    sudo modprobe -r $module
done

# Then killed a few processes that were not giving me any value
sudo killall gvfsd
killall nm-applet
That already gave me quite some spare cycles. To get it even further, I use WindowMaker, which is pretty extreme on the efficiency side. I like it because I'm old and used it for the last 20 years, but you can find a lightweight windowmanager, like LXDE or something like that, instead of Gnome.

Then I ramped up my realtime capabilities. I have eight pseudocores so I made sure to put them in performance mode, and to activate the realtime timer in the kernel

Code: Select all

sudo modprobe snd-hrtimer # Load the ALSA high res timer for my MIDI stuff

for i in 0 1 2 3 4 5 6 7; do
    echo -n performance | sudo tee /sys/devices/system/cpu/cpu$i/cpufreq/scaling_governor
done
I then made two scripts, one to get into "audo mode" and another to restore network connectivity and other services, and I run them to switch back and forth.

Second step, tuning DrumGizmo

With all that, things already went way better, but still had some xruns. I saw, someone suggested 2048 frames. That is 85ms of latency, just on the internal latency. I would not like to be recording on that, so went more into trying to get this thing to work on 256 frames.

First thing, was to load the set in memory, but as anyone who tried may tell you, Crocell set will not fit in the memory of a regular system, so I had to chop it down somehow. Now, I have a Medeli dd 315 as MIDI input for drums. You can find it rebranded by Alesis, and a bunch of other lesser known brands, but is the same thing. That means I have just 10 sounds in a set. With that in mind, I created a customized Crocell set, that I called CrocellKit_custom.xml with my chosen 10 sounds:

Code: Select all

    <instrument name="CrashL" file="CrashL/CrashL.xml">
    <instrument name="CrashR" file="CrashR/CrashR.xml">
    <instrument name="HihatClosed" group="hihat" file="HihatClosed/HihatClosed.xml">
    <instrument name="HihatOpen" group="hihat" file="HihatOpen/HihatOpen.xml">
    <instrument name="HihatPedal" group="hihat" file="HihatPedal/HihatPedal.xml">
    <instrument name="KDrumR" file="KDrumR/KDrumR.xml">
    <instrument name="RideR" file="RideR/RideR.xml">
    <instrument name="Snare" file="Snare/Snare.xml">
    <instrument name="Tom1" file="Tom1/Tom1.xml">
    <instrument name="Tom2" file="Tom2/Tom2.xml">
and a companion midi mappings file for it.

After that, voila! I could lode it in less than 3GB! Even better, I had my loved HihatPedal sound, and the whole setup sounded great. Now, the kit was still using more CPU than I wanted. I played around on the setup and found out that adding it as 16 Channel Fan Out works better. Seems like Ardour can mix channels in a very efficient way, and you have the added benefit of tweaking the mics one by one. I strongly recommend using it that way.

I tried to go even further and remove the Floor Toms channels (which I was not using) and the Ambient Mics, but that didn't seemed to do any difference.

One last bit, "warming up the set" helps also prevent xruns. I did this several times and seems to work, you get a few xruns first time you bang it relly hard, then things go smooth.

Hope this helps
chaot4
Established Member
Posts: 61
Joined: Sat Apr 16, 2016 9:59 am
Been thanked: 1 time

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by chaot4 »

theatomicdog wrote: Sun Mar 15, 2020 2:01 pm First thing, was to load the set in memory, but as anyone who tried may tell you, Crocell set will not fit in the memory of a regular system, so I had to chop it down somehow. Now, I have a Medeli dd 315 as MIDI input for drums. You can find it rebranded by Alesis, and a bunch of other lesser known brands, but is the same thing. That means I have just 10 sounds in a set. With that in mind, I created a customized Crocell set, that I called CrocellKit_custom.xml with my chosen 10 sounds:
Can you please say why you don't use the diskstreaming feature which should solve the issue of too large RAM requirements?
theatomicdog
Established Member
Posts: 9
Joined: Mon May 27, 2019 10:00 pm
Been thanked: 1 time

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by theatomicdog »

chaot4 wrote: Fri Apr 03, 2020 7:27 pm Can you please say why you don't use the diskstreaming feature which should solve the issue of too large RAM requirements?
That would solve the RAM problem by reading from disk, but I try to avoid in a DAW, to prevent disk I/O from being a bottleneck. If the linux caching works fine in your case, then go for it. As I mentioned, tuning the system to your needs is probably the best approach. For me, preloading all samples in RAM helped get xruns under control at lower latency settings.
tseaver
Established Member
Posts: 391
Joined: Mon Mar 13, 2017 6:07 am
Has thanked: 10 times
Been thanked: 95 times

Re: Excessive XRUNS with Ubuntu Studio, jackd and Ardour.

Post by tseaver »

@theatomicdog wrote:
That would solve the RAM problem by reading from disk, but I try to avoid in a DAW, to prevent disk I/O from being a bottleneck.
The conventional wisdom these days among folks using large (i.e., orchestral) sample libraries is to put them on the fastest available SSD you have (including USB 3.0 / Lightning / Thunderbolt, etc.). SSDs are so much faster than spinning disks for streaming that you can even afford to purge the RAM parts. I would imagine that DG kits would be similarly served.
Ubuntu, Mixbus32C; acoustic blues / country / jazz
Post Reply