How much RT optimization is still needed?

Optimize your system for ultimate performance.

Moderators: MattKingUSA, khz

User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

How much RT optimization is still needed?

Post by khz »

Outsourced by viewtopic.php?p=121705#p121705:

According to this thread (*1), many of the computer hardware / (programming) techniques used back then no longer exist today.
So it makes no sense to use these optimizations anymore.
Can we get together the current state of what is needed to make a good LAW (Audio RT OS), collect and collate evidence?
Who is familiar with this?
Ideally, this should also be updated in https://wiki.linuxaudio.org/wiki/system_configuration. or generally on linuxaudio and linuxmusicians.

What do you think about this?
How - if necessary - do we update our knowledge?

Single fragments of @x42 (*1):

set permissions to the audio group to /dev/hpet e /dev/rtc;

Which apps use those? It should not be needed anymore, nor should any app use those for audio timing on a modern system (kernel newer than 2.6.39).

I wrote most of the content on this page in 2006/7 and sadly most of it is out of date and has since been frankensteined by various wiki editors. Take it with a huge grain of salt.

[...] Installing jackd asks to enable reatime-permissions. I don’t use jack, I only install the package to conveniently setup rt-permissons and groups. After that setting up rtirq-init (and the threadirqs kernel boot option) is sufficient. – The only manual change I made is to re-apply rtirq settings after suspend/resume cycle (similar to https://github.com/jhernberg/udev-rtirq 2).
That’s all.

On this i7, I don’t even need to change the CPU governor. CPU frequency change are orders or magnitude faster than audio process cycles.

. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: How much RT optimization is still needed?

Post by khz »

Are there any users here on LM who are technically familiar with this topic?
If a lot in the wiki is really outdated, which is very likely, it would be useful to keep the knowledge up to date and to adapt it.
I still do a lot of what is recommended in the wiki, especially what Realtimequickconfigscan shows.

If many things were obsolete and you only need "rtirq-init", "rt-permissons and groups" and optionally a RT kernel it would be much easier.
But the limits.conf are still up-to-date?

Is there no interest to keep the wiki and our knowledge on LM up-to-date?
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
turbidh20
Established Member
Posts: 190
Joined: Sun Aug 04, 2019 12:48 pm
Location: UK
Has thanked: 8 times
Been thanked: 2 times

Re: How much RT optimization is still needed?

Post by turbidh20 »

I think this would be really useful and something we can point new (and old!) users at.

Unfortunately it requires technically proficient people who really understand how this stuff works. Personally, I've just cobbled together what I think is relevant, mainly from the Arch Wiki, such as Realtime Process Management and Professional Audio.

I've installed a few Arch packages - 'realtime-privileges', 'rtirq' and 'rtapps' and I'm using the default Arch kernel. The only specific tweak I've made is in '99-sysctl.conf', which is

Code: Select all

fs.inotify.max_user_watches = 600000"
That's as much as I recall doing for realtime audio. I'm using Jack (QJackCtl), no Pulseaudio.

There is one AUR package - 'tuned-git' I use, which when using its 'latency-performance' profile gives a noticeable boost to cutting DSP load.
Arch Linux (i7-4790k/GTX970); UMC204HD; MPK Mini MKII; Jack2 (QjackCtl) & Ardour/Mixbus
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: How much RT optimization is still needed?

Post by merlyn »

turbidh20 wrote:There is one AUR package - 'tuned-git' I use, which when using its 'latency-performance' profile gives a noticeable boost to cutting DSP load.
I tried this and it didn't affect my DSP load. I'm not suggesting there is anything wrong with tuned -- I think I had already done everything that tuned does.

I think updating the guide to optimising Linux for realtime audio is a good idea. For me to find out what information is still relevant I would want to set up a new system, as I have done everything at some point and I don't remember what it all was.

It seems the tweaks involving timers may not still be necessary.

The important tweaks seem to be :

CPU governor to performance
limits.conf or realtime privileges on Arch
threadirqs kernel parameter
swappiness -- can't hurt
rtirq -- not everyone seems to get a benefit from this. I have a PCI card and I do. I'm less convinced rtirq is useful for USB audio, as rtirq raises the priority of all USB devices -- that could mean e.g. the mouse competes with the soundcard.

The tweaks may be less important now than back in kernel 2.6 days. One approach would be for users to try the system and use the tweaks to reduce Xruns if necessary, or to get a lower buffer size without Xruns.

Recently I started using my built-in audio for Firefox by bringing the output into a mixer along with my good soundcard. I noticed the performance is very slightly worse, so that makes me think interrupts are important. Of course this isn't particularly scientific as my hardware is old. :)
User avatar
turbidh20
Established Member
Posts: 190
Joined: Sun Aug 04, 2019 12:48 pm
Location: UK
Has thanked: 8 times
Been thanked: 2 times

Re: How much RT optimization is still needed?

Post by turbidh20 »

merlyn wrote: Mon Jul 27, 2020 11:56 pm
turbidh20 wrote:There is one AUR package - 'tuned-git' I use, which when using its 'latency-performance' profile gives a noticeable boost to cutting DSP load.
I tried this and it didn't affect my DSP load. I'm not suggesting there is anything wrong with tuned -- I think I had already done everything that tuned does.
Yes, of course. The thing I like is that I only switch to it for DAW work as it runs my hardware hard and I can just switch back to "balanced" for normal usage. I get the reduction in DSP load even when already running the performance governor. I also turn turbo boost on and off on the CPU as required (even on conservative my temps get very high when compiling).

'latency-performance' changes these settings -

Code: Select all

#
# tuned configuration
#
[main]
summary = Optimize for deterministic performance at the cost of increased power consumption
[cpu]
force_latency = cstate.id:1|3
governor = performance
energy_perf_bias = performance
min_perf_pct = 100
[sysctl]
kernel.sched_min_granularity_ns = 10000000
vm.dirty_ratio = 10
vm.dirty_background_ratio = 3
vm.swappiness = 10
kernel.sched_migration_cost_ns = 5000000
I'm going to look into all this again and maybe I can have a specific profile just for DAW work.
Arch Linux (i7-4790k/GTX970); UMC204HD; MPK Mini MKII; Jack2 (QjackCtl) & Ardour/Mixbus
User avatar
turbidh20
Established Member
Posts: 190
Joined: Sun Aug 04, 2019 12:48 pm
Location: UK
Has thanked: 8 times
Been thanked: 2 times

Re: How much RT optimization is still needed?

Post by turbidh20 »

merlyn wrote: Mon Jul 27, 2020 11:56 pm rtirq -- not everyone seems to get a benefit from this. I have a PCI card and I do. I'm less convinced rtirq is useful for USB audio, as rtirq raises the priority of all USB devices -- that could mean e.g. the mouse competes with the soundcard.
I've just played with this again to be sure and, on my system, it's absolutely needed. Without prioritising xhci (USB) I'm getting many xruns just playing back a session in Mixbus.
Arch Linux (i7-4790k/GTX970); UMC204HD; MPK Mini MKII; Jack2 (QjackCtl) & Ardour/Mixbus
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: How much RT optimization is still needed?

Post by merlyn »

turbidh20 wrote:I've just played with this again to be sure and, on my system, it's absolutely needed. Without prioritising xhci (USB) I'm getting many xruns just playing back a session in Mixbus.
That's good information. I have a USB 1.1 interface which I've used to test this, and to raise its priority I add 'ohci' to rtirq.conf, which means the mouse and keyboard also get a raised priority. Does using 'xhci' mean you can raise the priority of only the soundcard?

This is my rtirq.conf with USB left out at the moment :

Code: Select all

# IRQ thread service names
# (space separated list, from higher to lower priority).
# RTIRQ_NAME_LIST="rtc snd usb i8042" # old
RTIRQ_NAME_LIST="snd_ice1"

# Highest priority.
RTIRQ_PRIO_HIGH=95

# Priority decrease step.
RTIRQ_PRIO_DECR=5

# Lowest priority.
RTIRQ_PRIO_LOW=10

# Whether to reset all IRQ threads to SCHED_OTHER.
RTIRQ_RESET_ALL=1

# On kernel configurations that support it,
# which services should be NOT threaded 
# (space separated list).
#RTIRQ_NON_THREADED="rtc snd"

# Process names which will be forced to the
# highest realtime priority range (99-91)
# (space separated list, from highest to lower priority).
# RTIRQ_HIGH_LIST="timer"
What is your rtirq.conf ?
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: How much RT optimization is still needed?

Post by khz »

merlyn wrote: Mon Jul 27, 2020 11:56 pm swappiness -- can't hurt
I don't have a swap partition anymore since I consider it pointless with 16/32/... GB RAM and more. Swap is also not really wanted in the audio area. So I don't put that in either.
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
turbidh20
Established Member
Posts: 190
Joined: Sun Aug 04, 2019 12:48 pm
Location: UK
Has thanked: 8 times
Been thanked: 2 times

Re: How much RT optimization is still needed?

Post by turbidh20 »

merlyn wrote: Tue Jul 28, 2020 4:59 pmWhat is your rtirq.conf ?

Code: Select all

# IRQ thread service names
# (space separated list, from higher to lower priority).
#RTIRQ_NAME_LIST="snd usb i8042 snd_hda_intel:card1"
RTIRQ_NAME_LIST="i801 xhci"

# Highest priority.
RTIRQ_PRIO_HIGH=90

# Priority decrease step.
RTIRQ_PRIO_DECR=5

# Lowest priority.
RTIRQ_PRIO_LOW=51

# Whether to reset all IRQ threads to SCHED_OTHER.
RTIRQ_RESET_ALL=0

# On kernel configurations that support it,
# which services should be NOT threaded
# (space separated list).
RTIRQ_NON_THREADED="rtc snd"

# Process names which will be forced to the
# highest realtime priority range (99-91)
# (space separated list, from highest to lower priority).
#RTIRQ_HIGH_LIST="timer"
My 204HD comes up as IRQ 18 on i801_smb (driver is xhci), so I added that in. In tests it doesn't appear to make any difference if it's there though as long as xhci is right up there. "ehci" is my other USB interface. My MB has a combo of 6 USB (2 and 3) interfaces - Intel and ASMedia (6 in lsusb).

As you can see from that I did have the onboard Intel card in there until I disabled it in the BIOS. I use loopback (ALSA) for everything non-Jack, apart from the odd game (stuttering audio) where I have to disable Jack and use the 204HD directly (I swap .asoundrc files for Jack on and off, from QJackCtl).

What does "RTIRQ_RESET_ALL" set to "1" actually do?

Edit: to clarify - using xhci puts all 9 USB 3 IRQs at the top! This would appear to be a drawback. Saying that, my 204HD is on IRQ 18 by itself but if I don't put xhci in it doesn't work and xruns abound.
Arch Linux (i7-4790k/GTX970); UMC204HD; MPK Mini MKII; Jack2 (QjackCtl) & Ardour/Mixbus
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: How much RT optimization is still needed?

Post by merlyn »

turbidh20 wrote:What does "RTIRQ_RESET_ALL" set to "1" actually do?
I just tried it and on Arch nothing. :) When I used Ubuntu setting it to '1' meant all other IRQs were on scheduler TS, now on Arch they're on scheduler FF with a priority of 50. I should ask rncbc about that :)

Code: Select all

$ rtirq status

    PID CLS RTPRIO  NI PRI %CPU STAT COMMAND	
    365 FF      95   - 135  0.0 S    irq/20-snd_ice1	
    142 FF      50   -  90  0.0 S    irq/9-acpi	
    156 FF      50   -  90  0.0 S    irq/8-rtc0	
    226 FF      50   -  90  0.0 S    irq/17-ehci_hcd	
    229 FF      50   -  90  0.0 S    irq/14-pata_ati	
    230 FF      50   -  90  0.0 S    irq/15-pata_ati	
    234 FF      50   -  90  0.0 S    irq/19-ehci_hcd	
    235 FF      50   -  90  0.0 S    irq/27-ahci[000	
    257 FF      50   -  90  0.0 S    irq/16-ohci_hcd	
    258 FF      50   -  90  0.0 S    irq/16-ohci_hcd	
    260 FF      50   -  90  0.0 S    irq/18-ohci_hcd	
    261 FF      50   -  90  0.0 S    irq/18-ohci_hcd	
    264 FF      50   -  90  0.0 S    irq/18-ohci_hcd	
    355 FF      50   -  90  0.1 S    irq/16-snd_hda_	
    356 FF      50   -  90  0.0 S    irq/28-snd_hda_	
    371 FF      50   -  90  0.0 S    irq/30-radeon	
    432 FF      50   -  90  0.0 S    irq/29-enp2s0	
      9 TS       -   0  19  0.0 S    ksoftirqd/0	
     23 TS       -   0  19  0.0 S    ksoftirqd/1	
     31 TS       -   0  19  0.0 S    ksoftirqd/2	
     39 TS       -   0  19  0.0 S    ksoftirqd/3
User avatar
sysrqer
Established Member
Posts: 2520
Joined: Thu Nov 14, 2013 11:47 pm
Has thanked: 319 times
Been thanked: 148 times
Contact:

Re: How much RT optimization is still needed?

Post by sysrqer »

khz wrote: Tue Jul 28, 2020 5:10 pm
merlyn wrote: Mon Jul 27, 2020 11:56 pm swappiness -- can't hurt
I don't have a swap partition anymore since I consider it pointless with 16/32/... GB RAM and more. Swap is also not really wanted in the audio area. So I don't put that in either.
I have run out of ram (16gb) with ardour before, which i think is down ardour or a plugin perhaps having a memory leak but it can happen. I think it's worth having swap of some kind and size, I usually do a 1gb swapfile since I have the space for it, since if you reach that level then the system runs slow as hell but you can usually save the project.
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: How much RT optimization is still needed?

Post by khz »

Depending on what you do, if you need several very large/many GB samples in your DAW, it's different. But Swap is no fun. 32 GB and more RAM is not uncommon on a desktop computer these days.
A plugin with a memory leak should not be the rule.
IMHO
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
sysrqer
Established Member
Posts: 2520
Joined: Thu Nov 14, 2013 11:47 pm
Has thanked: 319 times
Been thanked: 148 times
Contact:

Re: How much RT optimization is still needed?

Post by sysrqer »

Well I'm just saying about my experience. I'm not it sure why having a file on disk to be used if necessary could be bad. Even worse is losing your work which has happened to me more than once.
merlyn
Established Member
Posts: 1392
Joined: Thu Oct 11, 2018 4:13 pm
Has thanked: 168 times
Been thanked: 247 times

Re: How much RT optimization is still needed?

Post by merlyn »

I have 11GB (don't ask :)) of RAM and a swap file the same size. Total overkill i know. I have seen the swap file get used even with swappiness=10, so my swappiness is now swappiness=1 and I haven't seen it get used.

Ideally the swap file would never be used and with 32GB I would consider having no swap file.

I ran out of memory once when I made a typo in GIMP and increased the size of an image by 1000% or something else equally stupid. :) As sysrqer said the swap file is just an insurance policy against this sort of thing, and allows you to kill the process before memory and swap run out.
User avatar
khz
Established Member
Posts: 1648
Joined: Thu Apr 17, 2008 6:29 am
Location: German
Has thanked: 42 times
Been thanked: 92 times

Re: How much RT optimization is still needed?

Post by khz »

It cannot hurt to have a swap partition.
I have 32 GB of Ram in my computer and I did not have a swap partition because I don't do much with samples and *hope* only use stable plugins. :-D
. . . FZ - Does humor belongs in Music?
. . GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
Post Reply