Threadirqs kernel option with generic kernels

Optimize your system for ultimate performance.

Moderators: khz, MattKingUSA

User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Threadirqs kernel option with generic kernels

Post by lilith »

I installed the rtirq script from Rui and have modified Grub according to:

https://wiki.linuxaudio.org/wiki/system_configuration

i.e.

GRUB_CMDLINE_LINUX="threadirqs" and updated grub.

However when running

Code: Select all

grep -e "CONFIG_IRQ_FORCED_THREADING=y" -e "CONFIG_PREEMPT=y" /boot/config-`uname -r`
I only get CONFIG_IRQ_FORCED_THREADING=y. CONFIG_PREEMPT=y is missing.

I'm using the generic Debian 10 kernel:

Linux marco 4.19.0-8-amd64 #1 SMP Debian 4.19.98-1 (2020-01-26) x86_64 GNU/Linux

Is this normal, unimportant or am I missing something?
User avatar
thetotalchaos
Established Member
Posts: 186
Joined: Mon Sep 29, 2014 8:29 pm
Contact:

Re: Threadirqs kernel option with generic kernels

Post by thetotalchaos »

You are missing that the generic Debian Linux kernel doesn't have PREEMPT option enabled like:

Code: Select all

biser@totalchaos:~$ uname -a
Linux totalchaos 4.19.0-6-lzk-rt-amd64 #1 SMP PREEMPT RT Debian 4.19.67-2 (2019-08-28) x86_64 GNU/Linux
biser@totalchaos:~$ 
I am currently using a custom realtime kernel from the LibraZiK repository. But the default Archlinux kernel, or the Linux-Lowlatency used by Ubuntu Studio will output PREEMPT nevertheless. Debian turns off this option to provide more stability for servers, but provides Linux RT in their official repositories. You should use it instead.
Check out my latest music album Le Femme Robotique 2021
https://totalchaos-music.bandcamp.com/a ... -viii-2006
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

thetotalchaos wrote: Sat Mar 14, 2020 9:35 pm You are missing that the generic Debian Linux kernel doesn't have PREEMPT option enabled like:

Code: Select all

biser@totalchaos:~$ uname -a
Linux totalchaos 4.19.0-6-lzk-rt-amd64 #1 SMP PREEMPT RT Debian 4.19.67-2 (2019-08-28) x86_64 GNU/Linux
biser@totalchaos:~$ 
I am currently using a custom realtime kernel from the LibraZiK repository. But the default Archlinux kernel, or the Linux-Lowlatency used by Ubuntu Studio will output PREEMPT nevertheless. Debian turns off this option to provide more stability for servers, but provides Linux RT in their official repositories. You should use it instead.
I see. So does that mean that all this rtirq thing does not work with my generic kernel? Because the heading in the wiki is "This is only needed for so-called generic kernels, ie. standard kernels that are not tweaked for lowlatency performance. You can check if your kernel already includes this option with the following command". I thought it's especially for generic kernels.
User avatar
thetotalchaos
Established Member
Posts: 186
Joined: Mon Sep 29, 2014 8:29 pm
Contact:

Re: Threadirqs kernel option with generic kernels

Post by thetotalchaos »

Regardless of how the particular Linux kernel is called, if you don't have PREEMPT option enabled, you cannot assign priorities whatsoever. Neither by Jack nor Rtirq. If you have PREEMPT enabled, but not RT, you can assign priorities up to 89. Have that in mind when you set your system. The reason is that values between 90-99 can interrupt other processes in the CPU. This is the magic in the Realtime kernels in one sentence. :wink:
Check out my latest music album Le Femme Robotique 2021
https://totalchaos-music.bandcamp.com/a ... -viii-2006
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

thetotalchaos wrote: Sat Mar 14, 2020 9:49 pm Regardless of how the particular Linux kernel is called, if you don't have PREEMPT option enabled, you cannot assign priorities whatsoever. Neither by Jack nor Rtirq. If you have PREEMPT enabled, but not RT, you can assign priorities up to 89. Have that in mind when you set your system. The reason is that values between 90-99 can interrupt other processes in the CPU. This is the magic in the Realtime kernels in one sentence. :wink:
Hmm.. I always thought this will work with kernels >= 2.6.31 as written in the wiki. I'll try the RT kernel also.
User avatar
khz
Established Member
Posts: 1449
Joined: Thu Apr 17, 2008 6:29 am
Location: German

Re: Threadirqs kernel option with generic kernels

Post by khz »

You can install a more recent RT kernel via the backports. https://wiki.debian.org/Backports

Code: Select all

apt-get -t buster-backports install linux-image-rt-amd64
https://packages.debian.org/buster-back ... e-rt-amd64
FZ - Does humor belongs in Music?
GNU/LINUX@AUDIO ~ /Wiki $ Howto.Info && GNU/Linux Debian installing >> Linux Audio Workstation LAW
  • I don't care about the freedom of speech because I have nothing to say.
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

khz wrote: Sun Mar 15, 2020 7:43 am You can install a more recent RT kernel via the backports. https://wiki.debian.org/Backports

Code: Select all

apt-get -t buster-backports install linux-image-rt-amd64
https://packages.debian.org/buster-back ... e-rt-amd64
Thanks, I installed the standard RT kernel from Debian and it makes a huge difference! I will double check, but here are some findings

When using the RT kernel I can run Renoise and run the xruncounterscript in parallel. I get the first xrun at > 90% typically.
With the generic non RT PREEMPT kernel the first xrun can happen at already 20- 30%.

With RT kernel the CPU load is evenly distributed across the 4 CPUs, which is not the case for the generic kernel. I have to double check this.
With RT kernel I didn't get any sporadic xrun up to now, which was the case after installing rtirq, but without using the RT kernel.

Does anybody know what happens if one is using the non RT PREEMPT kernel with rtirq? Will it just don't work? Obviously it caused sporadic xruns, so somthing must have happend.
tramp
Established Member
Posts: 1761
Joined: Mon Jul 01, 2013 8:13 am

Re: Threadirqs kernel option with generic kernels

Post by tramp »

I wonder what you wonder about.
What did you think, do, the developers who do the rt-kernel patches does ? it is just to get a extension on the kernel name? :lol:
Well, the rt-kernel patches allow priority's for user space tasks, which aren't allowed by "normal" non PREEMPT kernel, and exactly that is, what we, as pro audio users been after. While in early day's of the rt-project, a lot of issues appears when user space could use though high priority's, this day's nearly all culprits been resolved. a lot work has been put into the kernel scheduler, to make all this work smooth.
But just that most of this patches been accepted in upstream, didn't mean that they are switched on by default. At least, debian, for example, is in use on servers much more then on desktop systems, and imagine yourself, how many of this desktop systems been meant to be used as "pro audio system".
So, what did that mean? You need to be aware that you've to sail your ship always a bit outside of the main floor. Otherwise you end up with a "perfect server system". :lol:
On the road again.
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

tramp wrote: Sun Mar 15, 2020 5:48 pm I wonder what you wonder about.
What did you think, do, the developers who do the rt-kernel patches does ? it is just to get a extension on the kernel name? :lol:
Well, the rt-kernel patches allow priority's for user space tasks, which aren't allowed by "normal" non PREEMPT kernel, and exactly that is, what we, as pro audio users been after. While in early day's of the rt-project, a lot of issues appears when user space could use though high priority's, this day's nearly all culprits been resolved. a lot work has been put into the kernel scheduler, to make all this work smooth.
But just that most of this patches been accepted in upstream, didn't mean that they are switched on by default. At least, debian, for example, is in use on servers much more then on desktop systems, and imagine yourself, how many of this desktop systems been meant to be used as "pro audio system".
So, what did that mean? You need to be aware that you've to sail your ship always a bit outside of the main floor. Otherwise you end up with a "perfect server system". :lol:
Yes, I always thought the stock kernels will do the job and I did tests in the past comparing these two and found no difference. Pretty sure there was something wrong. I just got a xrun again. Can these be caused in general by other processes with a higher priority than that of my soundcard (95), e.g. watchdog or other stuff that's listed here?

Code: Select all

    99  FF    61 139   - [watchdogd]
    99  FF    38 139   - [posixcputmr/3]
    99  FF    37 139   - [migration/3]
    99  FF    30 139   - [posixcputmr/2]
    99  FF    29 139   - [migration/2]
    99  FF    22 139   - [posixcputmr/1]
    99  FF    21 139   - [migration/1]
    99  FF    17 139   - [migration/0]
    99  FF    16 139   - [posixcputmr/0]
    95  FF   190 135   - [irq/28-xhci_hcd]
    95  FF  1393 135   - /home/marco/Schreibtisch/renoise
tramp
Established Member
Posts: 1761
Joined: Mon Jul 01, 2013 8:13 am

Re: Threadirqs kernel option with generic kernels

Post by tramp »

Often it is hard to say what exactly trigger a xrun.
For my system, I tried very hard to find the best boot parameters. I end up with:

Code: Select all

ro quiet systemd.show_status=1 threadirqs intel_pstate=disable processor.max_cstate=1 intel_idle.max_cstate=0 idle=poll
on a Quad Core model: Intel Core i5-7400 with Kernel: 5.4.24-rt15

I didn't use the rtirq script or any additional priority settings other then the default.
On the road again.
tramp
Established Member
Posts: 1761
Joined: Mon Jul 01, 2013 8:13 am

Re: Threadirqs kernel option with generic kernels

Post by tramp »

I should mention that I build the kernel tick-less, so no posixcputmr is running.

Code: Select all

    99  FF 139 [watchdogd]
    99  FF 139 [migration/3]
    99  FF 139 [migration/2]
    99  FF 139 [migration/1]
    99  FF 139 [migration/0]
    85  FF 125 /usr/bin/jackd -P85 -dalsa -dhw:0 -r48000 -p128 -n2 -s -S -Xseq
    80  FF 120 mhwaveedit
    50  FF  90 [irq/9-acpi]
    50  FF  90 [irq/8-rtc0]
    50  FF  90 [irq/1-i8042]
    50  FF  90 [irq/17-snd_hda_]
On the road again.
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

lilith wrote: Sun Mar 15, 2020 3:46 pm
khz wrote: Sun Mar 15, 2020 7:43 am You can install a more recent RT kernel via the backports. https://wiki.debian.org/Backports

Code: Select all

apt-get -t buster-backports install linux-image-rt-amd64
https://packages.debian.org/buster-back ... e-rt-amd64
Thanks, I installed the standard RT kernel from Debian and it makes a huge difference! I will double check, but here are some findings

When using the RT kernel I can run Renoise and run the xruncounterscript in parallel. I get the first xrun at > 90% typically.
With the generic non RT PREEMPT kernel the first xrun can happen at already 20- 30%.

With RT kernel the CPU load is evenly distributed across the 4 CPUs, which is not the case for the generic kernel. I have to double check this.
With RT kernel I didn't get any sporadic xrun up to now, which was the case after installing rtirq, but without using the RT kernel.

Does anybody know what happens if one is using the non RT PREEMPT kernel with rtirq? Will it just don't work? Obviously it caused sporadic xruns, so somthing must have happend.
Now it's worse again with the RT PREEMPT Kernel. I don't know what's going on here.

Code: Select all

******************** SYSTEM CHECK *********************

    Sound Playback: USB-Audio - ZOOM R8
     Sound Capture: USB-Audio - ZOOM R8
      Graphic Card: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller (rev 06)
  Operating System: Debian GNU/Linux 10 (buster)
            Kernel: Linux 4.19.0-8-rt-amd64
      Architecture: x86-64
               CPU: Intel(R) Core(TM) i5-4460  CPU @ 3.20GHz

****************** jackd dbus running ******************


********************** Pulseaudio **********************

Connection failure: Connection refused
pa_context_connect() failed: Connection refused
    pulse is not active

********************** Test 1 Core *********************

Samplerate is 48000Hz 
Buffersize is 1024 
Buffer/Periods  3
jack running with realtime priority 80
Xrun 1 at DSP load 79.54% use 16.64ms from 21.34ms jack cycle time
Last time I tried I repeated it 3 times and it went always above 95%
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

I deinstalled rtirq together with the kxdefault settings and now the xruncounterscript crashes with segfault again.

Code: Select all

marco@marco:~/realtimeconfig/realtimeconfigquickscan$ cd ~/src/xruncounter/xruncounter-master/
marco@marco:~/src/xruncounter/xruncounter-master$ ./xruncounter 

******************** SYSTEM CHECK *********************

Segmentation fault
And in the journal I get:

Code: Select all

Mär 20 01:16:47 marco kernel: xruncounter[1410]: segfault at 0 ip 00007fc1968e35d7 sp 00007ffcb1f30eb8 error 4 in libc-2.28.so[7fc196865000+148000]
@tramp : Do you know why that happens? I'm pretty sure that the script ran in the past without the kxdefault settings.
tramp
Established Member
Posts: 1761
Joined: Mon Jul 01, 2013 8:13 am

Re: Threadirqs kernel option with generic kernels

Post by tramp »

I've no idea.
In order to debug this, you may build a xruncounter debugging version with

Code: Select all

gcc -g -Wall xruncounter.c -lm `pkg-config --cflags --libs jack` -o xruncounter
and then run it in gdb.

Code: Select all

gdb ./xruncounter
write "run" on the prompt and when it crashed write "bt full". Put the output here and I'll have a look.
On the road again.
User avatar
lilith
Established Member
Posts: 1494
Joined: Fri May 27, 2016 11:41 pm
Location: bLACK fOREST
Contact:

Re: Threadirqs kernel option with generic kernels

Post by lilith »

tramp wrote: Fri Mar 20, 2020 7:13 am I've no idea.
In order to debug this, you may build a xruncounter debugging version with

Code: Select all

gcc -g -Wall xruncounter.c -lm `pkg-config --cflags --libs jack` -o xruncounter
and then run it in gdb.

Code: Select all

gdb ./xruncounter
write "run" on the prompt and when it crashed write "bt full". Put the output here and I'll have a look.
Thanks, I'll try this later.
Post Reply