Quick kernel benchmark and discussion

[barbouze]

Hi!

After reading this article, I decided to check my setup realtime capabilities depending on kernel choices.
I'm using Linux Mint 18.1 on an i5 2500K with 8gB of ram and for each benchmark, after a clean boot I used the provided script running the following command:

cyclictest -l300000 -m -n -Sp80 -i500 -h400 -q

after having launched

(while true; do hackbench > /dev/null; done) &

The tested kernels were:
- generic kernel provided by Ubuntu
- low latency kernel from Ubuntu repos
- Liquorix kernel
- patched kernel following this wiki page

Each plot is named following this convention: "kernel version - CPU scaling gorvernor"

4.10.0-24-generic #28~16.04.1-Ubuntu SMP x86_64 - performance.png (7.09 KiB) Viewed 4030 times

4.10.0-24-lowlatency #28~16.04.1-Ubuntu SMP PREEMPT x86_64 - performance.png (6.97 KiB) Viewed 4030 times

4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - ondemand.png (6.49 KiB) Viewed 4030 times

[barbouze]

4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - performance.png (6.43 KiB) Viewed 4029 times

4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - schedutil.png (6.43 KiB) Viewed 4029 times

4.11.5-rt1 #1 SMP PREEMPT RT x86_64 - performance.png (6.13 KiB) Viewed 4029 times

[barbouze]

I was quite surprised by the results, specifically regarding the Liquorix kernel and the RT one. The first has consistent high latencies peaks despite a great advantage over the lowlatency version and the latest outperforms by a wide margin all the others.

I'm not an expert in data analysis nor system benchmarks so take this with a pinch of salt

[sysrqer]

That's an interesting comparison although I can't say I fully understand what it shows.
I would be interested to know how this kernel fairs in your tests - https://launchpad.net/~nick-athens30/+a ... xenialmub/
This is what I've been using lately as there is currently a bug affecting me with resuming from suspend with Liquorix.

[barbouze]

That's an interesting comparison although I can't say I fully understand what it shows.

Quoted directly from this PDF : Cyclictest Measures latency of response to a stimulus. And hackbench is "both a benchmark and a stress test for the Linux kernel scheduler".

So those plots shows the realtime latency of my system under stress for a given number of times (300000 in this case). Logically, less is better and max latency should be as low as possible. I will test your kernel as soon as possible.

Imho, there is something to dig here and I would be glad to bring, individually or as a collective effort, a simple analysis tool for system tuning like raboof's realtimeconfigquickscan

To be honest, I was quite surprised to see that my system was less responsive than a Raspberry Pi3 but then I got it: I have far more processing power but my system is a heavyweight designed for general desktop usage with some background processes running thus the higher latencies.

[barbouze]

Here are the results:

4.9.0-340-mub #201706241843-Ubuntu SMP PREEMPT x86_64 - performance.png (6.53 KiB) Viewed 4001 times

Imho, not quite different from the Liquorix one, less consistent as it can give latency spikes a bit more.

[Jack Winter]

What cyclictest does is to test the kernel scheduling latency, that is to say how long it might take from asking the kernel to run a thread until it actually starts running it. It does this by trying to run threads at a certain time, and then checking how long it took the kernel to actually start running the thread.

This is the principal benefit of using a low latency or realtime kernel. It's not going to work very well to run with a buffersize giving a deadline of 1.4ms when it occasionally takes the kernel already 2ms to start running a thread. My i7 desktop with intel gpu, seems to manage a max scheduling delay of 80-90us or so. With just a lowlatency kernel or say with a nividia gpu or other troublesome hardware/driver, that goes up into the single or even double digit ms range. And yes hackbench is a good way to put a high load on the system which will speed up detecting peaks in scheduling latency..

[autostatic]

Thanks for these tests! So the stock Ubuntu kernels are doing quite well and especially a lowlatency one is already basically all you need. I will test some more on the RPi also with the standard kernel. If the differences are as small as in your case I might as well use a standard kernel.

[barbouze]

Thanks for these tests! So the stock Ubuntu kernels are doing quite well and especially a lowlatency one is already basically all you need. I will test some more on the RPi also with the standard kernel. If the differences are as small as in your case I might as well use a standard kernel.

You're welcome! I agree that using the lowlatency flavor is the easiest way to go for a quick boost as it is already in the Ubuntu repos. Again, this is on my setup so results may vary on yours, specially on a RPi.

[sysrqer]

Here are the results:
Imho, not quite different from the Liquorix one, less consistent as it can give latency spikes a bit more.

Thanks. That's pretty much was I was expecting, I think it is quite similar to the Liquorix kernel in the patches it uses.