Page 1 of 1

Quick kernel benchmark and discussion

Posted: Wed Jun 28, 2017 10:59 am
by barbouze
Hi!

After reading this article, I decided to check my setup realtime capabilities depending on kernel choices.
I'm using Linux Mint 18.1 on an i5 2500K with 8gB of ram and for each benchmark, after a clean boot I used the provided script running the following command:
cyclictest -l300000 -m -n -Sp80 -i500 -h400 -q
after having launched
(while true; do hackbench > /dev/null; done) &
The tested kernels were:
- generic kernel provided by Ubuntu
- low latency kernel from Ubuntu repos
- Liquorix kernel
- patched kernel following this wiki page

Each plot is named following this convention: "kernel version - CPU scaling gorvernor"
4.10.0-24-generic #28~16.04.1-Ubuntu SMP x86_64 - performance.png
4.10.0-24-generic #28~16.04.1-Ubuntu SMP x86_64 - performance.png (7.09 KiB) Viewed 3679 times
4.10.0-24-lowlatency #28~16.04.1-Ubuntu SMP PREEMPT x86_64 - performance.png
4.10.0-24-lowlatency #28~16.04.1-Ubuntu SMP PREEMPT x86_64 - performance.png (6.97 KiB) Viewed 3679 times
4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - ondemand.png
4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - ondemand.png (6.49 KiB) Viewed 3679 times

Re: Quick kernel benchmark and discussion

Posted: Wed Jun 28, 2017 11:02 am
by barbouze
4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - performance.png
4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - performance.png (6.43 KiB) Viewed 3678 times
4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - schedutil.png
4.11.0-6.1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 4.11-13 x86_64 - schedutil.png (6.43 KiB) Viewed 3678 times
4.11.5-rt1 #1 SMP PREEMPT RT x86_64 - performance.png
4.11.5-rt1 #1 SMP PREEMPT RT x86_64 - performance.png (6.13 KiB) Viewed 3678 times

Re: Quick kernel benchmark and discussion

Posted: Wed Jun 28, 2017 11:06 am
by barbouze
I was quite surprised by the results, specifically regarding the Liquorix kernel and the RT one. The first has consistent high latencies peaks despite a great advantage over the lowlatency version and the latest outperforms by a wide margin all the others.

I'm not an expert in data analysis nor system benchmarks so take this with a pinch of salt :D

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 9:55 am
by sysrqer
That's an interesting comparison although I can't say I fully understand what it shows.
I would be interested to know how this kernel fairs in your tests - https://launchpad.net/~nick-athens30/+a ... xenialmub/
This is what I've been using lately as there is currently a bug affecting me with resuming from suspend with Liquorix.

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 10:26 am
by barbouze
That's an interesting comparison although I can't say I fully understand what it shows.
Quoted directly from this PDF : Cyclictest Measures latency of response to a stimulus. And hackbench is "both a benchmark and a stress test for the Linux kernel scheduler".

So those plots shows the realtime latency of my system under stress for a given number of times (300000 in this case). Logically, less is better and max latency should be as low as possible. I will test your kernel as soon as possible.

Imho, there is something to dig here and I would be glad to bring, individually or as a collective effort, a simple analysis tool for system tuning like raboof's realtimeconfigquickscan :D

To be honest, I was quite surprised to see that my system was less responsive than a Raspberry Pi3 but then I got it: I have far more processing power but my system is a heavyweight designed for general desktop usage with some background processes running thus the higher latencies.

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 10:53 am
by barbouze
Here are the results:
4.9.0-340-mub #201706241843-Ubuntu SMP PREEMPT x86_64 - performance.png
4.9.0-340-mub #201706241843-Ubuntu SMP PREEMPT x86_64 - performance.png (6.53 KiB) Viewed 3650 times
Imho, not quite different from the Liquorix one, less consistent as it can give latency spikes a bit more.

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 12:17 pm
by Jack Winter
What cyclictest does is to test the kernel scheduling latency, that is to say how long it might take from asking the kernel to run a thread until it actually starts running it. It does this by trying to run threads at a certain time, and then checking how long it took the kernel to actually start running the thread.

This is the principal benefit of using a low latency or realtime kernel. It's not going to work very well to run with a buffersize giving a deadline of 1.4ms when it occasionally takes the kernel already 2ms to start running a thread. My i7 desktop with intel gpu, seems to manage a max scheduling delay of 80-90us or so. With just a lowlatency kernel or say with a nividia gpu or other troublesome hardware/driver, that goes up into the single or even double digit ms range. And yes hackbench is a good way to put a high load on the system which will speed up detecting peaks in scheduling latency..

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 5:24 pm
by autostatic
Thanks for these tests! So the stock Ubuntu kernels are doing quite well and especially a lowlatency one is already basically all you need. I will test some more on the RPi also with the standard kernel. If the differences are as small as in your case I might as well use a standard kernel.

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 7:37 pm
by barbouze
AutoStatic wrote:Thanks for these tests! So the stock Ubuntu kernels are doing quite well and especially a lowlatency one is already basically all you need. I will test some more on the RPi also with the standard kernel. If the differences are as small as in your case I might as well use a standard kernel.
You're welcome! I agree that using the lowlatency flavor is the easiest way to go for a quick boost as it is already in the Ubuntu repos. Again, this is on my setup so results may vary on yours, specially on a RPi.

Re: Quick kernel benchmark and discussion

Posted: Thu Jun 29, 2017 9:11 pm
by sysrqer
barbouze wrote:Here are the results:
Imho, not quite different from the Liquorix one, less consistent as it can give latency spikes a bit more.
Thanks. That's pretty much was I was expecting, I think it is quite similar to the Liquorix kernel in the patches it uses.