change in 'memlock' value of 'limits.conf' in the wiki

How can I make this site better? Let me know what you'd like to see!

Moderators: MattKingUSA, khz

User avatar
Yeri
Established Member
Posts: 158
Joined: Tue Dec 09, 2008 6:33 pm
Location: Spain

change in 'memlock' value of 'limits.conf' in the wiki

Post by Yeri »

Hi,

I've spent some time having problems with the newer version of hydrogen (0.9.4).
I didn't realize what it could be.. different problems reported with gdb.
Gabriel from the hydrogen-devel mailing list suggested that it could be a ram memory problem. So I changed the 'memlock' value in the /etc/security/limits.conf from 500000 (value recommended at the wiki) to 'unlimited'.
Now hydrogen seems to run without problems (at least it starts.. before the change I hadn't managed even to start it)

My suggestion is to change this value in the proposed example of limits.conf file at the 'system tunning and configuration' section of the wiki.
It may help other people to avoid having the same problem as me, since the 'system tunning and configuration section' of the LM wiki was the main source of information I used to tune up my system.

This is my particular case and maybe the value 'unlimited' can have other drawbacks.
What do you think about that ?

Best,

Gerard.

SR
Established Member
Posts: 218
Joined: Wed May 07, 2008 6:01 pm
Location: Houston, Tx

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by SR »

I wouldn't expect it to cause any issues. The Linux desktop systems I support at work are used to run some extremely memory intensive applications. All of our application vendors demand memlock (and a number of other parameters) be unlimited. AFAIK the only drawback is that it allows one user to monopolize all of the system resources.

studio32
Established Member
Posts: 2415
Joined: Mon Apr 07, 2008 5:09 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by studio32 »

I suggest to follow jackaudio.org
@audio - rtprio 99
@audio - memlock unlimited

User avatar
raboof
Established Member
Posts: 1710
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Contact:

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by raboof »

I say Be Bold, Make the Change!

studio32
Established Member
Posts: 2415
Joined: Mon Apr 07, 2008 5:09 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by studio32 »

studio32 wrote:I suggest to follow jackaudio.org
@audio - rtprio 99
@audio - memlock unlimited

could you put this in the wiki Yeri?

adaw
Established Member
Posts: 3
Joined: Wed Jul 29, 2009 5:49 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by adaw »

I had unlimited in my limits.conf but now it looks like this:
# amarokapp seems to swallow all memory during session logout?, limit to 2GB
#@audio - memlock unlimited
@audio - memlock 2097152
I had the problem that sometimes the system did not shut down.
There was nothing I could do. Sometimes the shutdown succeeded after five to ten minutes, sometimes I just switched off the pc after 30 min.
There was no chance to get to the console and the hd was working all the time.

I thought about a fresh install until I found this in my messages file:
Mar 4 18:49:24 callisto kernel: nscd invoked oom-killer: gfp_mask=0x1200d2, order=0, oomkilladj=0
Mar 4 18:49:24 callisto kernel: Pid: 3263, comm: nscd Tainted: P 2.6.26.8-51-rt_1000hz #1
... more messages ...
Mar 4 18:49:34 callisto kernel: Out of memory: kill process 1420 (amarokapp) score 66269 or a child
Mar 4 18:49:34 callisto kernel: Killed process 1420 (amarokapp)
oom-killer is the out of memory killer.

I had amarok with the jack output plugin and qjackctl running when I logged out and the system was again not responding but came back after a while. Then I found the oom-killer messages.
Somehow amarok managed to eat all memory and the system wasn't even able to end the session and thus terminate amarok.
The problem disappeared after I changed limits.conf to half of my system memory which seems to be enough for hydrogen and all other audio apps.

So for short: If you set memlock to unlimited to solve one problem, you might just get another one.
I will always reserve some memory for the system.

User avatar
Yeri
Established Member
Posts: 158
Joined: Tue Dec 09, 2008 6:33 pm
Location: Spain

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by Yeri »

We are trying to discover what makes hydrogen to crash with low 'memlock' values, since it seems not to be consuming too much memory.
adaw wrote:I had the problem that sometimes the system did not shut down.
The problem pointed out by adaw is logical since setting 'memlock' to 'unlimited', could cause a buggy application to freeze the system.
Anyway I consider that it is worth to mention in the wiki that setting 'memlock' to 'unlimited' could prevent from some unexpected and undesired behaviors (with the aforementioned risks).
studio32 wrote:could you put this in the wiki Yeri?
I plan to do the wiki update when I have some more information about this issue, in order to make it as complete as possible.

Best,

Gerard.

User avatar
raboof
Established Member
Posts: 1710
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Contact:

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by raboof »

limits.conf supports setting 'hard' and 'soft' limits.

Perhaps we should consider setting the 'hard' limit to unlimited, but the 'soft' to 500meg or so. Looks like that way, you could award more memory to those applications that need it (like hydrogen, apparently), but your system will be protected against other apps getting out of control.

Seems to work: 'ulimit -l' to read the current value, 'ulimit -l unlimited' to set it to unlimited for the current terminal, and that setting seems to be passed on to applications started from there.

adaw
Established Member
Posts: 3
Joined: Wed Jul 29, 2009 5:49 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by adaw »

How about this for hard limit:
Allow audio group to use 87.5% of your system memory:

Code: Select all

echo Recommended value for memlock: `awk '/MemTotal/ {print $2/8*7}' /proc/meminfo`
I don't know if this works, it's just a guess.


ulimits:
bash: "ulimit -Sl"
bash: "help ulimit"

Code: Select all

> ulimit -Sl 10000
> ulimit -l
10000
> ulimit -l unlimited
bash: ulimit: max locked memory: cannot modify limit: Operation not permitted
> ulimit -Sl unlimited
> ulimit -l
2097152

User avatar
Yeri
Established Member
Posts: 158
Joined: Tue Dec 09, 2008 6:33 pm
Location: Spain

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by Yeri »

wiki update done.

I've decided not to mention anything about 'hard' and 'soft' limits because in most cases, setting the soft limit to a lower value than 'unlimited' (and the 'hard' to unlimited) results with the same 'problem'.
Applications can raise themselves the soft limit inside the boundaries of the hard limit, but it seems that if they don't explicitly do that, it happens as if both were set like the 'soft'.

User avatar
raboof
Established Member
Posts: 1710
Joined: Tue Apr 08, 2008 11:58 am
Location: Deventer, NL
Contact:

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by raboof »

Yeri wrote:wiki update done.
Looking good!
I've decided not to mention anything about 'hard' and 'soft' limits because in most cases, setting the soft limit to a lower value than 'unlimited' (and the 'hard' to unlimited) results with the same 'problem'.
Applications can raise themselves the soft limit inside the boundaries of the hard limit, but it seems that if they don't explicitly do that, it happens as if both were set like the 'soft'.
Correct. However, you can also manually raise the 'soft' limit for specific applications such as hydrogen with ulimit.

I agree we should not recommend this on the wiki without a good explanation of what this means and how to use it, but it sounds like a desirable way of doing things.

studio32
Established Member
Posts: 2415
Joined: Mon Apr 07, 2008 5:09 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by studio32 »

Nice job. This thread is a good example about a good interaction between forum and wiki, and getting a very useful WIKI.

Thanks

studio32
Established Member
Posts: 2415
Joined: Mon Apr 07, 2008 5:09 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by studio32 »

We have nothing in the wiki about ALSA Sequencer right?
If you can not have midi sequencing enabled, or if you have midi error message, the Alsa midi sequencer module of the kernel may not be loaded. Try that:

sudo modprobe snd-seq

If it works, to have your system load it at bootup, you have to add it to the /etc/modules file, do:

sudo su -c 'echo snd-seq >> /etc/modules'

adaw
Established Member
Posts: 3
Joined: Wed Jul 29, 2009 5:49 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by adaw »

I did some research.
The memlock value is the amount of memory in KB that a process can lock into the physical memory (ram), this memory cannot be moved to the page file (swap) which avoids time consuming page in/out. See mlock(2)

There is also "the maximum resident set size", which seems to be related with memlock, not shure about that.

You can lookup the actual limit of a process in the proc(5) filesystem.
e.g for jackd
> cat /proc/$(pgrep jackd)/limits
You can also lookup the locked memory of a process
> grep VmLck /proc/$(pgrep jackd)/status
VmLck: 37824 kB
> grep VmLck /proc/$(pgrep hydrogen)/status
VmLck: 834592 kB
You can see that 500000 is not enough for hydrogen, but 1000000 would probably be enough.

I found a (suid) program that you can use to alter hard limits:
set_rlimits
Having
@audio /bin/bash memlock=4021352
In /etc/set_rlimits.conf (4021352 is MemTotal in my /proc/meminfo) gives me the possibility to get a shell with the maximum of memlock with
set_rlimits bash
set_rlimits does not work with memlock=unlimited

studio32
Established Member
Posts: 2415
Joined: Mon Apr 07, 2008 5:09 pm

Re: change in 'memlock' value of 'limits.conf' in the wiki

Post by studio32 »

I got the advise in jack messages not to use unlimited but @audio - memlock 971112
in /etc/limits.conf instead...

Post Reply