PaX issues:

[bugboom]

Hardware: Dual Pentium 4 Xeon(Irwindale) 2.8GHz, 2GB ECC RAM.

With grsecurity 10/31/07, and also on earlier 2.6.23 patches applied against 2.6.23.1, with PaX enabled, the error logs(and consoles) fill up with these messages, and the machine slowly becomes unusable afterwards:

kernel: Bad page state in process 'httpd'
kernel: page:c186aae0 flags:0x80000001 mapping:00000000 mapcount:0 count:0
kernel: Trying to fix it up, but a reboot is needed

This is with Apache 2.0.61 with the worker MPM. Booting the kernel with nosmp "solves" this problem, but obviously isn't a real solution.

(I didn't include the backtrace, as it would be meaningless, as I've compiled the kernel with kernel symbols disabled, and since it's a fairly heavily-loaded production machine, I really can't do a ton of testing)

I also disabled PaX using chpax for mysql(4.0.24):

chpax -v /usr/sbin/mysqld

----[ chpax 0.7 : Current flags for /usr/sbin/mysqld (peMrxs) ]----

* Paging based PAGE_EXEC : disabled
* Trampolines : not emulated
* mprotect() : restricted
* mmap() base : not randomized
* ET_EXEC base : not randomized
* Segmentation based PAGE_EXEC : disabled

...but the mysqld process' virtual memory usage grows and grows and grows over time, eventually causing it to seg fault. This doesn't happen when PaX is disabled at kernel configuration/compilation time.

I'll post the kernel .config file and more information later when I have time...

[PaX Team]

Hardware: Dual Pentium 4 Xeon(Irwindale) 2.8GHz, 2GB ECC RAM.

what of PaX did you enable on this? in particular, i'm wondering about SEGMEXEC as that's the one that plays with page locking for vma mirroring (the PageLocked bit is the one set in the flags field that triggers the error message). since your chip has hw NX support, you could try to run without SEGMEXEC (if it was enabled) and see if it changes the behaviour (so that i know it is indeed a problem in vma mirroring).

(I didn't include the backtrace, as it would be meaningless, as I've compiled the kernel with kernel symbols disabled, and since it's a fairly heavily-loaded production machine, I really can't do a ton of testing)

actually the backtrace (the full logs in fact) is important as well, even if you didn't enable kernel symbols because i can track them back from System.map (obviously i'll need that file too . so can you email me these logs + corresponding System.map please?

I also disabled PaX using chpax for mysql(4.0.24):
[...]
...but the mysqld process' virtual memory usage grows and grows and grows over time, eventually causing it to seg fault. This doesn't happen when PaX is disabled at kernel configuration/compilation time.

can you determine which mapping in mysqld grows beyond limits (if anything)? taking a few snapshots of /proc/pid/maps (and maybe smaps) would help determine that.

[bugboom]

I've emailed the requested files.

[PaX Team]

I've emailed the requested files.

i hope i've fixed this in the latest test patch, can you give it a try please?