grsec and X11R6

[zedshaw]

Hi folks.

I've been struggling with getting grsecurity and Debian to work (mostly Debian's fault), and have almost got the configuration I want. The only thing that is a little strange is that, when X11R6 starts (no ACLs in place), the screen goes blank and I can't switch to a virtual terminal or anything. I've checked the logs and there's nothing in the syslog/kernel stream (can we get a tag for grsecurity so the grsec messages are easier to find?).

This isn't such a big deal, as the box I finally intend to run this kernel on is a server with no X, but I would like to know what options have the potential to mess wtih X so I can tinker with the configuration.

Here's the specs:

Debian Woody
Kernel 2.4.19
grsecurity 1.9.5
gradm 1.4
All options turned on, with openwall non-exec stacks. Also fails with PaX on.

Thanks. Let me know if there's anything else that is of interest.

[spender]

use chpax to disable pax on the XFree86 binary.

http://pageexec.virtualave.net/chpax.c

-Brad

[boi42]

I recently tried to find out if there was any measurable performance impact when using grsec on a workstation. I tried some old Xmark test to check XFree's 2D-performance - no difference. Good.

Then I tried the Quake3 timedemo to test 3D performance. The funniest thing happened: with demo001 I got 43.7 FPS without grsec, and 55.5 with grsec. As in, better performance with all those extra checks and hurdles in place. I can reproduce it perfectly - retrying or even rebooting and retrying doesn't make a difference.

The system in question is a P3 with an NVidia card in it, NVidia proprietary drivers installed, kernels are 2.4.20 with and without grsec 1.9.8-rc2 patch. For the grsec enhanced kernel, I'm using segmentation-based stack protection and all address space randomization features except the ET_EXEC thingy. I'm not using ACLs (yet). Quake3 won't run without removing stack protection with chpax.

Anyone know why performance is better with all the extra baggage? My only guess would be an accidentally improved address layout with respect to caching.

Cheers,
Boi

[PaX Team]

Then I tried the Quake3 timedemo to test 3D performance. The funniest thing happened: with demo001 I got 43.7 FPS without grsec, and 55.5 with grsec. As in, better performance with all those extra checks and hurdles in place. I can reproduce it perfectly - retrying or even rebooting and retrying doesn't make a difference.

how reliable is the time/frame rate measurement method of quake? also, is it reproducible on other video cards? (just trying to make sure it's not a 'false positive' result)

For the grsec enhanced kernel, I'm using segmentation-based stack protection and all address space randomization features except the ET_EXEC thingy. I'm not using ACLs (yet). Quake3 won't run without removing stack protection with chpax.

what does quake execute in non-executable pages? (note that it's not only the stack)

Anyone know why performance is better with all the extra baggage? My only guess would be an accidentally improved address layout with respect to caching.

i have no idea what it could be, but i don't think it's randomization per se as it is applied to linear (virtual) addresses whereas on IA-32 the caches work on physical addresses (something we don't touch, at least not intentionally . you could try to disable randomization as well (chpax -r) on the X server and/or quake to see if that makes a difference.