nvidia driver compiled with grsec kernel causes Xorg core

[x14sg1]

Hello,

I can compile NVIDIA driver 313.18 with a grsec kernel (grsecurity-2.9.1-3.7.3-201301181518.patch) and it compiles but I see this error
in the dmesg output afterwards:

[ 287.212925] grsec: From 192.168.1.100: denied RWX mprotect of /tmp/nv-tmp-KcpJJe by /tmp/nv-tmp-sIPdsc[nv-tmp-sIPdsc:4854] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:4853] uid/euid:0/0 gid/egid:0/0

When I try to start X, the X server core dumps while trying to initialize GLX
(I did disable MPROTECT on /usr/bin/Xorg)

[ 139.705687] grsec: Segmentation fault occurred at (nil) in /usr/bin/Xorg[X:3686] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/xinit[xinit:3685] uid/euid:0/0 gid/egid:0/0

However, If I do the following, the X server doesn't dump core

1) boot into a non-grsecurity kernel
2) uninstall nvidia
3) reinstall in with "-s -N"
4) boot into a grsec kernel that already has the nvidia kernel module (which was compiled while running a non-grsec kernel)
5) start X and it works

My grsec kernel config is

CONFIG_GRKERNSEC=y
CONFIG_GRKERNSEC_CONFIG_AUTO=y
CONFIG_GRKERNSEC_CONFIG_PRIORITY_SECURITY=y
# CONFIG_GRKERNSEC_KERN_LOCKOUT is not set
# CONFIG_GRKERNSEC_IO is not set
# CONFIG_PAX_USERCOPY is not set
# CONFIG_PAX_MEMORY_UDEREF is not set
CONFIG_PAX_KERNEXEC_MODULE_TEXT=25

and I have disabled the PAX plugins by changing
ifndef DISABLE_PAX_PLUGINS
to
ifdef ENABLE_PAX_PLUGINS

Any help or suggestions are appreciated

[PaX Team]

[ 287.212925] grsec: From 192.168.1.100: denied RWX mprotect of /tmp/nv-tmp-KcpJJe by /tmp/nv-tmp-sIPdsc[nv-tmp-sIPdsc:4854] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[sh:4853] uid/euid:0/0 gid/egid:0/0

i guess the nvidia installer creates some temporary executable which then tries to do runtime codegen, you'd have to disable MPROTECT on that binary (which probably means patching the installer that creates this executable) or just enable softmode while you're installing this driver.

[x14sg1]

I had a while loop running "paxctl -cm /tmp/nv*" but it couldn't get to the
temporary executables before they were run.

The installer is binary

Looks like I will try softmode next.

[x14sg1]

Just some more info

I tracked the Xorg core dump problem to /usr/lib/tls/libnvidia-tls.so.XXX.XX not getting created.

Because of the RWX error listed above when compiling the NVIDIA driver, the library doesn't get created but the build still succeeds