grsecurity forums

Hi,
I'm experiencing a +3 points on loadaverage with a grsecurity (1.9.9e) enabled kernel (2.4.20), with all the rand_* set to 0 in proc. So if my previous load average was about 2 / 3, now it's 5 / 6, with spikes to 10 / 12.

Is this normal?

littlejohn
PS
The server is a biproc P4 1Ghz, with 1Gb ram (just to give you an idea).

Are you using the page-based implementation of PaX or the segmentation-based implementation? Check your kernel configuration. You should be using the segmentation-based implementation, as it has a negligible performance hit. If grsecurity is causing the additional hit, the only thing it could be is the page-based implementation of PaX.

-Brad

this is my configuration:

#
# Grsecurity
#
CONFIG_GRKERNSEC=y
# CONFIG_GRKERNSEC_LOW is not set
# CONFIG_GRKERNSEC_MID is not set
# CONFIG_GRKERNSEC_HI is not set
CONFIG_GRKERNSEC_CUSTOM=y

#
# Address Space Protection
#
CONFIG_GRKERNSEC_PAX_NOEXEC=y
# CONFIG_GRKERNSEC_PAX_PAGEEXEC is not set
# CONFIG_GRKERNSEC_PAX_SEGMEXEC is not set
# CONFIG_GRKERNSEC_PAX_MPROTECT is not set
# CONFIG_GRKERNSEC_PAX_ASLR is not set
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_RTC=y
# CONFIG_GRKERNSEC_PROC_MEMMAP is not set
# CONFIG_GRKERNSEC_HIDESYM is not set

#
# ACL options
#
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
CONFIG_GRKERNSEC_ACL_TIMEOUT=30

#
# Filesystem Protections
#
CONFIG_GRKERNSEC_PROC=y
CONFIG_GRKERNSEC_PROC_USER=y
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_MOUNT=y
CONFIG_GRKERNSEC_CHROOT_DOUBLE=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_CHMOD=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_FINDTASK=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_CHROOT_CAPS=y

#
# Kernel Auditing
#
CONFIG_GRKERNSEC_AUDIT_GROUP=y
CONFIG_GRKERNSEC_AUDIT_GID=1007
CONFIG_GRKERNSEC_EXECLOG=y
CONFIG_GRKERNSEC_RESLOG=y
CONFIG_GRKERNSEC_CHROOT_EXECLOG=y
CONFIG_GRKERNSEC_AUDIT_CHDIR=y
CONFIG_GRKERNSEC_AUDIT_MOUNT=y
CONFIG_GRKERNSEC_AUDIT_IPC=y
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_TIME=y

#
# Executable Protections
#
CONFIG_GRKERNSEC_EXECVE=y
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_RANDPID=y
CONFIG_GRKERNSEC_TPE=y
# CONFIG_GRKERNSEC_TPE_ALL is not set
CONFIG_GRKERNSEC_TPE_GID=1005
#
# Network Protections
#
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_RANDISN=y
CONFIG_GRKERNSEC_RANDID=y
CONFIG_GRKERNSEC_RANDSRC=y
CONFIG_GRKERNSEC_RANDRPC=y
CONFIG_GRKERNSEC_RANDPING=y
# CONFIG_GRKERNSEC_SOCKET is not set

#
# Sysctl support
#
CONFIG_GRKERNSEC_SYSCTL=y

#
# Logging options
#
CONFIG_GRKERNSEC_FLOODTIME=30
CONFIG_GRKERNSEC_FLOODBURST=4

it seems the PaX pagexec is disabled.

in /proc I have:

kernel.grsecurity.tpe = 0
kernel.grsecurity.exec_logging = 0
kernel.grsecurity.rand_rpc = 0
kernel.grsecurity.dmesg = 0
kernel.grsecurity.audit_mount = 0
kernel.grsecurity.audit_chdir = 0
kernel.grsecurity.audit_ipc = 0
kernel.grsecurity.altered_pings = 1
kernel.grsecurity.rand_isns = 0
kernel.grsecurity.rand_tcp_src_ports = 0
kernel.grsecurity.rand_ip_ids = 0
kernel.grsecurity.rand_pids = 0
kernel.grsecurity.chroot_caps = 0
kernel.grsecurity.chroot_execlog = 1
kernel.grsecurity.chroot_enforce_chdir = 0
kernel.grsecurity.chroot_restrict_nice = 1
kernel.grsecurity.chroot_deny_mknod = 1
kernel.grsecurity.chroot_deny_chmod = 1
kernel.grsecurity.chroot_deny_pivot = 1
kernel.grsecurity.chroot_deny_chroot = 1
kernel.grsecurity.chroot_deny_fchdir = 0
kernel.grsecurity.chroot_deny_mount = 1
kernel.grsecurity.chroot_deny_shmat = 0
kernel.grsecurity.chroot_deny_unix = 0
kernel.grsecurity.chroot_findtask = 0
kernel.grsecurity.chroot_deny_sysctl = 0
kernel.grsecurity.timechange_logging = 0

sorry for the long post.

littlejohn

I don't think grsecurity is the cause.

-Brad

spender wrote:I don't think grsecurity is the cause.

-Brad

Strange to say, but it seems to me grsecurity is the cause of load increase. I meanly have about 3000 processes generated (most of them are fork() and exec() performed by deamons) in 10 minutes. With a 2.4.18 vanilla kernel (using the same .config of the grsec kernel) the load is beetween 1 and 2, with spikes to 4. With a 2.4.20 grsecurity patched kernel (using the options i posted in the previous posts) the load rises to 4-5.

dunno,
lj

littlejohn wrote:Strange to say, but it seems to me grsecurity is the cause of load increase. I meanly have about 3000 processes generated (most of them are fork() and exec() performed by deamons) in 10 minutes.

what kind of daemons are these and what do they do mainly (in terms of syscalls)? my bet is that for whatever reason there's some lock contention in the kernel, so knowing how to reproduce it or at least what kind of activity exacerbates it will help finding it.

PaX Team wrote:what kind of daemons are these and what do they do mainly (in terms of syscalls)?

the daemons running and generating the load are almost network related ones:
apache
apache-ssl
courier (imap and pop3 with and without ssl support, with all the related helper programs. courier generates about 1000 processes in 10 minutes)
mysql (i always have >150 instances of this)
postfix

it's a debian woody system (this is for checking the software versions).

in terms of syscall i can imagine an heavy use of the file (open, close, unlink), ipc, signal and process (fork, wait and so on) syscalls. If you want i can provide a more detailed report.

lj

Have you compared to a 2.4.20 kernel? Could you try a 2.4.20 kernel with grsecurity patched in and the main CONFIG_GRKERNSEC enabled, but no other options, and then compare this to the clean 2.4.20 kernel?

-Brad

I have experienced somethign similar to him in load increases. Of 6 servers 3 of them had the issue and they are all celeron processors. After booting with grsecurity patched kernel kjournald seems to freak out.


Again this has only happend to machines with celeron processors.

ameen wrote:I have experienced somethign similar to him in load increases. Of 6 servers 3 of them had the issue and they are all celeron processors. After booting with grsecurity patched kernel kjournald seems to freak out.

what part of PaX was enabled (if anything)?

This is the sysctl used on it, decided too use sysctl until we get a stable config.


# Grsecurity sysctl config
#
# Acl system
kernel.grsecurity.acl = 0

# Audit

# Logging
kernel.grsecurity.audit_mount = 1
kernel.grsecurity.forkfail_logging = 1
kernel.grsecurity.timechange_logging = 1

# Chroot restrictions
kernel.grsecurity.chroot_caps = 0
kernel.grsecurity.chroot_deny_chmod = 0
kernel.grsecurity.chroot_deny_chroot = 0
kernel.grsecurity.chroot_deny_mknod = 1
kernel.grsecurity.chroot_deny_pivot = 1
kernel.grsecurity.chroot_deny_shmat = 1
kernel.grsecurity.chroot_deny_sysctl = 1
kernel.grsecurity.chroot_deny_unix = 0
kernel.grsecurity.chroot_enforce_chdir = 1
kernel.grsecurity.chroot_execlog = 0
kernel.grsecurity.chroot_findtask = 0
kernel.grsecurity.chroot_restrict_nice = 1

# Misc settings
kernel.grsecurity.dmesg = 1
kernel.grsecurity.fifo_restrictions = 1
kernel.grsecurity.forkfail_logging = 1
kernel.grsecurity.linking_restrictions = 1
kernel.grsecurity.timechange_logging = 1
kernel.grsecurity.signal_logging = 1
kernel.grsecurity.execve_limiting = 0
kernel.grsecurity.exec_logging = 0

# Networking
kernel.grsecurity.altered_pings = 1
kernel.grsecurity.rand_pids = 1
kernel.grsecurity.rand_ip_ids = 1
kernel.grsecurity.rand_isns = 1
kernel.grsecurity.rand_rpc = 1
kernel.grsecurity.rand_tcp_src_ports = 1

# Grsec lock
# Reboot must be invoked inorder to make changes to grsecurity, after
# grsec_lock is set to a value greater than zero (0)
#
kernel.grsecurity.grsec_lock = 0

PaX cannot be turned on/off via sysctl. Can you please paste the relevant section of your kernel config that shows which PaX options were enabled?

-Brad

#
# Address Space Protection
#
# CONFIG_GRKERNSEC_PAX_NOEXEC is not set
CONFIG_GRKERNSEC_PAX_ASLR=y
CONFIG_GRKERNSEC_PAX_RANDKSTACK=y
CONFIG_GRKERNSEC_PAX_RANDUSTACK=y
CONFIG_GRKERNSEC_PAX_RANDMMAP=y
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_RTC=y
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_HIDESYM=y

ameen wrote:# CONFIG_GRKERNSEC_PAX_NOEXEC is not set

this would have been the only one i could suspect, but it's not enabled. it would be helpful if you guys could pinpoint which tasks (beyond kjournald) are consuming more CPU than usual and provide some strace outputs and per task CPU usage stats (to determine where the problem is). also, can you tell us what gcc version you used to compile the kernel?