grsecurity _really_ slow on sparc64

[jelly]

Hi, we've noticed a degradation of performance between
linux-2.4.28-grsec and linux-2.4.29-grsec on sparc64 architecture.

lmbench shows an order-of-magnitude slowdown for
integer operations, this run on 2.4.30 w/ and w/o grsecurity:

Code: Select all

Host                 OS  intgr intgr  intgr  intgr  intgr   
                          bit   add    mul    div    mod
--------- ------------- ------ ------ ------ ------ ------
vlatko     Linux 2.4.30 1.5500 1.5500   48.3   87.8   97.0
vlatko    Linux 2.4.30- 1.5500 1.5500  919.7  958.3  978.7


In real life, on an Ultra-5 (UltraSPARC IIi CPU @270MHz, 128MB RAM),
running the usual sparc64 kernel + sparc32 userspace (Debian
3.1 for sparc), this makes eg. an "apt-get update" run for 3-5
minutes with 2.4.30-grsecurity, vs. ~1 min with the plain 2.4.30
kernel.

The problem was introduced between linux-2.4.28 (grsec 2.1.0)
and linux-2.4.29 (grsec 2.1.1). The local kernel expert has narrowed
the visible slowdown down to these options:

Code: Select all

CONFIG_GRKERNSEC_PAX_MPROTECT=y
CONFIG_GRKERNSEC_PAX_EMUPLT=y
CONFIG_GRKERNSEC_PAX_DLRESOLVE=y


Turning them off makes the system work at decent speed again.

Here are the complete lmbench results for 2.4.30 with and without
Grsecurity (just ignore the comments in Croatian, this is from an
internal list):

http://kosjenka.srce.hr/~jelly/grsec-sparc.txt

Any ideas?

Regards,
Zoran
[/u]

[PaX Team]

Hi, we've noticed a degradation of performance between
linux-2.4.28-grsec and linux-2.4.29-grsec on sparc64 architecture.

can you get the numbers for 2.4.28 where you say the performance was better?

the thing is, all archs but i386, ia64 and x86_64 need PLT emulation because of some bad choices made in userland many moons ago when their ELF ABI was created. in practical terms, each time a library call is made, the kernel (under PaX/MPROTECT) will have to do some emulation and that's much more expensive than the one instruction call it'd take otherwise. i bet that if you linked lmbench binaries statically, you'd see better performance. alternatively, you can turn off MPROTECT but that opens up the exploit vector for runtime code generation (shellcode injection/execution).

now what makes your report interesting is that sparc64 support (including PLT emulation) was added to PaX around 2.4.20, so you should have been experiencing the performance impact ever since, not 2.4.29. i took a brief look at the .28/.29 diff and nothing really stands out that could impact sparc64, so i'll need more information: cat /proc/self/maps output under both kernels with randomization turned off.

[jelly]

i took a brief look at the .28/.29 diff and nothing really stands out that could impact sparc64, so i'll need more information: cat /proc/self/maps output under both kernels with randomization turned off.

2.4.28-grsec:

Code: Select all

00010000-00014000 r-xp 00000000 03:05 131316 /usr/local/bin/rcat
00022000-00024000 rwxp 00002000 03:05 131316 /usr/local/bin/rcat
00024000-00046000 rwxp 00000000 00:00 0
70000000-7001a000 r-xp 00000000 03:02 61442  /lib/ld-2.3.2.so
70028000-7002a000 rwxp 00018000 03:02 61442  /lib/ld-2.3.2.so
7002c000-70160000 r-xp 00000000 03:02 61563  /lib/libc-2.3.2.so
70160000-7016c000 ---p 00134000 03:02 61563  /lib/libc-2.3.2.so
7016c000-70178000 rwxp 00130000 03:02 61563  /lib/libc-2.3.2.so
70178000-7017a000 rwxp 00000000 00:00 0
efffe000-f0000000 rwxp 00000000 00:00 0


2.4.29-grsec:

Code: Select all

00010000-00014000 r-xp 00000000 03:05 131316 /usr/local/bin/rcat
00022000-00024000 rw-p 00002000 03:05 131316 /usr/local/bin/rcat
00024000-00046000 rw-p 00000000 00:00 0
70000000-7001a000 r-xp 00000000 03:02 61442  /lib/ld-2.3.2.so
7001a000-7001c000 r-xp 00000000 00:00 0
70028000-7002a000 rw-p 00018000 03:02 61442  /lib/ld-2.3.2.so
7002c000-70160000 r-xp 00000000 03:02 61563  /lib/libc-2.3.2.so
70160000-7016c000 ---p 00134000 03:02 61563  /lib/libc-2.3.2.so
7016c000-70178000 rw-p 00130000 03:02 61563  /lib/libc-2.3.2.so
70178000-7017a000 rw-p 00000000 00:00 0
efffc000-f0000000 rw-p ffffffffffffe000 00:00


I've also noticed some changes in the .configs of the kernels used:
diff -u config-2.4.28-grsec config-2.4.29-grsec|grep -v '^ '

Code: Select all

@@ -958,13 +952,13 @@
+CONFIG_GRKERNSEC_PAX_PAGEEXEC=y
-CONFIG_GRKERNSEC_PAX_RANDEXEC=y
@@ -1031,10 +1025,7 @@
-CONFIG_GRKERNSEC_RANDISN=y
-CONFIG_GRKERNSEC_RANDID=y
-CONFIG_GRKERNSEC_RANDRPC=y


[jelly]

can you get the numbers for 2.4.28 where you say the performance was better?

http://kosjenka.srce.hr/~jelly/28vs29.txt
These were run on a different (slower) machine, so they're not directly comparable to the previous results.

i bet that if you linked lmbench binaries statically, you'd see better performance.

There's one run with statically linked benchmark binaries which shows improvement even on 2.4.28-grsec. The 2.4.29-grsec tests are still running, I might update the results later.

now what makes your report interesting is that sparc64 support (including PLT emulation) was added to PaX around 2.4.20, so you should have been experiencing the performance impact ever since, not 2.4.29.

We have been using GrSecurity since 2.4.22. While I'm aware PaX/GrSecurity introduces some performance impact, there's a marked worsening between those two releases.

[PaX Team]

Code: Select all

+CONFIG_GRKERNSEC_PAX_PAGEEXEC=y

and that about explains it: you didn't even have nonexecutable pages enforced (see how you have rwx mappings on .28, such things don't exist under a fully utilized PaX), no wonder PLT emulation didn't kick in.