Random SEGV on address randomization

[bron]

Greetings,

There seems to be a problem with address randomization on which varius (as in random) commands / apps get signal 11 without any apparent reason. By enabling the following two options:

Code: Select all

CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y


i get - on a very rare occations - segv to different utilities.

from log:

Code: Select all

May 19 11:38:42 localhost kernel: grsec: signal 11 sent to /bin/ls[ls:7809] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/find[find:29551] uid/euid:0/0 gid/egid:0/0


on a normal basis i get 3-5 segv/week by looking at logs. it's not related to any specific app .. i get it from ls, gawk, zcat to gcc/firefox etc. i was able to reproduce it more often doing the following on a direcrory tree with 10's of thousant of files (about 90k) i run:

Code: Select all

find /usr/src -exec ls -l {} 1> /dev/null \;


i get about 2-3 segv to ls.

at first i thought it was related to bad hardware, mem etc. (becouse segv's where to rare to notice) but i tested it with two different machines (one p4/rimm and one athlon tbird/ddr mem). Also with the original distributions kernels (centos) and vanila kernels (same config as grsec) i get nothing.

at neither of them i used kernel stack randomization.

full conf

Code: Select all

CONFIG_GRKERNSEC=y
CONFIG_GRKERNSEC_CUSTOM=y

CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_HIDESYM=y

CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=5
CONFIG_GRKERNSEC_ACL_TIMEOUT=30

CONFIG_GRKERNSEC_PROC=y
CONFIG_GRKERNSEC_PROC_USER=y
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y

CONFIG_GRKERNSEC_RESLOG=y
CONFIG_GRKERNSEC_AUDIT_MOUNT=y
CONFIG_GRKERNSEC_AUDIT_IPC=y
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y

CONFIG_GRKERNSEC_EXECVE=y
CONFIG_GRKERNSEC_SHM=y
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_RANDPID=y
CONFIG_GRKERNSEC_TPE=y
CONFIG_GRKERNSEC_TPE_ALL=y
CONFIG_GRKERNSEC_TPE_GID=1005

CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_RANDSRC=y

CONFIG_GRKERNSEC_SYSCTL=y

CONFIG_GRKERNSEC_FLOODTIME=10
CONFIG_GRKERNSEC_FLOODBURST=4

CONFIG_PAX=y

CONFIG_PAX_EI_PAX=y
CONFIG_PAX_HAVE_ACL_FLAGS=y

CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
CONFIG_PAX_MPROTECT=y

CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y
CONFIG_PAX_NOVSYSCALL=y


grsecurity-2.1.5 on 2.6.11.7 and .9

[PaX Team]

Code: Select all

CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y


can you try to find out which of these causes the segfaults (by enabling only one of them at a time)?[quote]

Code: Select all

May 19 11:38:42 localhost kernel: grsec: signal 11 sent to /bin/ls[ls:7809] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/find[find:29551] uid/euid:0/0 gid/egid:0/0

can you enable coredumping and get/analyze the coredump?

on a sidenote, what's your glibc version (and distro)?

[bron]

i narrow it down to the combination of CONFIG_PAX_RANDMMAP and CONFIG_PAX_PAGEEXEC. if i change with chpax one of this options to false all works ok. with CONFIG_PAX_SEGMEXEC + RAND seems ok.

now .. i'm trying this on an i386 hardware ... so, i don't know if pageexec should always behave ..

i have an opteron based machine but not the time (and the means) at this time to check it ...

backtrace to core files gives ...
(recomiled 'ls' with debuging symbols )

Code: Select all

coredump# gdb ./ls core.17622
Using host libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `./ls /usr/local/src/xc/programs/Xserver/hw/xfree86/sdk/CVS/Tag'.
Program terminated with signal 11, Segmentation fault.
#0  0x08034911 in ?? ()
(gdb) bt
#0  0x08034911 in ?? ()
#1  0x080494f8 in ?? ()
#2  0x00000000 in ?? ()


always dies before it even start executing ..

glibc 2.3.4 centos4/redhat el4

cheers

[PaX Team]

now .. i'm trying this on an i386 hardware ... so, i don't know if pageexec should always behave ..

yes it should, however PAGEEXEC on 2.6 is special in that it has a speed-up logic and a corresponding change in the address space layout (executable mappings go 'down' if possible) that apparently causes issues with ld.so or whoever.

backtrace to core files gives ...
(recomiled 'ls' with debuging symbols )

Code: Select all

coredump# gdb ./ls core.17622
Using host libthread_db library "/lib/tls/libthread_db.so.1".

Core was generated by `./ls /usr/local/src/xc/programs/Xserver/hw/xfree86/sdk/CVS/Tag'.
Program terminated with signal 11, Segmentation fault.
#0  0x08034911 in ?? ()
(gdb) bt
#0  0x08034911 in ?? ()
#1  0x080494f8 in ?? ()
#2  0x00000000 in ?? ()


this looks like that you don't actually have debug info everywhere . in any case, can you dump some memory and register context as well please? e.g., for the above: x/16i $pc, x/16i 0x080494f8, x/16x $sp, info reg (adapt as necessary).

[bron]

ehm... i thought smth is wrong here

here you go with memory and registers ...
... i have limited experience with this debuging (i usualy use printf's ) and not much time to work with it ...

Code: Select all

Program terminated with signal 11, Segmentation fault.
#0  0x08034911 in ?? ()
(gdb) bt
#0  0x08034911 in ?? ()
#1  0x080494f8 in ?? ()
#2  0x00000000 in ?? ()
(gdb) x/16i $pc
0x8034911:      mov    (%edx),%eax
0x8034913:      mov    %eax,0x1f0(%ebx)
0x8034919:      movzbl 0x36c(%ebx),%ecx
0x8034920:      lea    0x2c(%ebx),%esi
0x8034926:      mov    %esi,%eax
0x8034928:      incl   0x30(%ebx)
0x803492e:      add    $0x1c0,%eax
0x8034933:      and    $0xfc,%cl
0x8034936:      or     $0x1,%cl
0x8034939:      mov    %cl,0x36c(%ebx)
0x803493f:      mov    %esi,0xfffffe08(%ebp)
0x8034945:      mov    0xffffff14(%ebp),%esi
0x803494b:      addl   $0x1,0x1c4(%ebx)
0x8034952:      mov    0xac(%esi),%edi
0x8034958:      adcl   $0x0,0x1c8(%ebx)
0x803495f:      mov    %eax,0xc(%esi)
(gdb) x/16i 0x08034911
0x8034911:      mov    (%edx),%eax
0x8034913:      mov    %eax,0x1f0(%ebx)
0x8034919:      movzbl 0x36c(%ebx),%ecx
0x8034920:      lea    0x2c(%ebx),%esi
0x8034926:      mov    %esi,%eax
0x8034928:      incl   0x30(%ebx)
0x803492e:      add    $0x1c0,%eax
0x8034933:      and    $0xfc,%cl
0x8034936:      or     $0x1,%cl
0x8034939:      mov    %cl,0x36c(%ebx)
0x803493f:      mov    %esi,0xfffffe08(%ebp)
0x8034945:      mov    0xffffff14(%ebp),%esi
0x803494b:      addl   $0x1,0x1c4(%ebx)
0x8034952:      mov    0xac(%esi),%edi
0x8034958:      adcl   $0x0,0x1c8(%ebx)
0x803495f:      mov    %eax,0xc(%esi)
(gdb) x/16i 0x080494f8
0x80494f8:      add    %al,(%eax)
0x80494fa:      add    %al,(%eax)
0x80494fc:      adc    %ebx,0x4(%edi)
0x80494ff:      or     %al,(%eax)
0x8049501:      add    %al,(%eax)
0x8049503:      add    %al,(%eax)
0x8049505:      add    %al,(%eax)
0x8049507:      add    %al,(%eax)
0x8049509:      add    %al,(%eax)
0x804950b:      add    %bh,%al
0x804950d:      xchg   %eax,%esp
0x804950e:      add    $0x8,%al
0x8049510:      add    %al,(%eax)
0x8049512:      add    %al,(%eax)
0x8049514:      cmp    %dl,0x804(%edi)
0x804951a:      add    %al,(%eax)
(gdb) x/16x $sp
0xbb73da54:     0x080494f8      0x00000000      0x00000000      0x00000000
0xbb73da64:     0x00000000      0x00000000      0x00000000      0x00000000
0xbb73da74:     0x00000000      0x00000000      0x00000000      0x00000000
0xbb73da84:     0x00000000      0x00000000      0x00000000      0x00000000
(gdb) info reg
eax            0xffffffff       -1
ecx            0x0      0
edx            0x0      0
ebx            0x8048fd4        134516692
esp            0xbb73da54       0xbb73da54
ebp            0xbb73dd20       0xbb73dd20
esi            0x80494f8        134518008
edi            0x0      0
eip            0x8034911        0x8034911
eflags         0x10246  66118
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x0      0
(gdb)                                       


.. i could send you the core if it's easier.

[PaX Team]

.. i could send you the core if it's easier.

could you first change the PAX_DELTA_MMAP_LEN macro in include/asm-i386/elf.h to 14 please and try again?