applicaton crashed at _start()
Posted: Tue Mar 01, 2011 2:46 am
i'm using the WindRiver2-SP4(2.6.21), the grsecurity patch is grsecurity-2.1.10-2.6.21.5-200706182032.patch, Intel Core2 L7400.
we found that during the system start-up, sometimes(not always, approximately 8%), one application will receive the SIGSEGV signal, the message looks like:
-----------------------------
Feb 5 23:22:06 localhost kernel: grsec: signal 11 sent to /sbin/modprobe[modprobe:2074] uid/euid:0/0 gid/egid:0/0, parent /lib/udev/modprobe[modprobe:2072] uid/euid:0/0 gid/egid:0/0
-----------------------------
the applications were random, i saw /lib/udev/scsi_id, /sbin/modprobe, /usr/sbin/crond, arping, xined, udevd, /sbin/pam_console_apply at least.(so i don't think it's the application's problem.)
after check the core dump, it show the EIP always point to the _start, one example as below:
-------------------------------
gdb /sbin/udevtrigger -c core.udevtrigger
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5)
Reading symbols from /sbin/udevtrigger...(no debugging symbols found)...done.
warning: exec file is newer than core file.
Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.5.so.debug...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Core was generated by `/sbin/udevtrigger'.
Program terminated with signal 11, Segmentation fault.
#0 0x03780810 in _start () from /lib/ld-linux.so.2
(gdb) info reg
eax 0x0 0
ecx 0x0 0
edx 0x0 0
ebx 0x0 0
esp 0xbe8bbab0 0xbe8bbab0
ebp 0x0 0x0
esi 0x0 0
edi 0x0 0
eip 0x3780810 0x3780810 <_start>
eflags 0x212 [ AF IF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x0 0
-------------------------------
i dump the kernel stack before the SIGSEGV was sent to the application, it always look like below:
-------------------------------
Feb 7 02:55:35 localhost kernel: [<c0153aba>] show_trace_log_lvl+0x1a/0x30
Feb 7 02:55:35 localhost kernel: [<c0154352>] show_trace+0x12/0x20
Feb 7 02:55:35 localhost kernel: [<c01544a6>] dump_stack+0x16/0x20
Feb 7 02:55:35 localhost kernel: [<c018077c>] force_sig_info+0xbc/0xd0
Feb 7 02:55:35 localhost kernel: [<c01541fe>] do_trap+0x5e/0x160
Feb 7 02:55:35 localhost kernel: [<c0155219>] do_iret_error+0x99/0xb0
Feb 7 02:55:35 localhost kernel: [<c047f9c3>] error_code+0x73/0x80
-------------------------------
i don't know why there is iret error, after changing the kernel configuration, we found that disabling the CONFIG_PAX_PAGEEXEC will make the kernel/application work well. I thought that may be the NX protection problem, but i don't find the log like ' PAX: execution attempt it'.
could you give me some hint/help on this problem? anything else we need for the analysis further?
thanks
the configuration segment as below:
CONFIG_PAX=y
CONFIG_PAX_SOFTMODE=y
CONFIG_PAX_EI_PAX=y
CONFIG_PAX_PT_PAX_FLAGS=y
# CONFIG_PAX_NO_ACL_FLAGS is not set
# CONFIG_PAX_HAVE_ACL_FLAGS is not set
CONFIG_PAX_HOOK_ACL_FLAGS=y
CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
# CONFIG_PAX_SEGMEXEC is not set
CONFIG_PAX_EMUTRAMP=y
# CONFIG_PAX_MPROTECT is not set
CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDKSTACK=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y
CONFIG_PAX_MEMORY_SANITIZE=y
CONFIG_PAX_MEMORY_UDEREF=y
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
# CONFIG_SECURITY_NETWORK_XFRM is not set
CONFIG_SECURITY_CAPABILITIES=m
# CONFIG_SECURITY_ROOTPLUG is not set
CONFIG_SECURITY_BSDJAIL=m
we found that during the system start-up, sometimes(not always, approximately 8%), one application will receive the SIGSEGV signal, the message looks like:
-----------------------------
Feb 5 23:22:06 localhost kernel: grsec: signal 11 sent to /sbin/modprobe[modprobe:2074] uid/euid:0/0 gid/egid:0/0, parent /lib/udev/modprobe[modprobe:2072] uid/euid:0/0 gid/egid:0/0
-----------------------------
the applications were random, i saw /lib/udev/scsi_id, /sbin/modprobe, /usr/sbin/crond, arping, xined, udevd, /sbin/pam_console_apply at least.(so i don't think it's the application's problem.)
after check the core dump, it show the EIP always point to the _start, one example as below:
-------------------------------
gdb /sbin/udevtrigger -c core.udevtrigger
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-23.el5)
Reading symbols from /sbin/udevtrigger...(no debugging symbols found)...done.
warning: exec file is newer than core file.
Reading symbols from /lib/ld-linux.so.2...Reading symbols from /usr/lib/debug/lib/ld-2.5.so.debug...(no debugging symbols found)...done.
(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Core was generated by `/sbin/udevtrigger'.
Program terminated with signal 11, Segmentation fault.
#0 0x03780810 in _start () from /lib/ld-linux.so.2
(gdb) info reg
eax 0x0 0
ecx 0x0 0
edx 0x0 0
ebx 0x0 0
esp 0xbe8bbab0 0xbe8bbab0
ebp 0x0 0x0
esi 0x0 0
edi 0x0 0
eip 0x3780810 0x3780810 <_start>
eflags 0x212 [ AF IF ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x0 0
-------------------------------
i dump the kernel stack before the SIGSEGV was sent to the application, it always look like below:
-------------------------------
Feb 7 02:55:35 localhost kernel: [<c0153aba>] show_trace_log_lvl+0x1a/0x30
Feb 7 02:55:35 localhost kernel: [<c0154352>] show_trace+0x12/0x20
Feb 7 02:55:35 localhost kernel: [<c01544a6>] dump_stack+0x16/0x20
Feb 7 02:55:35 localhost kernel: [<c018077c>] force_sig_info+0xbc/0xd0
Feb 7 02:55:35 localhost kernel: [<c01541fe>] do_trap+0x5e/0x160
Feb 7 02:55:35 localhost kernel: [<c0155219>] do_iret_error+0x99/0xb0
Feb 7 02:55:35 localhost kernel: [<c047f9c3>] error_code+0x73/0x80
-------------------------------
i don't know why there is iret error, after changing the kernel configuration, we found that disabling the CONFIG_PAX_PAGEEXEC will make the kernel/application work well. I thought that may be the NX protection problem, but i don't find the log like ' PAX: execution attempt it'.
could you give me some hint/help on this problem? anything else we need for the analysis further?
thanks
the configuration segment as below:
CONFIG_PAX=y
CONFIG_PAX_SOFTMODE=y
CONFIG_PAX_EI_PAX=y
CONFIG_PAX_PT_PAX_FLAGS=y
# CONFIG_PAX_NO_ACL_FLAGS is not set
# CONFIG_PAX_HAVE_ACL_FLAGS is not set
CONFIG_PAX_HOOK_ACL_FLAGS=y
CONFIG_PAX_NOEXEC=y
CONFIG_PAX_PAGEEXEC=y
# CONFIG_PAX_SEGMEXEC is not set
CONFIG_PAX_EMUTRAMP=y
# CONFIG_PAX_MPROTECT is not set
CONFIG_PAX_ASLR=y
CONFIG_PAX_RANDKSTACK=y
CONFIG_PAX_RANDUSTACK=y
CONFIG_PAX_RANDMMAP=y
CONFIG_PAX_MEMORY_SANITIZE=y
CONFIG_PAX_MEMORY_UDEREF=y
# CONFIG_KEYS is not set
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
# CONFIG_SECURITY_NETWORK_XFRM is not set
CONFIG_SECURITY_CAPABILITIES=m
# CONFIG_SECURITY_ROOTPLUG is not set
CONFIG_SECURITY_BSDJAIL=m