2.4.26-grsec2 Kernel oops

[rocky]

recently switched our shell server to grsec and have repeatedly, at random, recieved the following kernel oops. searched the forums and mailing list didn't see anything about this. is this a misconfiguration problem on our side? or something with grsec. any help and information is greatly appreciated.

-Rocky

kernel oops also available @ http://www.xmission.com/~rocky/deadshel ... 3.ksymoops

ksymoops 2.4.5 on i686 2.4.26-grsec. Options used
-V (default)
-k /proc/ksyms (default)
-l /proc/modules (default)
-o /lib/modules/2.4.26-grsec/ (default)
-m /boot/System.map-2.4.26-grsec2-2 (specified)

Error (regular_file): read_ksyms stat /proc/ksyms failed
No modules in ksyms, skipping objects
No ksyms, skipping lsmod
Jun 8 00:23:12 xmission.xmission.com kernel: Unable to handle kernel NULL pointer dereference at virtual address 00000008
Jun 8 00:23:12 xmission.xmission.com kernel: 001e0d0a
Jun 8 00:23:12 xmission.xmission.com kernel: *pde = 00000000
Jun 8 00:23:12 xmission.xmission.com kernel: Oops: 0000
Jun 8 00:23:12 xmission.xmission.com kernel: CPU: 1
Jun 8 00:23:12 xmission.xmission.com kernel: EIP: 0010:[<001e0d0a>] Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
Jun 8 00:23:12 xmission.xmission.com kernel: EFLAGS: 00010286
Jun 8 00:23:12 xmission.xmission.com kernel: eax: 00000000 ebx: bffff210 ecx: dd81d960 edx: d7516000
Jun 8 00:23:12 xmission.xmission.com kernel: esi: 0000002b edi: 00000013 ebp: d73d9dc4 esp: d73d9d4c
Jun 8 00:23:12 xmission.xmission.com kernel: ds: 0018 es: 0018 ss: 0018
Jun 8 00:23:12 xmission.xmission.com kernel: Process exim (pid: 20041, stackpage=d73d9000)
Jun 8 00:23:12 xmission.xmission.com kernel: Stack: dfffbf60 d751624a 00003301 00000008 00000008 00000008 00000008 00000000
Jun 8 00:23:12 xmission.xmission.com kernel: bffff2a8 d73d9e64 080bbba6 bffff2b4 00000003 2400000f 7273752f 6962732f
Jun 8 00:23:12 xmission.xmission.com kernel: 78652f6e 2d206d69 3120634d 30615842 30302d75 624f3330 2030302d 00000000
Jun 8 00:23:12 xmission.xmission.com kernel: Call Trace: [<00003301>] [<00000008>] [<00000008>] [<00000008>] [<00000008>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>] [<00000003>] [<00000000>] [<00000000>] [<00000000>] [<00000000>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>] [<00000000>] [<00000000>] [<0003b75e>] [<0003021c>] [<00006e38>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000303>] [<00034adf>] [<00000000>] [<0006710b>] [<0006710b>] [<00010101>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>] [<00000000>] [<00030002>] [<00000001>] [<00000034>] [<000734d0>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>] [<00200034>] [<00180019>] [<00000006>] [<00000034>] [<000000c0>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<000000c0>] [<00000005>] [<00000004>] [<00000003>] [<000000f4>] [<00000013>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000013>] [<00000004>] [<00000001>] [<00000001>] [<00000000>] [<00000000>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>] [<00000000>] [<00000000>] [<00000000>] [<00000000>] [<00000000>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<0001ffed>] [<00000000>] [<00000000>] [<00000008>] [<00000003>] [<0000000e>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>] [<0001ffed>] [<00000000>] [<00000c37>] [<000020b3>] [<0000000b>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<0000002b>] [<0000002b>] [<0000000b>] [<00000023>] [<00000246>] [<0000002b>]
Jun 8 00:23:12 xmission.xmission.com kernel: [<00000000>]
Jun 8 00:23:12 xmission.xmission.com kernel: Code: 8b 40 08 50 e8 b1 52 00 00 83 c4 08 eb 05 b8 20 c7 65 c0 ba


>>EIP; 001e0d0a <gr_handle_exec_args+1a6/37e> <=====

>>ebx; bffff210 <_etext+bfdf4dee/bfef5bfe>
>>ecx; dd81d960 <_end+1d01d960/3f599fc0>
>>edx; d7516000 <_end+16d16000/3f599fc0>
>>ebp; d73d9dc4 <_end+16bd9dc4/3f599fc0>
>>esp; d73d9d4c <_end+16bd9d4c/3f599fc0>

Trace; 00003301 <show_interrupts+11/1d8>
Trace; 00000008 Before first symbol
Trace; 00000008 Before first symbol
Trace; 00000008 Before first symbol
Trace; 00000008 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000003 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 0003b75e <do_execve+256/3b0>
Trace; 0003021c <shmem_unlink+14/34>
Trace; 00006e38 <IRQ0xd0_interrupt+8/10>
Trace; 00000303 Before first symbol
Trace; 00034adf <getblk+43/4c>
Trace; 00000000 Before first symbol
Trace; 0006710b <do_get_write_access+517/53c>
Trace; 0006710b <do_get_write_access+517/53c>
Trace; 00010101 <change_page_attr+51/d9>
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00030002 <shmem_statfs+2e/5c>
Trace; 00000001 Before first symbol
Trace; 00000034 Before first symbol
Trace; 000734d0 <ext2_remount+a0/124>
Trace; 00000000 Before first symbol
Trace; 00200034 <gr_acl_handle_chmod+53c/a60>
Trace; 00180019 <ip_route_output_slow+4e9/610>
Trace; 00000006 Before first symbol
Trace; 00000034 Before first symbol
Trace; 000000c0 Before first symbol
Trace; 000000c0 Before first symbol
Trace; 00000005 Before first symbol
Trace; 00000004 Before first symbol
Trace; 00000003 Before first symbol
Trace; 000000f4 Before first symbol
Trace; 00000013 Before first symbol
Trace; 00000013 Before first symbol
Trace; 00000004 Before first symbol
Trace; 00000001 Before first symbol
Trace; 00000001 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 0001ffed <lock_kiovec+a9/e8>
Trace; 00000000 Before first symbol
Trace; 00000000 Before first symbol
Trace; 00000008 Before first symbol
Trace; 00000003 Before first symbol
Trace; 0000000e Before first symbol
Trace; 00000000 Before first symbol
Trace; 0001ffed <lock_kiovec+a9/e8>
Trace; 00000000 Before first symbol
Trace; 00000c37 Before first symbol
Trace; 000020b3 <system_call+33/40>
Trace; 0000000b Before first symbol
Trace; 0000002b Before first symbol
Trace; 0000002b Before first symbol
Trace; 0000000b Before first symbol
Trace; 00000023 Before first symbol
Trace; 00000246 Before first symbol
Trace; 0000002b Before first symbol
Trace; 00000000 Before first symbol

Code; 001e0d0a <gr_handle_exec_args+1a6/37e>
00000000 <_EIP>:
Code; 001e0d0a <gr_handle_exec_args+1a6/37e> <=====
0: 8b 40 08 mov 0x8(%eax),%eax <=====
Code; 001e0d0d <gr_handle_exec_args+1a9/37e>
3: 50 push %eax
Code; 001e0d0e <gr_handle_exec_args+1aa/37e>
4: e8 b1 52 00 00 call 52ba <_EIP+0x52ba> 001e5fc4 <gr_to_filename3+0/110>
Code; 001e0d13 <gr_handle_exec_args+1af/37e>
9: 83 c4 08 add $0x8,%esp
Code; 001e0d16 <gr_handle_exec_args+1b2/37e>
c: eb 05 jmp 13 <_EIP+0x13> 001e0d1d <gr_handle_exec_args+1b9/37e>
Code; 001e0d18 <gr_handle_exec_args+1b4/37e>
e: b8 20 c7 65 c0 mov $0xc065c720,%eax
Code; 001e0d1d <gr_handle_exec_args+1b9/37e>
13: ba 00 00 00 00 mov $0x0,%edx


1 error issued. Results may not be reliable.

[PaX Team]

recently switched our shell server to grsec and have repeatedly, at random, recieved the following kernel oops. searched the forums and mailing list didn't see anything about this. is this a misconfiguration problem on our side? or something with grsec. any help and information is greatly appreciated.

weird bug, it happens in the gr_parent_task_fullpath() macro when accessing the f_dentry field... except that the exec_file pointer is already checked against NULL so eax couldn't be NULL at that point. could you post the full disassembly of the gr_handle_exec_args function please?

[rocky]

weird bug, it happens in the gr_parent_task_fullpath() macro when accessing the f_dentry field... except that the exec_file pointer is already checked against NULL so eax couldn't be NULL at that point. could you post the full disassembly of the gr_handle_exec_args function please?

pardon my ignorance as i am not the greatest root hacker around, but how exactly do i go about getting a “ full disassembly of the gr_handle_exec_args function”? Thanks for your help.

[PaX Team]

pardon my ignorance as i am not the greatest root hacker around, but how exactly do i go about getting a “ full disassembly of the gr_handle_exec_args function”? Thanks for your help.

sorry, was a bit terse . so, in the kernel directory where you compiled grsec issue objdump -d grsecurity/grsec_exec.o and from the output cut/paste the parts belonging to gr_handle_exec_args (will be a few hundred lines at most, feel free to email it instead, to pageexec at freemail.hu and dev at grsecurity.net).

[rocky]

sorry, was a bit terse . so, in the kernel directory where you compiled grsec issue objdump -d grsecurity/grsec_exec.o and from the output cut/paste the parts belonging to gr_handle_exec_args (will be a few hundred lines at most, feel free to email it instead, to pageexec at freemail.hu and dev at grsecurity.net).

oh no terseness at all. i only wanted to find out how to give you the information you needed. i emailed it to the two email addys you posted and a copy can also be found at http://www.xmission.com/~rocky/deadshel ... grsec_exec

[PaX Team]

i emailed it to the two email addys you posted and a copy can also be found at http://www.xmission.com/~rocky/deadshel ... grsec_exec

ok, we have an SMP race here, the parent task can exit and its exec_file member turned to NULL between the check against NULL and the actual dereference.

[rocky]

ok, we have an SMP race here, the parent task can exit and its exec_file member turned to NULL between the check against NULL and the actual dereference.

Cool, so this is something that is fixable?

Thanks again for looking into this, i owe ya a drink

[spender]

It was fixed today in CVS.

-Brad

[rocky]

It was fixed today in CVS.

-Brad

Thank you as well.