grsecurity forums

with 2.6.36.1-grsec i have no problem but the 2.6.36.2-grsec not work
btw the its on vmware here is pic with the error :
http://img87.imageshack.us/img87/2738/o ... el2010.png

my config : http://pastebin.ca/2020927

I'm seeing something similar with grsecurity-2.2.1-2.6.36.2-201012192125.patch. At first I thought it was missing the initrd, but it was there and then I thought I'd glance at the forums and saw this thread. This was going from 2.6.36-grsec (which boots fine) to 2.6.36.2-grsec, Dell PE 1950 running Debian Lenny 32-bit.

<snip>
[ 17.849380] rtc_cmos 00:04: setting system clock to 2010-12-22 13:44:05 UTC (1293025445)
[ 17.850641] sda: sda1 sda2 sda3 sda4
[ 17.920380] sd 0:0:1:0: [sdb] Write Protect is off
[ 17.920409] Initalizing network drop monitor service
[ 17.979989] sd 0:0:0:0: [sda] Attached SCSI disk
[ 17.980829] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
[ 18.076508] sdb: sdb1
[ 18.117013] sd 0:0:1:0: [sdb] Attached SCSI disk
[ 18.144878] md: Waiting for all devices to be available before autodetect
[ 18.185724] md: If you don't use raid, use raid=noautodetect
[ 18.220050] md: Autodetecting RAID arrays.
[ 18.244770] md: Scanned 0 and added 0 devices.
[ 18.271568] md: autorun ...
[ 18.288483] md: ... autorun DONE.
[ 18.318512] EXT3-fs: barriers not enabled
[ 18.342927] kjournald starting. Commit interval 5 seconds
[ 18.342952] EXT3-fs (sda3): mounted filesystem with writeback data mode
[ 18.342968] VFS: Mounted root (rootfs filesystem) on device 0:1.
[ 18.452551] Freeing unused kernel memory: 372k freed
[ 18.482679] Kernel panic - not syncing: No init found. Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.
[ 18.557544] Pid: 1, comm: swapper Not tainted 2.6.36.2-nx #1
[ 18.591625] Call Trace:
[ 18.606469] [<00444089>] ? panic+0x4d/0x13c
[ 18.632228] [<0000220e>] ? init_post+0x104/0x106
[ 18.660589] [<01a053f1>] ? 0x1a053f1
[ 18.682707] [<01a05256>] ? 0x1a05256
[ 18.704827] [<00003fe6>] ? kernel_thread_helper+0x6/0x12
</snip>

For the heck of it, I rebuilt the initrd and got this on reboot:

<snip>
[ 18.116015] VFS: Cannot open root device "sda3" or unknown-block(8,3)
[ 18.154781] Please append a correct "root=" boot option; here are the available partitions:
[ 18.480306] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,3)
[ 18.530202] Pid: 1, comm: swapper Not tainted 2.6.36.2-nx #1
[ 18.564282] Call Trace:
[ 18.579122] [<00444089>] ? panic+0x4d/0x13c
[ 18.604884] [<01a05c5e>] ? mount_block_root+0x1e9/0x1f8
[ 18.636886] [<00008001>] ? save_sched_clock_state+0x8/0x1a
[ 18.670448] [<00002933>] ? sched_groups+0xf/0x10
[ 18.698807] [<01a05ce8>] ? mount_root+0x7b/0x8f
[ 18.726646] [<01a05e3e>] ? prepare_namespace+0x142/0x172
[ 18.759171] [<01a053e7>] ? kernel_init+0x191/0x1a1
[ 18.788574] [<01a05256>] ? kernel_init+0x0/0x1a1
[ 18.816935] [<00003fe6>] ? kernel_thread_helper+0x6/0x12
</snip>

The new patch (grsecurity-2.2.1-2.6.36.2-201012221906.patch) works just fine. Thanks!

Er, perhaps I spoke too soon. It boots up just fine. I can't actually log into the box though. Gets most of the way through the login process via ssh or serial (i.e. I get the usual banner but no bash prompt). Rebooting into an older kernel, I grabbed this from the kernel logs. There were a bunch of these. No grsec errors to speak of though in the logs.

kernel: [ 245.260044] INFO: task exim:5041 blocked for more than 120 seconds.
kernel: [ 245.260048] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
kernel: [ 245.260052] exim D 00004140 0 5041 5034 0x00000000
kernel: [ 245.260058] f6f99200 00000086 f656206c 00004140 00004140 f6f993a8 c5244140 00000001
kernel: [ 245.260066] 13bed638 00000009 0038b426 f6f993ac f6f993a8 00004c36 c5244140 f6f99200
kernel: [ 245.260074] c5244588 c51250c8 004449c3 00000000 f59cdea0 f59cdea8 0008b108 00444cee
kernel: [ 245.260081] Call Trace:
kernel: [ 245.260092] [<00004140>] ? runqueues+0x0/0x15c
kernel: [ 245.260096] [<00004140>] ? runqueues+0x0/0x15c
kernel: [ 245.260102] [<0038b426>] ? __skb_to_sgvec+0x80/0x1b8
kernel: [ 245.260107] [<00004c36>] ? show_interrupts+0x1e2/0x6e8
kernel: [ 245.260113] [<004449c3>] ? io_schedule+0x68/0xab
kernel: [ 245.260119] [<0008b108>] ? sync_page+0x3c/0x3f
kernel: [ 245.260123] [<00444cee>] ? __wait_on_bit+0x33/0x58
kernel: [ 245.260127] [<0008b0cc>] ? sync_page+0x0/0x3f
kernel: [ 245.260131] [<0008b2ea>] ? wait_on_page_bit+0x80/0x89
kernel: [ 245.260136] [<0004621f>] ? wake_bit_function+0x0/0x3c
kernel: [ 245.260141] [<0008ba56>] ? filemap_fdatawait_range+0x67/0xfd
kernel: [ 245.260145] [<0009073c>] ? do_writepages+0x19/0x25
kernel: [ 245.260150] [<0008bb7d>] ? __filemap_fdatawrite_range+0x54/0x60
kernel: [ 245.260155] [<0008bbc7>] ? filemap_write_and_wait_range+0x3e/0x4c
kernel: [ 245.260159] [<000d7f49>] ? vfs_fsync_range+0x24/0x5a
kernel: [ 245.260163] [<000d7fed>] ? vfs_fsync+0x11/0x15
kernel: [ 245.260166] [<000d800d>] ? do_fsync+0x1c/0x2b
kernel: [ 245.260171] [<00445cad>] ? syscall_call+0x7/0xb

A bit of a side-track, but did you or anyone else change any software?

The exim-version in debian testing and stable should have been fixed a year ago, but debian stable still did contain a remote exploitable bug with root escalation.
http://lists.exim.org/lurker/message/20 ... d0.en.html
http://www.reddit.com/r/netsec/comments ... led_on_my/
Some people found a version of dropbear preventing sshd from starting...

I don't know if grsecurity prevents this exploit, but that's because I never saw an advantage from using exim. Since it looks like a overflow exploit grsecurity might prevent some damage depending on your configuration.

moseleymark wrote:Er, perhaps I spoke too soon. It boots up just fine. I can't actually log into the box though. Gets most of the way through the login process via ssh or serial (i.e. I get the usual banner but no bash prompt).

can you do this experiment please: in arch/x86/kernel/traps.c:do_general_protection() just after the gp_in_kernel label comment out the if/return lines. this way we'll see any otherwise silent UDEREF violation as an oops in your logs. due to recent changes in i386/UDEREF, the kernel is very picky now about the proper choice of KERNEL_DS/USER_DS and it's probably the problem here as well.

Here's the results after commenting out those two lines. This happens a couple of seconds after /etc/rc.local runs.

[ 48.682476] PAX: suspicious general protection fault: 0000 [#1] SMP
[ 48.684004] last sysfs file: /sys/devices/pci0000:00/0000:00:1e.0/0000:0f:0d.0/local_cpus
[ 48.684004] Modules linked in: dm_snapshot dm_mirror dm_region_hash dm_log dm_mod reiserfs joydev intel_agp i2c_i801 i2c_core agpgart evdev i5000_eda]
[ 48.684004]
[ 48.684004] Pid: 10, comm: kworker/0:1 Not tainted 2.6.36.2-nx #1 0DT097/PowerEdge 1950
[ 48.684004] EIP: 0060:[<001f29da>] EFLAGS: 00010246 CPU: 0
[ 48.684004] EIP is at csum_partial_copy_generic+0x11a/0x140
[ 48.684004] EAX: 00000000 EBX: 001f29bf ECX: ffffffff EDX: 00000002
[ 48.684004] ESI: f5d0309e EDI: f6088000 EBP: f5d0309e ESP: f7495c08
[ 48.684004] DS: 007b ES: 0068 FS: 00d8 GS: 0068 SS: 0068
[ 48.684004] Process kworker/0:1 (pid: 10, ti=f7494000 task=f7452880 task.ti=f7494000)
[ 48.684004] Stack:
[ 48.684004] 00000002 00000000 f5d03000 0038e976 f5d0309e f6088000 00000002 00000000
[ 48.684004] <0> f7495c48 00000000 0038e718 f6088000 00000000 f5d0309f f5d0309e f7495ec4
[ 48.684004] <0> 00000000 f7495ebc f6088000 f3c0bc00 00000002 003c3372 00000002 f7495c68
[ 48.684004] Call Trace:
[ 48.684004] [<0038e976>] ? csum_partial_copy_fromiovecend+0x192/0x1e7
[ 48.684004] [<0038e718>] ? copy_from_user+0x78/0xa9
[ 48.684004] [<003c3372>] ? ip_generic_getfrag+0x43/0x8a
[ 48.684004] [<003c328c>] ? ip_append_data+0x83f/0x8e2
[ 48.684004] [<003c332f>] ? ip_generic_getfrag+0x0/0x8a
[ 48.684004] [<00020000>] ? pgd_free+0x53/0x85
[ 48.684004] [<00029a2c>] ? get_parent_ip+0x8/0x19
[ 48.684004] [<000353a3>] ? __local_bh_disable+0x73/0x76
[ 48.684004] [<003de60a>] ? udp_sendmsg+0x4ab/0x5a2
[ 48.684004] [<00004040>] ? cpu_info+0x0/0x12
[ 48.684004] [<00840108>] ? 0x840108
[ 48.684004] [<01084103>] ? 0x1084103
[ 48.684004] [<00285e00>] ? scsi_alloc_queue+0x2b/0x45
[ 48.684004] [<003e3dc0>] ? inet_sendmsg+0x6b/0x75
[ 48.684004] [<00386ad6>] ? sock_sendmsg+0xaf/0xc9
[ 48.684004] [<0009c827>] ? set_page_address+0xb7/0x153
[ 48.684004] [<000aa000>] ? get_vm_area_caller+0x0/0x42
[ 48.684004] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 48.684004] [<0009c946>] ? page_address+0x83/0x9e
[ 48.684004] [<0009c975>] ? kunmap_high+0x14/0x84
[ 48.684004] [<003c2980>] ? ip_append_page+0x30e/0x3db
[ 48.684004] [<00029a2c>] ? get_parent_ip+0x8/0x19
[ 48.684004] [<000353a3>] ? __local_bh_disable+0x73/0x76
[ 48.684004] [<0000c040>] ? show_shared_cpu_map_func+0x47/0x4b
[ 48.684004] [<00445a15>] ? _raw_spin_lock_bh+0x8/0x1e
[ 48.684004] [<00386b23>] ? kernel_sendmsg+0x33/0x43
[ 48.684004] [<00004040>] ? cpu_info+0x0/0x12
[ 48.684004] [<004111e3>] ? xs_send_kvec+0x7f/0x87
[ 48.684004] [<00004040>] ? cpu_info+0x0/0x12
[ 48.684004] [<000044fe>] ? do_invalid_op+0x3a/0x70
[ 48.684004] [<00411351>] ? xs_sendpages+0x166/0x183
[ 48.684004] [<00004466>] ? do_invalid_TSS+0x38/0x4b
[ 48.684004] [<00004000>] ? irq_stat+0x0/0x32
[ 48.684004] [<0041154a>] ? xs_udp_send_request+0x37/0xcd
[ 48.684004] [<0040fbb1>] ? xprt_transmit+0x157/0x234
[ 48.684004] [<0040d9fd>] ? call_transmit+0x1a5/0x1df
[ 48.684004] [<004131ee>] ? __rpc_execute+0x68/0x190
[ 48.684004] [<00042262>] ? process_one_work+0x17a/0x25c
[ 48.684004] [<0041334a>] ? rpc_async_schedule+0x0/0x8
[ 48.684004] [<000425a4>] ? worker_thread+0x100/0x1fa
[ 48.684004] [<00004640>] ? gcwq_nr_running+0x0/0x2f
[ 48.684004] [<00004640>] ? gcwq_nr_running+0x0/0x2f
[ 48.684004] [<000424a4>] ? worker_thread+0x0/0x1fa
[ 48.684004] [<00045e7b>] ? kthread+0x65/0x6a
[ 48.684004] [<00045e16>] ? kthread+0x0/0x6a
[ 48.684004] [<00003fa6>] ? kernel_thread_helper+0x6/0x12
[ 48.684004] Code: 26 89 5f f8 90 90 90 8b 5e fc 11 d8 26 89 5f fc 83 d0 00 83 c2 40 49 0f 8d 28 ff ff ff 8b 54 24 18 83 e2 03 74 21 83 fa 02 72 12 <6
[ 48.684004] EIP: [<001f29da>] csum_partial_copy_generic+0x11a/0x140 SS:ESP 0068:f7495c08
[ 50.879501] ---[ end trace fb5e4720d8d3d2ab ]---
[ 50.907741] PAX: suspicious general protection fault: 0000 [#2] SMP
[ 50.911345] last sysfs file: /sys/devices/pci0000:00/0000:00:1e.0/0000:0f:0d.0/local_cpus
[ 50.911345] Modules linked in: dm_snapshot dm_mirror dm_region_hash dm_log dm_mod reiserfs joydev intel_agp i2c_i801 i2c_core agpgart evdev i5000_eda]
[ 50.911345]
[ 50.911345] Pid: 10, comm: kworker/0:1 Tainted: G D 2.6.36.2-nx #1 0DT097/PowerEdge 1950
[ 50.911345] EIP: 0060:[<0001cf41>] EFLAGS: 00010046 CPU: 0
[ 50.911345] EIP is at is_prefetch+0x94/0x1b6
[ 50.911345] EAX: 00000068 EBX: 00000000 ECX: f74957b8 EDX: ffffffff
[ 50.911345] ESI: ffffffff EDI: f74957b8 EBP: 00045b7e ESP: f74956c8
[ 50.911345] DS: 0068 ES: 0068 FS: 00d8 GS: 0068 SS: 0068
[ 50.911345] Process kworker/0:1 (pid: 10, ti=f7494000 task=f7452880 task.ti=f7494000)
[ 50.911345] Stack:
[ 50.911345] f74957b8 00045b8d 02e40900 00000014 f84051b4 000568d9 f74957b8 f7452880
[ 50.911345] <0> fffffffc f74957b8 00000000 0001d08f 00000012 00000000 f74957b8 00000060
[ 50.911345] <0> 00000000 fffffffc 0001d427 f7452880 00000000 127943e7 2f52ab11 884e2cce
[ 50.911345] Call Trace:
[ 50.911345] [<00045b8d>] ? flush_kthread_work+0x3/0x99
[ 50.911345] [<000568d9>] ? search_module_extables+0x37/0x58
[ 50.911345] [<0001d08f>] ? no_context+0x2c/0x1dc
[ 50.911345] [<0001d427>] ? __bad_area_nosemaphore+0x1e8/0x1f3
[ 50.911345] [<0001c000>] ? hpet_set_mode+0xca/0x194
[ 50.911345] [<00019000>] ? msi_compose_msg+0xc/0xb8
[ 50.911345] [<0002750d>] ? select_task_rq_fair+0x33d/0x644
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<0003b4c3>] ? lock_timer_base+0x19/0x35
[ 50.911345] [<00448b78>] ? do_page_fault+0x6f/0x546
[ 50.911345] [<0003b5eb>] ? mod_timer_pending+0xc/0xe
[ 50.911345] [<00448b09>] ? do_page_fault+0x0/0x546
[ 50.911345] [<0001d43c>] ? bad_area_nosemaphore+0xa/0xc
[ 50.911345] [<00030001>] ? default_handler+0x19/0x52
[ 50.911345] [<0044676c>] ? error_code+0x9c/0xb0
[ 50.911345] [<001a00d8>] ? cifs_stats_proc_show+0x17d/0x285
[ 50.911345] [<00045b7e>] ? kthread_data+0x6/0xa
[ 50.911345] [<00010002>] ? mce_suspend+0x2a/0x37
[ 50.911345] [<000438c1>] ? wq_worker_sleeping+0x9/0x77
[ 50.911345] [<00444546>] ? schedule+0x100/0x515
[ 50.911345] [<00004140>] ? runqueues+0x0/0x15c
[ 50.911345] [<00004140>] ? runqueues+0x0/0x15c
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<00046371>] ? prepare_to_wait+0x14/0x54
[ 50.911345] [<0014b402>] ? do_get_write_access+0x1f6/0x3aa
[ 50.911345] [<0003b5b8>] ? __mod_timer+0xd9/0xe2
[ 50.911345] [<0004621f>] ? wake_bit_function+0x0/0x3c
[ 50.911345] [<0014b5ce>] ? journal_get_write_access+0x18/0x26
[ 50.911345] [<001168ef>] ? __ext3_journal_get_write_access+0x13/0x32
[ 50.911345] [<0010b2c2>] ? ext3_reserve_inode_write+0x2d/0x59
[ 50.911345] [<0010b2ff>] ? ext3_mark_inode_dirty+0x11/0x27
[ 50.911345] [<0010b414>] ? ext3_dirty_inode+0x50/0x63
[ 50.911345] [<000d4cac>] ? __mark_inode_dirty+0x20/0x175
[ 50.911345] [<000cc405>] ? file_update_time+0xbd/0xde
[ 50.911345] [<0008beb0>] ? __generic_file_aio_write+0x143/0x28b
[ 50.911345] [<00023018>] ? check_preempt_curr+0x15/0x2a
[ 50.911345] [<0002a9a6>] ? ttwu_post_activation+0x4b/0x110
[ 50.911345] [<0002b699>] ? try_to_wake_up+0x136/0x140
[ 50.911345] [<0008c049>] ? generic_file_aio_write+0x51/0x93
[ 50.911345] [<000bba79>] ? do_sync_write+0xa0/0xe4
[ 50.911345] [<001de856>] ? blkiocg_update_completion_stats+0x43/0x105
[ 50.911345] [<0008c8e4>] ? mempool_free+0x61/0x64
[ 50.911345] [<001d1a48>] ? elv_queue_empty+0x1d/0x1e
[ 50.911345] [<001d645f>] ? __blk_run_queue+0x19/0x5b
[ 50.911345] [<0034f540>] ? ohci_init+0xb6/0x228
[ 50.911345] [<0004dbf2>] ? ktime_get_ts+0xd3/0xdb
[ 50.911345] [<0034f540>] ? ohci_init+0xb6/0x228
[ 50.911345] [<0005ab5f>] ? do_acct_process+0x60b/0x633
[ 50.911345] [<00285e3f>] ? scsi_next_command+0x25/0x2f
[ 50.911345] [<007a13f4>] ? 0x7a13f4
[ 50.911345] [<001d9807>] ? blk_done_softirq+0x53/0x5f
[ 50.911345] [<00035565>] ? __do_softirq+0x138/0x16b
[ 50.911345] [<00002a0c>] ? kstat+0x0/0x22
[ 50.911345] [<0003541e>] ? _local_bh_enable+0x73/0x82
[ 50.911345] [<001f33e8>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 50.911345] [<001deefc>] ? cfq_queue_empty+0x0/0x14
[ 50.911345] [<00313a30>] ? tg3_start_xmit_dma_bug+0x591/0x91b
[ 50.911345] [<0005abdb>] ? acct_process+0x54/0x66
[ 50.911345] [<00033465>] ? do_exit+0x20c/0x64a
[ 50.911345] [<00445cda>] ? restore_all_notrace+0x0/0x18
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<00033902>] ? do_group_exit+0x5f/0x82
[ 50.911345] [<0044727e>] ? oops_end+0x8f/0x93
[ 50.911345] [<001f29bf>] ? csum_partial_copy_generic+0xff/0x140
[ 50.911345] [<00446f9b>] ? do_general_protection+0x0/0x18c
[ 50.911345] [<0044676c>] ? error_code+0x9c/0xb0
[ 50.911345] [<001f29bf>] ? csum_partial_copy_generic+0xff/0x140
[ 50.911345] [<001f29da>] ? csum_partial_copy_generic+0x11a/0x140
[ 50.911345] [<00010246>] ? ftrace_define_fields_mce_record+0xcc/0x207
[ 50.911345] [<0038e976>] ? csum_partial_copy_fromiovecend+0x192/0x1e7
[ 50.911345] [<0038e718>] ? copy_from_user+0x78/0xa9
[ 50.911345] [<003c3372>] ? ip_generic_getfrag+0x43/0x8a
[ 50.911345] [<003c328c>] ? ip_append_data+0x83f/0x8e2
[ 50.911345] [<003c332f>] ? ip_generic_getfrag+0x0/0x8a
[ 50.911345] [<00020000>] ? pgd_free+0x53/0x85
[ 50.911345] [<00029a2c>] ? get_parent_ip+0x8/0x19
[ 50.911345] [<000353a3>] ? __local_bh_disable+0x73/0x76
[ 50.911345] [<003de60a>] ? udp_sendmsg+0x4ab/0x5a2
[ 50.911345] [<00004040>] ? cpu_info+0x0/0x12
[ 50.911345] [<00840108>] ? 0x840108
[ 50.911345] [<01084103>] ? 0x1084103
[ 50.911345] [<00285e00>] ? scsi_alloc_queue+0x2b/0x45
[ 50.911345] [<003e3dc0>] ? inet_sendmsg+0x6b/0x75
[ 50.911345] [<00386ad6>] ? sock_sendmsg+0xaf/0xc9
[ 50.911345] [<0009c827>] ? set_page_address+0xb7/0x153
[ 50.911345] [<000aa000>] ? get_vm_area_caller+0x0/0x42
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<0009c946>] ? page_address+0x83/0x9e
[ 50.911345] [<0009c975>] ? kunmap_high+0x14/0x84
[ 50.911345] [<003c2980>] ? ip_append_page+0x30e/0x3db
[ 50.911345] [<00029a2c>] ? get_parent_ip+0x8/0x19
[ 50.911345] [<000353a3>] ? __local_bh_disable+0x73/0x76
[ 50.911345] [<0000c040>] ? show_shared_cpu_map_func+0x47/0x4b
[ 50.911345] [<00445a15>] ? _raw_spin_lock_bh+0x8/0x1e
[ 50.911345] [<00386b23>] ? kernel_sendmsg+0x33/0x43
[ 50.911345] [<00004040>] ? cpu_info+0x0/0x12
[ 50.911345] [<004111e3>] ? xs_send_kvec+0x7f/0x87
[ 50.911345] [<00004040>] ? cpu_info+0x0/0x12
[ 50.911345] [<000044fe>] ? do_invalid_op+0x3a/0x70
[ 50.911345] [<00411351>] ? xs_sendpages+0x166/0x183
[ 50.911345] [<00004466>] ? do_invalid_TSS+0x38/0x4b
[ 50.911345] [<00004000>] ? irq_stat+0x0/0x32
[ 50.911345] [<0041154a>] ? xs_udp_send_request+0x37/0xcd
[ 50.911345] [<0040fbb1>] ? xprt_transmit+0x157/0x234
[ 50.911345] [<0040d9fd>] ? call_transmit+0x1a5/0x1df
[ 50.911345] [<004131ee>] ? __rpc_execute+0x68/0x190
[ 50.911345] [<00042262>] ? process_one_work+0x17a/0x25c
[ 50.911345] [<0041334a>] ? rpc_async_schedule+0x0/0x8
[ 50.911345] [<000425a4>] ? worker_thread+0x100/0x1fa
[ 50.911345] [<00004640>] ? gcwq_nr_running+0x0/0x2f
[ 50.911345] [<00004640>] ? gcwq_nr_running+0x0/0x2f
[ 50.911345] [<000424a4>] ? worker_thread+0x0/0x1fa
[ 50.911345] [<00045e7b>] ? kthread+0x65/0x6a
[ 50.911345] [<00045e16>] ? kthread+0x0/0x6a
[ 50.911345] [<00003fa6>] ? kernel_thread_helper+0x6/0x12
[ 50.911345] Code: 03 09 c2 83 fa 02 76 0c 31 c0 65 8a 55 00 89 d7 85 c0 eb 30 89 e0 25 00 e0 ff ff 8b 70 18 ff 40 14 83 c8 ff 31 db e8 62 6a 1d 00 <6
[ 50.911345] EIP: [<0001cf41>] is_prefetch+0x94/0x1b6 SS:ESP 0068:f74956c8
[ 50.911345] ---[ end trace fb5e4720d8d3d2ac ]---
[ 50.911345] Fixing recursive fault but reboot is needed!
[ 50.911345] BUG: scheduling while atomic: kworker/0:1/10/0x00000001
[ 50.911345] Modules linked in: dm_snapshot dm_mirror dm_region_hash dm_log dm_mod reiserfs joydev intel_agp i2c_i801 i2c_core agpgart evdev i5000_eda]
[ 50.911345] Pid: 10, comm: kworker/0:1 Tainted: G D 2.6.36.2-nx #1
[ 50.911345] Call Trace:
[ 50.911345] [<004444c1>] ? schedule+0x7b/0x515
[ 50.911345] [<00004140>] ? runqueues+0x0/0x15c
[ 50.911345] [<00004140>] ? runqueues+0x0/0x15c
[ 50.911345] [<00033313>] ? do_exit+0xba/0x64a
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<00033902>] ? do_group_exit+0x5f/0x82
[ 50.911345] [<00045b7e>] ? kthread_data+0x6/0xa
[ 50.911345] [<0044727e>] ? oops_end+0x8f/0x93
[ 50.911345] [<00446f9b>] ? do_general_protection+0x0/0x18c
[ 50.911345] [<0044676c>] ? error_code+0x9c/0xb0
[ 50.911345] [<00045b7e>] ? kthread_data+0x6/0xa
[ 50.911345] [<003b0068>] ? nfulnl_log_packet+0x484/0x77c
[ 50.911345] [<0001cf41>] ? is_prefetch+0x94/0x1b6
[ 50.911345] [<00010046>] ? fake_panic_get+0x0/0x12
[ 50.911345] [<00045b8d>] ? flush_kthread_work+0x3/0x99
[ 50.911345] [<000568d9>] ? search_module_extables+0x37/0x58
[ 50.911345] [<0001d08f>] ? no_context+0x2c/0x1dc
[ 50.911345] [<0001d427>] ? __bad_area_nosemaphore+0x1e8/0x1f3
[ 50.911345] [<0001c000>] ? hpet_set_mode+0xca/0x194
[ 50.911345] [<00019000>] ? msi_compose_msg+0xc/0xb8
[ 50.911345] [<0002750d>] ? select_task_rq_fair+0x33d/0x644
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<0003b4c3>] ? lock_timer_base+0x19/0x35
[ 50.911345] [<00448b78>] ? do_page_fault+0x6f/0x546
[ 50.911345] [<0003b5eb>] ? mod_timer_pending+0xc/0xe
[ 50.911345] [<00448b09>] ? do_page_fault+0x0/0x546
[ 50.911345] [<0001d43c>] ? bad_area_nosemaphore+0xa/0xc
[ 50.911345] [<00030001>] ? default_handler+0x19/0x52
[ 50.911345] [<0044676c>] ? error_code+0x9c/0xb0
[ 50.911345] [<001a00d8>] ? cifs_stats_proc_show+0x17d/0x285
[ 50.911345] [<00045b7e>] ? kthread_data+0x6/0xa
[ 50.911345] [<00010002>] ? mce_suspend+0x2a/0x37
[ 50.911345] [<000438c1>] ? wq_worker_sleeping+0x9/0x77
[ 50.911345] [<00444546>] ? schedule+0x100/0x515
[ 50.911345] [<00004140>] ? runqueues+0x0/0x15c
[ 50.911345] [<00004140>] ? runqueues+0x0/0x15c
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<00046371>] ? prepare_to_wait+0x14/0x54
[ 50.911345] [<0014b402>] ? do_get_write_access+0x1f6/0x3aa
[ 50.911345] [<0003b5b8>] ? __mod_timer+0xd9/0xe2
[ 50.911345] [<0004621f>] ? wake_bit_function+0x0/0x3c
[ 50.911345] [<0014b5ce>] ? journal_get_write_access+0x18/0x26
[ 50.911345] [<001168ef>] ? __ext3_journal_get_write_access+0x13/0x32
[ 50.911345] [<0010b2c2>] ? ext3_reserve_inode_write+0x2d/0x59
[ 50.911345] [<0010b2ff>] ? ext3_mark_inode_dirty+0x11/0x27
[ 50.911345] [<0010b414>] ? ext3_dirty_inode+0x50/0x63
[ 50.911345] [<000d4cac>] ? __mark_inode_dirty+0x20/0x175
[ 50.911345] [<000cc405>] ? file_update_time+0xbd/0xde
[ 50.911345] [<0008beb0>] ? __generic_file_aio_write+0x143/0x28b
[ 50.911345] [<00023018>] ? check_preempt_curr+0x15/0x2a
[ 50.911345] [<0002a9a6>] ? ttwu_post_activation+0x4b/0x110
[ 50.911345] [<0002b699>] ? try_to_wake_up+0x136/0x140
[ 50.911345] [<0008c049>] ? generic_file_aio_write+0x51/0x93
[ 50.911345] [<000bba79>] ? do_sync_write+0xa0/0xe4
[ 50.911345] [<001de856>] ? blkiocg_update_completion_stats+0x43/0x105
[ 50.911345] [<0008c8e4>] ? mempool_free+0x61/0x64
[ 50.911345] [<001d1a48>] ? elv_queue_empty+0x1d/0x1e
[ 50.911345] [<001d645f>] ? __blk_run_queue+0x19/0x5b
[ 50.911345] [<0034f540>] ? ohci_init+0xb6/0x228
[ 50.911345] [<0004dbf2>] ? ktime_get_ts+0xd3/0xdb
[ 50.911345] [<0034f540>] ? ohci_init+0xb6/0x228
[ 50.911345] [<0005ab5f>] ? do_acct_process+0x60b/0x633
[ 50.911345] [<00285e3f>] ? scsi_next_command+0x25/0x2f
[ 50.911345] [<007a13f4>] ? 0x7a13f4
[ 50.911345] [<001d9807>] ? blk_done_softirq+0x53/0x5f
[ 50.911345] [<00035565>] ? __do_softirq+0x138/0x16b
[ 50.911345] [<00002a0c>] ? kstat+0x0/0x22
[ 50.911345] [<0003541e>] ? _local_bh_enable+0x73/0x82
[ 50.911345] [<001f33e8>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 50.911345] [<001deefc>] ? cfq_queue_empty+0x0/0x14
[ 50.911345] [<00313a30>] ? tg3_start_xmit_dma_bug+0x591/0x91b
[ 50.911345] [<0005abdb>] ? acct_process+0x54/0x66
[ 50.911345] [<00033465>] ? do_exit+0x20c/0x64a
[ 50.911345] [<00445cda>] ? restore_all_notrace+0x0/0x18
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<00033902>] ? do_group_exit+0x5f/0x82
[ 50.911345] [<0044727e>] ? oops_end+0x8f/0x93
[ 50.911345] [<001f29bf>] ? csum_partial_copy_generic+0xff/0x140
[ 50.911345] [<00446f9b>] ? do_general_protection+0x0/0x18c
[ 50.911345] [<0044676c>] ? error_code+0x9c/0xb0
[ 50.911345] [<001f29bf>] ? csum_partial_copy_generic+0xff/0x140
[ 50.911345] [<001f29da>] ? csum_partial_copy_generic+0x11a/0x140
[ 50.911345] [<00010246>] ? ftrace_define_fields_mce_record+0xcc/0x207
[ 50.911345] [<0038e976>] ? csum_partial_copy_fromiovecend+0x192/0x1e7
[ 50.911345] [<0038e718>] ? copy_from_user+0x78/0xa9
[ 50.911345] [<003c3372>] ? ip_generic_getfrag+0x43/0x8a
[ 50.911345] [<003c328c>] ? ip_append_data+0x83f/0x8e2
[ 50.911345] [<003c332f>] ? ip_generic_getfrag+0x0/0x8a
[ 50.911345] [<00020000>] ? pgd_free+0x53/0x85
[ 50.911345] [<00029a2c>] ? get_parent_ip+0x8/0x19
[ 50.911345] [<000353a3>] ? __local_bh_disable+0x73/0x76
[ 50.911345] [<003de60a>] ? udp_sendmsg+0x4ab/0x5a2
[ 50.911345] [<00004040>] ? cpu_info+0x0/0x12
[ 50.911345] [<00840108>] ? 0x840108
[ 50.911345] [<01084103>] ? 0x1084103
[ 50.911345] [<00285e00>] ? scsi_alloc_queue+0x2b/0x45
[ 50.911345] [<003e3dc0>] ? inet_sendmsg+0x6b/0x75
[ 50.911345] [<00386ad6>] ? sock_sendmsg+0xaf/0xc9
[ 50.911345] [<0009c827>] ? set_page_address+0xb7/0x153
[ 50.911345] [<000aa000>] ? get_vm_area_caller+0x0/0x42
[ 50.911345] [<00445968>] ? _raw_spin_lock_irqsave+0xc/0x25
[ 50.911345] [<0009c946>] ? page_address+0x83/0x9e
[ 50.911345] [<0009c975>] ? kunmap_high+0x14/0x84
[ 50.911345] [<003c2980>] ? ip_append_page+0x30e/0x3db
[ 50.911345] [<00029a2c>] ? get_parent_ip+0x8/0x19
[ 50.911345] [<000353a3>] ? __local_bh_disable+0x73/0x76
[ 50.911345] [<0000c040>] ? show_shared_cpu_map_func+0x47/0x4b
[ 50.911345] [<00445a15>] ? _raw_spin_lock_bh+0x8/0x1e
[ 50.911345] [<00386b23>] ? kernel_sendmsg+0x33/0x43
[ 50.911345] [<00004040>] ? cpu_info+0x0/0x12
[ 50.911345] [<004111e3>] ? xs_send_kvec+0x7f/0x87
[ 50.911345] [<00004040>] ? cpu_info+0x0/0x12
[ 50.911345] [<000044fe>] ? do_invalid_op+0x3a/0x70
[ 50.911345] [<00411351>] ? xs_sendpages+0x166/0x183
[ 50.911345] [<00004466>] ? do_invalid_TSS+0x38/0x4b
[ 50.911345] [<00004000>] ? irq_stat+0x0/0x32
[ 50.911345] [<0041154a>] ? xs_udp_send_request+0x37/0xcd
[ 50.911345] [<0040fbb1>] ? xprt_transmit+0x157/0x234
[ 50.911345] [<0040d9fd>] ? call_transmit+0x1a5/0x1df
[ 50.911345] [<004131ee>] ? __rpc_execute+0x68/0x190
[ 50.911345] [<00042262>] ? process_one_work+0x17a/0x25c
[ 50.911345] [<0041334a>] ? rpc_async_schedule+0x0/0x8
[ 50.911345] [<000425a4>] ? worker_thread+0x100/0x1fa
[ 50.911345] [<00004640>] ? gcwq_nr_running+0x0/0x2f
[ 50.911345] [<00004640>] ? gcwq_nr_running+0x0/0x2f
[ 50.911345] [<000424a4>] ? worker_thread+0x0/0x1fa
[ 50.911345] [<00045e7b>] ? kthread+0x65/0x6a
[ 50.911345] [<00045e16>] ? kthread+0x0/0x6a
[ 50.911345] [<00003fa6>] ? kernel_thread_helper+0x6/0x12

moseleymark wrote:Here's the results after commenting out those two lines. This happens a couple of seconds after /etc/rc.local runs.

great! can you send the corresponding vmlinux (not bzImage) file to me please?

On its way. This is just the vmlinux right at the root of the kernel's directory left-over from debian make-kpkg, so if it's the wrong format, let me know and I can re-roll.

moseleymark wrote:On its way. This is just the vmlinux right at the root of the kernel's directory left-over from debian make-kpkg, so if it's the wrong format, let me know and I can re-roll.

it's all fine, i fixed two bugs, we'll see what else comes up next .

New patch compiles and boots just fine. I've got a bunch of machines up with it for about an hour now without a peep in the kernel logs. I mention the compile part because I saw in another thread that an OP's compile bailed in the same section that you fixed. If it helps, mine was compiled on gcc 4.3.2 (i.e. stock Lenny gcc).