Grsec on 2.6.31.4/.5 and KVM Guest Kernel Oops

[moseleymark]

I was wondering if there's a trick to booting a KVM guest with a grsec-enabled kernel. I can boot into a non-grsec-patched 2.6.31.4 and 2.6.31.5 on a KVM guest just fine, but booting with a grsec-enabled kernel 2.6.31.4 on a KVM guest just hangs and for a grsec-enabled kernel, I get a nice long Oops almost immediately (pasted below). After the Oops, nothing else happens, regardless of how long you wait.

This is on a Dell Poweredge 1950, running Debian Lenny in 32-bit mode. The host kernel is the same one as I'm using on the guest (grsec-patched 2.6.31.5). The ACL system is not enabled at the moment and no grsec errors pop up on the host when I try to boot the guest. This is my first stab at a grsec-enabled KVM guest, so it's not a 'it used to work till...' sort of thing, but rather a "I'm probably not doing something right" thing :)

Here's a diff of the configs (I can post the full ones if it'd help). You can see that it's basically identical except for the grsec/pax stuff.

# diff config-2.6.31.5-vscg config-2.6.31.5-vsc
4c4
< # Fri Oct 30 18:25:53 2009
---
> # Fri Oct 30 18:43:49 2009
65c65
< CONFIG_LOCALVERSION="-vscg"
---
> CONFIG_LOCALVERSION="-vsc"
190d189
< CONFIG_SLABINFO=y
279a279
> CONFIG_X86_ALIGNMENT_16=y
364d363
< # CONFIG_EFI is not set
379d377
< # CONFIG_COMPAT_VDSO is not set
2302d2299
< # CONFIG_PROC_KCORE is not set
2304d2300
< CONFIG_PROC_PAGE_MONITOR=y
2477d2472
< # CONFIG_LATENCYTOP is not set
2497d2491
< # CONFIG_FUNCTION_TRACER is not set
2507d2500
< # CONFIG_STACK_TRACER is not set
2530,2531d2522
< CONFIG_DEBUG_RODATA=y
< CONFIG_DEBUG_RODATA_TEST=y
2552a2544,2680
>
> #
> # Grsecurity
> #
> CONFIG_GRKERNSEC=y
> # CONFIG_GRKERNSEC_LOW is not set
> # CONFIG_GRKERNSEC_MEDIUM is not set
> # CONFIG_GRKERNSEC_HIGH is not set
> CONFIG_GRKERNSEC_CUSTOM=y
>
> #
> # Address Space Protection
> #
> CONFIG_GRKERNSEC_KMEM=y
> CONFIG_GRKERNSEC_IO=y
> CONFIG_GRKERNSEC_PROC_MEMMAP=y
> CONFIG_GRKERNSEC_BRUTE=y
> CONFIG_GRKERNSEC_MODHARDEN=y
> CONFIG_GRKERNSEC_HIDESYM=y
>
> #
> # Role Based Access Control Options
> #
> # CONFIG_GRKERNSEC_NO_RBAC is not set
> CONFIG_GRKERNSEC_ACL_HIDEKERN=y
> CONFIG_GRKERNSEC_ACL_MAXTRIES=3
> CONFIG_GRKERNSEC_ACL_TIMEOUT=30
>
> #
> # Filesystem Protections
> #
> CONFIG_GRKERNSEC_PROC=y
> CONFIG_GRKERNSEC_PROC_USER=y
> CONFIG_GRKERNSEC_PROC_ADD=y
> CONFIG_GRKERNSEC_LINK=y
> CONFIG_GRKERNSEC_FIFO=y
> CONFIG_GRKERNSEC_CHROOT=y
> CONFIG_GRKERNSEC_CHROOT_MOUNT=y
> CONFIG_GRKERNSEC_CHROOT_DOUBLE=y
> CONFIG_GRKERNSEC_CHROOT_PIVOT=y
> CONFIG_GRKERNSEC_CHROOT_CHDIR=y
> CONFIG_GRKERNSEC_CHROOT_CHMOD=y
> CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
> CONFIG_GRKERNSEC_CHROOT_MKNOD=y
> CONFIG_GRKERNSEC_CHROOT_SHMAT=y
> CONFIG_GRKERNSEC_CHROOT_UNIX=y
> CONFIG_GRKERNSEC_CHROOT_FINDTASK=y
> CONFIG_GRKERNSEC_CHROOT_NICE=y
> CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
> CONFIG_GRKERNSEC_CHROOT_CAPS=y
>
> #
> # Kernel Auditing
> #
> # CONFIG_GRKERNSEC_AUDIT_GROUP is not set
> CONFIG_GRKERNSEC_EXECLOG=y
> CONFIG_GRKERNSEC_RESLOG=y
> CONFIG_GRKERNSEC_CHROOT_EXECLOG=y
> CONFIG_GRKERNSEC_AUDIT_CHDIR=y
> CONFIG_GRKERNSEC_AUDIT_MOUNT=y
> CONFIG_GRKERNSEC_SIGNAL=y
> CONFIG_GRKERNSEC_FORKFAIL=y
> CONFIG_GRKERNSEC_TIME=y
> CONFIG_GRKERNSEC_PROC_IPADDR=y
> # CONFIG_GRKERNSEC_AUDIT_TEXTREL is not set
>
> #
> # Executable Protections
> #
> CONFIG_GRKERNSEC_EXECVE=y
> CONFIG_GRKERNSEC_DMESG=y
> CONFIG_GRKERNSEC_HARDEN_PTRACE=y
> # CONFIG_GRKERNSEC_TPE is not set
>
> #
> # Network Protections
> #
> CONFIG_GRKERNSEC_RANDNET=y
> CONFIG_GRKERNSEC_BLACKHOLE=y
> CONFIG_GRKERNSEC_SOCKET=y
> # CONFIG_GRKERNSEC_SOCKET_ALL is not set
> # CONFIG_GRKERNSEC_SOCKET_CLIENT is not set
> # CONFIG_GRKERNSEC_SOCKET_SERVER is not set
>
> #
> # Sysctl support
> #
> CONFIG_GRKERNSEC_SYSCTL=y
> # CONFIG_GRKERNSEC_SYSCTL_ON is not set
>
> #
> # Logging Options
> #
> CONFIG_GRKERNSEC_FLOODTIME=1
> CONFIG_GRKERNSEC_FLOODBURST=100
>
> #
> # PaX
> #
> CONFIG_PAX=y
>
> #
> # PaX Control
> #
> CONFIG_PAX_SOFTMODE=y
> # CONFIG_PAX_EI_PAX is not set
> CONFIG_PAX_PT_PAX_FLAGS=y
> # CONFIG_PAX_NO_ACL_FLAGS is not set
> CONFIG_PAX_HAVE_ACL_FLAGS=y
> # CONFIG_PAX_HOOK_ACL_FLAGS is not set
>
> #
> # Non-executable pages
> #
> CONFIG_PAX_NOEXEC=y
> CONFIG_PAX_PAGEEXEC=y
> CONFIG_PAX_SEGMEXEC=y
> CONFIG_PAX_EMUTRAMP=y
> CONFIG_PAX_MPROTECT=y
> # CONFIG_PAX_NOELFRELOCS is not set
> CONFIG_PAX_KERNEXEC=y
>
> #
> # Address Space Layout Randomization
> #
> CONFIG_PAX_ASLR=y
> CONFIG_PAX_RANDKSTACK=y
> CONFIG_PAX_RANDUSTACK=y
> CONFIG_PAX_RANDMMAP=y
>
> #
> # Miscellaneous hardening features
> #
> CONFIG_PAX_MEMORY_SANITIZE=y
> CONFIG_PAX_MEMORY_UDEREF=y
> CONFIG_PAX_REFCOUNT=y
> CONFIG_PAX_USERCOPY=y



Here's the Oops (btw, the Oops below is after several kernel permutations, all yielding the same thing; the one below is with me trying with KERNEXEC off, for the heck of it):

root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /boot/vmlinuz-2.6.31.5-vsc root=/dev/vda1 ro console=tty0 console=ttyS0
,19200n8 quiet
[Linux-bzImage, setup=0x3400, size=0x3392d0]
initrd /boot/initrd.img-2.6.31.5-vsc
[Linux-initrd @ 0x37bbf000, 0x43074f bytes]

Loading, please wait...
[ 2.361457] BUG: unable to handle kernel paging request at fffa7000
[ 2.363994] IP: [<c1086c6b>]
[ 2.365294] *pdpt = 00000000015bb001 *pde = 0000000001586067 *pte = 0000000000000000
[ 2.368528] Oops: 0002 [#1] SMP
[ 2.370023] last sysfs file:
[ 2.370023] Modules linked in:
[ 2.370023]
[ 2.370023] Pid: 1194, comm: init Not tainted (2.6.31.5-vsc #1)
[ 2.370023] EIP: 0060:[<c1086c6b>] EFLAGS: 00010046 CPU: 0
[ 2.370023] EAX: 00000000 EBX: 00000246 ECX: 00000400 EDX: 00000163
[ 2.370023] ESI: c1f37a00 EDI: fffa7000 EBP: 80100008 ESP: f6cf5c88
[ 2.370023] DS: 0068 ES: 0068 FS: 00d8 GS: 0000 SS: 0068
[ 2.370023] Process init (pid: 1194, ti=f6cf4000 task=f6dfda00 task.ti=f6cf4000)
[ 2.370023] Stack:
[ 2.370023] fffa7000 00000206 00000000 00000000 c108a147 3ffd0067 80000000 fffb3520
[ 2.370023] <0> c1f37a00 c1095d1c 00000000 00000001 00000080 3f27a067 00000000 c1f41068
[ 2.370023] <0> 00000000 f6c8c600 f6cf5d54 00000000 00000000 003eefb4 00000000 00000001
[ 2.370023] Call Trace:
[ 2.370023] [<c108a147>] ?
[ 2.370023] [<c1095d1c>] ?
[ 2.370023] [<c109b0f9>] ?
[ 2.370023] [<c102c438>] ?
[ 2.370023] [<c10b5e12>] ?
[ 2.370023] [<c10b497b>] ?
[ 2.370023] [<c10debcf>] ?
[ 2.370023] [<c1093142>] ?
[ 2.370023] [<c10b4485>] ?
[ 2.370023] [<c10b4571>] ?
[ 2.370023] [<c10b5522>] ?
[ 2.370023] [<c1001d15>] ?
[ 2.370023] [<c10034e5>] ?
[ 2.370023] [<c11f6e90>] ?
[ 2.370023] [<c1003518>] ?
[ 2.370023] Code: 58 8d 74 26 00 89 c3 fa 90 8d 74 26 00 e8 76 d8 fe ff ba 13 00 00 00 89 f0 e8 86 8a f9 ff b9 00 04 00 00 89 04 24 31 c0 8b 3c 24 <f3> ab 8b 04 24 ba 13 00 00 00 e8 4e 88 f9 ff f6 c7 02 75 0f 89
[ 2.370023] EIP: [<c1086c6b>] SS:ESP 0068:f6cf5c88
[ 2.370023] CR2: 00000000fffa7000
[ 2.370023] ---[ end trace c5bcbc099d052769 ]---
[ 2.370023] note: init[1194] exited with preempt_count 2
[ 2.452423] ------------[ cut here ]------------
[ 2.454590] kernel BUG at arch/x86/mm/highmem_32.c:45!
[ 2.456995] invalid opcode: 0000 [#2] SMP
[ 2.459305] last sysfs file:
[ 2.460919] Modules linked in:
[ 2.462365]
[ 2.462365] Pid: 1194, comm: init Tainted: G D (2.6.31.5-vsc #1)
[ 2.462365] EIP: 0060:[<c101f63e>] EFLAGS: 00010286 CPU: 0
[ 2.462365] EAX: 3f27a163 EBX: 80000000 ECX: 00000163 EDX: 80000000
[ 2.462365] ESI: bf27a163 EDI: 3f27d067 EBP: c1f1cfa0 ESP: f6cf5ab0
[ 2.462365] DS: 0068 ES: 0068 FS: 00d8 GS: 0000 SS: 0068
[ 2.462365] Process init (pid: 1194, ti=f6cf4000 task=f6dfda00 task.ti=f6cf4000)
[ 2.462365] Stack:
[ 2.462365] 00000000 f747ff9c c1f42884 c1021309 00000000 fffff000 00000038 c1f4288c
[ 2.462365] <0> 00000007 c1738000 00000007 3f27d067 c1f1cfac c101f6f5 00000163 80000000
[ 2.462365] <0> c1095b26 c1e0cc60 00000246 c1e11080 3f27d067 00000000 c1066df2 00000000
[ 2.462365] Call Trace:
[ 2.462365] [<c1021309>] ?
[ 2.462365] [<c101f6f5>] ?
[ 2.462365] [<c1095b26>] ?
[ 2.462365] [<c1066df2>] ?
[ 2.462365] [<c1032dc2>] ?
[ 2.462365] [<c109b0f9>] ?
[ 2.462365] [<c102c438>] ?
[ 2.462365] [<c102fc29>] ?
[ 2.462365] [<c103107d>] ?
[ 2.462365] [<c103150a>] ?
[ 2.462365] [<c13e4e51>] ?
[ 2.462365] [<c102efcf>] ?
[ 2.462365] [<c101a1ff>] ?
[ 2.462365] [<c14f962f>] ?
[ 2.462365] [<c13e5c85>] ?
[ 2.462365] [<c101a4a7>] ?
[ 2.462365] [<c13e444e>] ?
[ 2.462365] [<c13e5c85>] ?
[ 2.462365] [<c1086c6b>] ?
[ 2.462365] [<c108a147>] ?
[ 2.462365] [<c1095d1c>] ?
[ 2.462365] [<c109b0f9>] ?
[ 2.462365] [<c102c438>] ?
[ 2.462365] [<c10b5e12>] ?
[ 2.462365] [<c10b497b>] ?
[ 2.462365] [<c10debcf>] ?
[ 2.462365] [<c1093142>] ?
[ 2.462365] [<c10b4485>] ?
[ 2.462365] [<c10b4571>] ?
[ 2.462365] [<c10b5522>] ?
[ 2.462365] [<c1001d15>] ?
[ 2.462365] [<c10034e5>] ?
[ 2.462365] [<c11f6e90>] ?
[ 2.462365] [<c1003518>] ?
[ 2.462365] Code: 6b c0 14 01 c2 a1 84 a4 5e c1 89 54 24 20 c1 e2 03 89 54 24 18 89 44 24 14 a1 0c 72 69 c1 29 d0 8b 50 04 8b 00 89 d6 09 c6 74 04 <0f> 0b eb fe a1 80 f1 6d c1 89 ce 89 df 89 44 24 1c 89 c8 83 e0
[ 2.462365] EIP: [<c101f63e>] SS:ESP 0068:f6cf5ab0
[ 2.576096] ---[ end trace c5bcbc099d05276a ]---
[ 2.578269] Fixing recursive fault but reboot is needed!
[ 2.580679] BUG: scheduling while atomic: init/1194/0x00000003
[ 2.583257] Modules linked in:
[ 2.585038] Pid: 1194, comm: init Tainted: G D 2.6.31.5-vsc #1
[ 2.587870] Call Trace:
[ 2.589344] [<c13e2634>] ?
[ 2.590945] [<c102eb05>] ?
[ 2.592555] [<c1030020>] ?
[ 2.594114] [<c1030f9a>] ?
[ 2.595727] [<c14fbb3a>] ?
[ 2.597324] [<c103150a>] ?
[ 2.598933] [<c13e4e51>] ?
[ 2.600601] [<c13e46d6>] ?
[ 2.602162] [<c14f1a4b>] ?
[ 2.603777] [<c10045c8>] ?
[ 2.605334] [<c1004639>] ?
[ 2.606977] [<c101f63e>] ?
[ 2.608534] [<c102240d>] ?
[ 2.610171] [<c13e444a>] ?
[ 2.611731] [<c10045c8>] ?
[ 2.613344] [<c11f6ea0>] ?
[ 2.614899] [<c13e444e>] ?
[ 2.616540] [<c10045c8>] ?
[ 2.618161] [<c101f63e>] ?
[ 2.619725] [<c1021309>] ?
[ 2.621378] [<c101f6f5>] ?
[ 2.622939] [<c1095b26>] ?
[ 2.624551] [<c1066df2>] ?
[ 2.626136] [<c1032dc2>] ?
[ 2.627750] [<c109b0f9>] ?
[ 2.629356] [<c102c438>] ?
[ 2.630957] [<c102fc29>] ?
[ 2.632568] [<c103107d>] ?
[ 2.634128] [<c103150a>] ?
[ 2.635734] [<c13e4e51>] ?
[ 2.637361] [<c102efcf>] ?
[ 2.638974] [<c101a1ff>] ?
[ 2.640628] [<c14f962f>] ?
[ 2.642190] [<c13e5c85>] ?
[ 2.643798] [<c101a4a7>] ?
[ 2.645357] [<c13e444e>] ?
[ 2.647000] [<c13e5c85>] ?
[ 2.648561] [<c1086c6b>] ?
[ 2.650210] [<c108a147>] ?
[ 2.651775] [<c1095d1c>] ?
[ 2.653389] [<c109b0f9>] ?
[ 2.654995] [<c102c438>] ?
[ 2.656594] [<c10b5e12>] ?
[ 2.658215] [<c10b497b>] ?
[ 2.659778] [<c10debcf>] ?
[ 2.661433] [<c1093142>] ?
[ 2.662998] [<c10b4485>] ?
[ 2.664611] [<c10b4571>] ?
[ 2.666253] [<c10b5522>] ?
[ 2.667824] [<c1001d15>] ?
[ 2.669444] [<c10034e5>] ?
[ 2.671055] [<c11f6e90>] ?
[ 2.672672] [<c1003518>] ?
[ 2.676505] ------------[ cut here ]------------
[ 2.678663] kernel BUG at arch/x86/mm/highmem_32.c:45!
[ 2.681002] invalid opcode: 0000 [#3] SMP
[ 2.681042] last sysfs file:
[ 2.681042] Modules linked in:
[ 2.681042]
[ 2.681042] Pid: 1, comm: init Tainted: G D (2.6.31.5-vsc #1)
[ 2.681042] EIP: 0060:[<c101f63e>] EFLAGS: 00010286 CPU: 0
[ 2.681042] EAX: 3f27a163 EBX: 80000000 ECX: 00000163 EDX: 80000000
[ 2.681042] ESI: bf27a163 EDI: c13e5c85 EBP: c1f1cf00 ESP: f7449db8
[ 2.681042] DS: 0068 ES: 0068 FS: 00d8 GS: 0033 SS: 0068
[ 2.681042] Process init (pid: 1, ti=f7448000 task=f7440000 task.ti=f7448000)
[ 2.681042] Stack:
[ 2.681042] c1586d98 00000000 c1018cf8 c1586d98 00000001 fffff000 00000038 00000000
[ 2.681042] <0> 00000007 c1738000 3f278067 c13e5c85 00000007 c101f6f5 00000163 80000000
[ 2.681042] <0> c1096abc c0000000 c1099a07 3f278067 00000000 3f278067 00000001 f7449e78
[ 2.681042] Call Trace:
[ 2.681042] [<c1586d98>] ?
[ 2.681042] [<c1018cf8>] ?
[ 2.681042] [<c1586d98>] ?
[ 2.681042] [<c13e5c85>] ?
[ 2.681042] [<c101f6f5>] ?
[ 2.681042] [<c1096abc>] ?
[ 2.681042] [<c1099a07>] ?
[ 2.681042] [<c103f98f>] ?
[ 2.681042] [<c10196bb>] ?
[ 2.681042] [<c10021a6>] ?
[ 2.681042] [<c1025000>] ?
[ 2.681042] [<c13fe000>] ?
[ 2.681042] [<c13e2e72>] ?
[ 2.681042] [<c13e5c85>] ?
[ 2.681042] [<c13e62ad>] ?
[ 2.681042] [<c13e5c85>] ?
[ 2.681042] [<c13e5c85>] ?
[ 2.681042] [<c13e444e>] ?
[ 2.681042] [<c13e5c85>] ?
[ 2.681042] [<c11f6e90>] ?
[ 2.681042] [<c1003518>] ?
[ 2.681042] Code: 6b c0 14 01 c2 a1 84 a4 5e c1 89 54 24 20 c1 e2 03 89 54 24 18 89 44 24 14 a1 0c 72 69 c1 29 d0 8b 50 04 8b 00 89 d6 09 c6 74 04 <0f> 0b eb fe a1 80 f1 6d c1 89 ce 89 df 89 44 24 1c 89 c8 83 e0
[ 2.681042] EIP: [<c101f63e>] SS:ESP 0068:f7449db8
[ 2.789854] ---[ end trace c5bcbc099d05276b ]---
[ 2.792106] note: init[1] exited with preempt_count 1
[ 2.794384] ------------[ cut here ]------------
[ 2.796606] kernel BUG at arch/x86/mm/highmem_32.c:45!
[ 2.798924] invalid opcode: 0000 [#4] SMP
[ 2.801261] last sysfs file:
[ 2.802047] Modules linked in:
[ 2.802047]
[ 2.802047] Pid: 1, comm: init Tainted: G D (2.6.31.5-vsc #1)
[ 2.802047] EIP: 0060:[<c101f63e>] EFLAGS: 00010286 CPU: 0
[ 2.802047] EAX: 3f27a163 EBX: 80000000 ECX: 00000163 EDX: 80000000
[ 2.802047] ESI: bf27a163 EDI: 3f277067 EBP: c1f1cee0 ESP: f7449b58
[ 2.802047] DS: 0068 ES: 0068 FS: 00d8 GS: 0000 SS: 0068
[ 2.802047] Process init (pid: 1, ti=f7448000 task=f7440000 task.ti=f7448000)
[ 2.802047] Stack:
[ 2.802047] 000000a0 c120fab0 c00b8320 c1718848 00000097 fffff000 00000038 00000004
[ 2.802047] <0> 00000007 c1738000 00000007 3f277067 c1f1ceec c101f6f5 00000163 80000000
[ 2.802047] <0> c1095b26 00000000 c124d828 c169f8b6 3f277067 00000000 00000046 00000000
[ 2.802047] Call Trace:
[ 2.802047] [<c120fab0>] ?
[ 2.802047] [<c101f6f5>] ?
[ 2.802047] [<c1095b26>] ?
[ 2.802047] [<c124d828>] ?
[ 2.802047] [<c109b0f9>] ?
[ 2.802047] [<c102c438>] ?
[ 2.802047] [<c102fc29>] ?
[ 2.802047] [<c103107d>] ?
[ 2.802047] [<c103150a>] ?
[ 2.802047] [<c13e4e51>] ?
[ 2.802047] [<c13e46d6>] ?
[ 2.802047] [<c14f1a4b>] ?
[ 2.802047] [<c10045c8>] ?
[ 2.802047] [<c1004639>] ?
[ 2.802047] [<c101f63e>] ?
[ 2.802047] [<c1586d98>] ?
[ 2.802047] [<c1018cf8>] ?
[ 2.802047] [<c1586d98>] ?
[ 2.802047] [<c1586d98>] ?
[ 2.802047] [<c13e444a>] ?
[ 2.802047] [<c10045c8>] ?
[ 2.802047] [<c11f6ea0>] ?
[ 2.802047] [<c13e444e>] ?
[ 2.802047] [<c13e5c85>] ?
[ 2.802047] [<c10045c8>] ?
[ 2.802047] [<c101f63e>] ?
[ 2.802047] [<c1586d98>] ?
[ 2.802047] [<c1018cf8>] ?
[ 2.802047] [<c1586d98>] ?
[ 2.802047] [<c13e5c85>] ?
[ 2.802047] [<c101f6f5>] ?
[ 2.802047] [<c1096abc>] ?
[ 2.802047] [<c1099a07>] ?
[ 2.802047] [<c103f98f>] ?
[ 2.802047] [<c10196bb>] ?
[ 2.802047] [<c10021a6>] ?
[ 2.802047] [<c1025000>] ?
[ 2.802047] [<c13fe000>] ?
[ 2.802047] [<c13e2e72>] ?
[ 2.802047] [<c13e5c85>] ?
[ 2.802047] [<c13e62ad>] ?
[ 2.802047] [<c13e5c85>] ?
[ 2.802047] [<c13e5c85>] ?
[ 2.802047] [<c13e444e>] ?
[ 2.802047] [<c13e5c85>] ?
[ 2.802047] [<c11f6e90>] ?
[ 2.802047] [<c1003518>] ?
[ 2.802047] Code: 6b c0 14 01 c2 a1 84 a4 5e c1 89 54 24 20 c1 e2 03 89 54 24 18 89 44 24 14 a1 0c 72 69 c1 29 d0 8b 50 04 8b 00 89 d6 09 c6 74 04 <0f> 0b eb fe a1 80 f1 6d c1 89 ce 89 df 89 44 24 1c 89 c8 83 e0
[ 2.802047] EIP: [<c101f63e>] SS:ESP 0068:f7449b58
[ 2.938349] ---[ end trace c5bcbc099d05276c ]---
[ 2.940524] Fixing recursive fault but reboot is needed!
[ 2.942879] BUG: scheduling while atomic: init/1/0x00000002
[ 2.945376] Modules linked in:
[ 2.947214] Pid: 1, comm: init Tainted: G D 2.6.31.5-vsc #1
[ 2.949959] Call Trace:
[ 2.951433] [<c13e2634>] ?
[ 2.953042] [<c1030020>] ?
[ 2.954646] [<c1030f9a>] ?
[ 2.956237] [<c14fbb3a>] ?
[ 2.957862] [<c103150a>] ?
[ 2.959422] [<c13e4e51>] ?
[ 2.961075] [<c13e46d6>] ?
[ 2.962637] [<c14f1a4b>] ?
[ 2.964249] [<c10045c8>] ?
[ 2.965810] [<c1004639>] ?
[ 2.967467] [<c101f63e>] ?
[ 2.969083] [<c120fab0>] ?
[ 2.970683] [<c10196bb>] ?
[ 2.972289] [<c13e444a>] ?
[ 2.973843] [<c10045c8>] ?
[ 2.975457] [<c11f6ea0>] ?
[ 2.977050] [<c13e444e>] ?
[ 2.978669] [<c10045c8>] ?
[ 2.980302] [<c101f63e>] ?
[ 2.981864] [<c120fab0>] ?
[ 2.983466] [<c101f6f5>] ?
[ 2.985026] [<c1095b26>] ?
[ 2.986689] [<c124d828>] ?
[ 2.988262] [<c109b0f9>] ?
[ 2.989878] [<c102c438>] ?
[ 2.991528] [<c102fc29>] ?
[ 2.993092] [<c103107d>] ?
[ 2.994694] [<c103150a>] ?
[ 2.996285] [<c13e4e51>] ?
[ 2.997897] [<c13e46d6>] ?
[ 2.999457] [<c14f1a4b>] ?
[ 3.001122] [<c10045c8>] ?
[ 3.002688] [<c1004639>] ?
[ 3.004300] [<c101f63e>] ?
[ 3.005908] [<c1586d98>] ?
[ 3.007514] [<c1018cf8>] ?
[ 3.009132] [<c1586d98>] ?
[ 3.010738] [<c1586d98>] ?
[ 3.012355] [<c13e444a>] ?
[ 3.013917] [<c10045c8>] ?
[ 3.015521] [<c11f6ea0>] ?
[ 3.017111] [<c13e444e>] ?
[ 3.018722] [<c13e5c85>] ?
[ 3.020371] [<c10045c8>] ?
[ 3.021931] [<c101f63e>] ?
[ 3.023541] [<c1586d98>] ?
[ 3.025105] [<c1018cf8>] ?
[ 3.026763] [<c1586d98>] ?
[ 3.028363] [<c13e5c85>] ?
[ 3.029969] [<c101f6f5>] ?
[ 3.031615] [<c1096abc>] ?
[ 3.033179] [<c1099a07>] ?
[ 3.034782] [<c103f98f>] ?
[ 3.036369] [<c10196bb>] ?
[ 3.037980] [<c10021a6>] ?
[ 3.039549] [<c1025000>] ?
[ 3.041248] [<c13fe000>] ?
[ 3.042855] [<c13e2e72>] ?
[ 3.044416] [<c13e5c85>] ?
[ 3.046025] [<c13e62ad>] ?
[ 3.047641] [<c13e5c85>] ?
[ 3.049250] [<c13e5c85>] ?
[ 3.050858] [<c13e444e>] ?
[ 3.052515] [<c13e5c85>] ?
[ 3.054117] [<c11f6e90>] ?
[ 3.055684] [<c1003518>] ?
[ 3.057362] input: ImExPS/2 Generic Explorer Mouse as /class/input/input1


If anybody's got grsec-enabled KVM guests on Intel i386 using 2.6.31.4 or 2.6.31.5 and don't mind posting your kernel config, I'd definlitely love to check them out. Any guidance would be most appreciated.

[moseleymark]

I rolled another 10 kernels and I finally isolated this down to CONFIG_PAX_MEMORY_SANITIZE. With that disabled and all the original grsec/pax settings re-enabled, the KVM guest boots up just fine in 2.6.31.5.

[PaX Team]

I rolled another 10 kernels and I finally isolated this down to CONFIG_PAX_MEMORY_SANITIZE. With that disabled and all the original grsec/pax settings re-enabled, the KVM guest boots up just fine in 2.6.31.5.

can you enable KALLSYMS to see the backtrace or decode the first oops you get under SANTIZE?

[xmx]

Hi moseleymark,

I am running a 2.6.31.5 (and before .4) grsec-kernel under KVM without any problems with CONFIG_PAX_MEMORY_SANITIZE enabled.

Regards
xmx

[moseleymark]

Here's the full backtrace:


Booting 'Debian GNU/Linux, kernel 2.6.31.5-vsc'

root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /boot/vmlinuz-2.6.31.5-vsc root=/dev/vda1 ro console=tty0 console=ttyS0
,19200n8 quiet
[Linux-bzImage, setup=0x3400, size=0x3515f0]
initrd /boot/initrd.img-2.6.31.5-vsc
[Linux-initrd @ 0x35dab000, 0x2244996 bytes]

Loading, please wait...
[ 4.501231] BUG: unable to handle kernel paging request at fffa7000
[ 4.503796] IP: [<0008be83>] free_hot_cold_page+0xbb/0x218
[ 4.505948] *pdpt = 00000000015ec001 *pde = 00000000015b7067 *pte = 0000000000000000
[ 4.509094] Oops: 0002 [#1] SMP
[ 4.510035] last sysfs file:
[ 4.510035] Modules linked in:
[ 4.510035]
[ 4.510035] Pid: 1165, comm: init Not tainted (2.6.31.5-vsc #1)
[ 4.510035] EIP: 0060:[<0008be83>] EFLAGS: 00010046 CPU: 0
[ 4.510035] EIP is at free_hot_cold_page+0xbb/0x218
[ 4.510035] EAX: 00000000 EBX: 00000246 ECX: 00000400 EDX: 00000001
[ 4.510035] ESI: c4b54a20 EDI: fffa7000 EBP: c3cc9c18 ESP: c3cc9bf0
[ 4.510035] DS: 0068 ES: 0068 FS: 00d8 GS: 0000 SS: 0068
[ 4.510035] Process init (pid: 1165, ti=c3cc8000 task=f6d40000 task.ti=c3cc8000)
[ 4.510035] Stack:
[ 4.510035] fffa7000 00000282 00000000 c3cc9c14 00000000 80100008 00000001 c4b54a20
[ 4.510035] <0> 00000001 c1e63700 c3cc9c20 0008c017 c3cc9c38 0008f450 00000206 c4b54a20
[ 4.510035] <0> 80000000 fffb3520 c3cc9c44 000a74d9 c4b5bf60 c3cc9ce4 0009b128 3a987067
[ 4.510035] Call Trace:
[ 4.510035] [<0008c017>] ? free_hot_page+0xa/0xc
[ 4.510035] [<0008f450>] ? put_page+0xcb/0xd1
[ 4.510035] [<000a74d9>] ? free_page_and_swap_cache+0x38/0x3b
[ 4.510035] [<0009b128>] ? unmap_vmas+0x4a2/0x7b8
[ 4.510035] [<003eefb4>] ? init_amd+0x278/0x559
[ 4.510035] [<00b5e040>] ? iret_exc+0x764550/0xa06510
[ 4.510035] [<000a0498>] ? exit_mmap+0x9d/0x11c
[ 4.510035] [<0002f0c4>] ? mmput+0x24/0x81
[ 4.510035] [<000bb798>] ? flush_old_exec+0x57a/0x82e
[ 4.510035] [<000b624d>] ? vfs_read+0x165/0x1a7
[ 4.510035] [<00203459>] ? set_fs+0xf/0x11
[ 4.510035] [<000ba26d>] ? kernel_read+0x3c/0x45
[ 4.510035] [<000e525f>] ? load_elf_binary+0x34e/0x17b7
[ 4.510035] [<0009867b>] ? page_address+0x77/0x98
[ 4.510035] [<0009867b>] ? page_address+0x77/0x98
[ 4.510035] [<000b9aaf>] ? put_arg_page+0x8/0xa
[ 4.510035] [<000b9d47>] ? copy_strings+0x209/0x21f
[ 4.510035] [<000b9e41>] ? search_binary_handler+0xa5/0x21c
[ 4.510035] [<000baea8>] ? do_execve+0x32d/0x481
[ 4.510035] [<00800000>] ? iret_exc+0x406510/0xa06510
[ 4.510035] [<00010000>] ? mce_disable_cpu+0x3/0x68
[ 4.510035] [<00010000>] ? mce_disable_cpu+0x3/0x68
[ 4.510035] [<000c8000>] ? inode_change_ok+0x68/0x119
[ 4.510035] [<000c8000>] ? inode_change_ok+0x68/0x119
[ 4.510035] [<00002ea1>] ? sys_execve+0x26/0x45
[ 4.510035] [<000047a2>] ? syscall_call+0x7/0xb
[ 4.510035] [<000b007b>] ? __kmalloc+0x20/0x18a
[ 4.510035] [<003f0000>] ? mce_create_device+0x30/0x14a
[ 4.510035] [<000bea8c>] ? putname+0x2d/0x2f
[ 4.510035] [<00002dec>] ? per_cpu__ftrace_cpu_disabled+0xc/0x15
[ 4.510035] [<002031f0>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 4.510035] [<000047d5>] ? restore_all_notrace+0x0/0x18
[ 4.510035] Code: 58 8d 74 26 00 89 c3 fa 90 8d 74 26 00 e8 3c d1 fe ff ba 13 00 00 00 89 f0 e8 49 5e f9 ff b9 00 04 00 00 89 45 d8 31 c0 8b 7d d8 <f3> ab 8b 45 d8 ba 13 00 00 00 e8 de 5b f9 ff f6 c7 02 75 0f 89
[ 4.510035] EIP: [<0008be83>] free_hot_cold_page+0xbb/0x218 SS:ESP 0068:c3cc9bf0
[ 4.510035] CR2: 00000000fffa7000
[ 4.510035] ---[ end trace 44bcce69b8e42733 ]---
[ 4.510035] note: init[1165] exited with preempt_count 2
[ 4.643166] ------------[ cut here ]------------
[ 4.645245] kernel BUG at arch/x86/mm/highmem_32.c:45!
[ 4.647486] invalid opcode: 0000 [#2] SMP
[ 4.649750] last sysfs file:
[ 4.651438] Modules linked in:
[ 4.653070]
[ 4.653070] Pid: 1165, comm: init Tainted: G D (2.6.31.5-vsc #1)
[ 4.653070] EIP: 0060:[<00021c10>] EFLAGS: 00010286 CPU: 0
[ 4.653070] EIP is at kmap_atomic_prot+0x75/0x124
[ 4.653070] EAX: 3a987163 EBX: c4aa8160 ECX: ba987163 EDX: 80000000
[ 4.653070] ESI: 00000007 EDI: c4355000 EBP: c3cc998c ESP: c3cc9958
[ 4.653070] DS: 0068 ES: 0068 FS: 00d8 GS: 0000 SS: 0068
[ 4.653070] Process init (pid: 1165, ti=c3cc8000 task=f6d40000 task.ti=c3cc8000)
[ 4.653070] Stack:
[ 4.653070] c20eed70 c3cc996c 003f5f3f c207b980 00000282 c207b980 c207b980 fffff000
[ 4.653070] <0> 00000038 00000007 00000000 00000007 c4355000 c3cc999c 00021cd3 00000163
[ 4.653070] <0> 80000000 c3cc9a3c 0009af40 3a98b067 00000000 3a98b067 f7402240 c4355000
[ 4.653070] Call Trace:
[ 4.653070] [<003f5f3f>] ? _spin_lock_irqsave+0x1a/0x34
[ 4.653070] [<00021cd3>] ? kmap_atomic+0x14/0x16
[ 4.653070] [<0009af40>] ? unmap_vmas+0x2ba/0x7b8
[ 4.653070] [<00400000>] ? iret_exc+0x6510/0xa06510
[ 4.653070] [<000a0498>] ? exit_mmap+0x9d/0x11c
[ 4.653070] [<0002f0c4>] ? mmput+0x24/0x81
[ 4.653070] [<00032963>] ? exit_mm+0xdc/0xe4
[ 4.653070] [<00033dd4>] ? do_exit+0x182/0x59c
[ 4.653070] [<00031ca1>] ? printk+0x15/0x17
[ 4.653070] [<00034255>] ? do_group_exit+0x67/0x90
[ 4.653070] [<003f6d3a>] ? oops_end+0x9b/0xa3
[ 4.653070] [<00031ca1>] ? printk+0x15/0x17
[ 4.653070] [<0001c66c>] ? no_context+0x221/0x22b
[ 4.653070] [<0001c8ac>] ? __bad_area_nosemaphore+0x193/0x19b
[ 4.653070] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.653070] [<0001c918>] ? bad_area_nosemaphore+0xd/0x10
[ 4.653070] [<00030001>] ? copy_process+0xb3c/0x1084
[ 4.653070] [<003f83a4>] ? do_page_fault+0x6b2/0x6d6
[ 4.653070] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 4.653070] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 4.653070] [<003f62c7>] ? error_code+0x77/0x80
[ 4.653070] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.653070] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.653070] [<003f62cb>] ? error_code+0x7b/0x80
[ 4.653070] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.653070] [<0008be83>] ? free_hot_cold_page+0xbb/0x218
[ 4.653070] [<00010046>] ? mce_disable_cpu+0x49/0x68
[ 4.653070] [<0008c017>] ? free_hot_page+0xa/0xc
[ 4.653070] [<0008f450>] ? put_page+0xcb/0xd1
[ 4.653070] [<000a74d9>] ? free_page_and_swap_cache+0x38/0x3b
[ 4.653070] [<0009b128>] ? unmap_vmas+0x4a2/0x7b8
[ 4.653070] [<003eefb4>] ? init_amd+0x278/0x559
[ 4.653070] [<00b5e040>] ? iret_exc+0x764550/0xa06510
[ 4.653070] [<000a0498>] ? exit_mmap+0x9d/0x11c
[ 4.653070] [<0002f0c4>] ? mmput+0x24/0x81
[ 4.653070] [<000bb798>] ? flush_old_exec+0x57a/0x82e
[ 4.653070] [<000b624d>] ? vfs_read+0x165/0x1a7
[ 4.653070] [<00203459>] ? set_fs+0xf/0x11
[ 4.653070] [<000ba26d>] ? kernel_read+0x3c/0x45
[ 4.653070] [<000e525f>] ? load_elf_binary+0x34e/0x17b7
[ 4.653070] [<0009867b>] ? page_address+0x77/0x98
[ 4.653070] [<0009867b>] ? page_address+0x77/0x98
[ 4.653070] [<000b9aaf>] ? put_arg_page+0x8/0xa
[ 4.653070] [<000b9d47>] ? copy_strings+0x209/0x21f
[ 4.653070] [<000b9e41>] ? search_binary_handler+0xa5/0x21c
[ 4.653070] [<000baea8>] ? do_execve+0x32d/0x481
[ 4.653070] [<00800000>] ? iret_exc+0x406510/0xa06510
[ 4.653070] [<00010000>] ? mce_disable_cpu+0x3/0x68
[ 4.653070] [<00010000>] ? mce_disable_cpu+0x3/0x68
[ 4.653070] [<000c8000>] ? inode_change_ok+0x68/0x119
[ 4.653070] [<000c8000>] ? inode_change_ok+0x68/0x119
[ 4.653070] [<00002ea1>] ? sys_execve+0x26/0x45
[ 4.653070] [<000047a2>] ? syscall_call+0x7/0xb
[ 4.653070] [<000b007b>] ? __kmalloc+0x20/0x18a
[ 4.653070] [<003f0000>] ? mce_create_device+0x30/0x14a
[ 4.653070] [<000bea8c>] ? putname+0x2d/0x2f
[ 4.653070] [<00002dec>] ? per_cpu__ftrace_cpu_disabled+0xc/0x15
[ 4.653070] [<002031f0>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 4.653070] [<000047d5>] ? restore_all_notrace+0x0/0x18
[ 4.653070] Code: 19 00 00 6b c0 14 01 c2 a1 84 c4 e2 c1 89 55 f0 c1 e2 03 89 55 ec 89 45 e8 a1 0c a2 06 c2 29 d0 8b 50 04 8b 00 89 d1 09 c1 74 04 <0f> 0b eb fe ff 15 2c 52 5b c1 8b 45 08 8b 0d 80 21 0b c2 8b 75
[ 4.653070] EIP: [<00021c10>] kmap_atomic_prot+0x75/0x124 SS:ESP 0068:c3cc9958
[ 4.843430] ---[ end trace 44bcce69b8e42734 ]---
[ 4.845545] Fixing recursive fault but reboot is needed!
[ 4.847831] BUG: scheduling while atomic: init/1165/0x00000003
[ 4.850315] Modules linked in:
[ 4.852135] Pid: 1165, comm: init Tainted: G D 2.6.31.5-vsc #1
[ 4.854865] Call Trace:
[ 4.856272] [<0002aa70>] __schedule_bug+0x5b/0x62
[ 4.858437] [<003f43b0>] schedule+0x71/0x8ea
[ 4.860496] [<000072a9>] ? write_ldt+0x22e/0x246
[ 4.862670] [<00031ca1>] ? printk+0x15/0x17
[ 4.864675] [<00033cf1>] do_exit+0x9f/0x59c
[ 4.866672] [<00031ca1>] ? printk+0x15/0x17
[ 4.868669] [<00034255>] do_group_exit+0x67/0x90
[ 4.870834] [<003f6d3a>] oops_end+0x9b/0xa3
[ 4.872868] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 4.875202] [<0000763a>] die+0x54/0x5a
[ 4.877065] [<003f6573>] do_trap+0x89/0xae
[ 4.879036] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 4.881271] [<00005ad2>] do_invalid_op+0x7b/0x85
[ 4.883444] [<00030002>] ? copy_process+0xb3d/0x1084
[ 4.885683] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 4.887978] [<00024e60>] ? __enqueue_entity+0x9c/0xa4
[ 4.890269] [<00024ee2>] ? enqueue_entity+0x7a/0x7f
[ 4.892584] [<00024ff1>] ? hrtick_update+0x27/0x29
[ 4.894767] [<00025022>] ? enqueue_task_fair+0x2f/0x33
[ 4.897064] [<003f62c7>] ? error_code+0x77/0x80
[ 4.899174] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 4.901402] [<00203200>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 4.903902] [<003f62cb>] error_code+0x7b/0x80
[ 4.906001] [<00310068>] ? proc_bulk+0x15c/0x563
[ 4.908137] [<00020068>] ? pgd_free+0x1a/0x13c
[ 4.910246] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 4.912431] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 4.914725] [<00010286>] ? mce_read+0x221/0x313
[ 4.916829] [<003f5f3f>] ? _spin_lock_irqsave+0x1a/0x34
[ 4.919187] [<00021cd3>] kmap_atomic+0x14/0x16
[ 4.921343] [<0009af40>] unmap_vmas+0x2ba/0x7b8
[ 4.923449] [<00400000>] ? iret_exc+0x6510/0xa06510
[ 4.925664] [<000a0498>] exit_mmap+0x9d/0x11c
[ 4.927717] [<0002f0c4>] mmput+0x24/0x81
[ 4.929634] [<00032963>] exit_mm+0xdc/0xe4
[ 4.931678] [<00033dd4>] do_exit+0x182/0x59c
[ 4.933703] [<00031ca1>] ? printk+0x15/0x17
[ 4.935698] [<00034255>] do_group_exit+0x67/0x90
[ 4.937830] [<003f6d3a>] oops_end+0x9b/0xa3
[ 4.939831] [<00031ca1>] ? printk+0x15/0x17
[ 4.941904] [<0001c66c>] no_context+0x221/0x22b
[ 4.944008] [<0001c8ac>] __bad_area_nosemaphore+0x193/0x19b
[ 4.946436] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.948613] [<0001c918>] bad_area_nosemaphore+0xd/0x10
[ 4.951013] [<00030001>] ? copy_process+0xb3c/0x1084
[ 4.953256] [<003f83a4>] do_page_fault+0x6b2/0x6d6
[ 4.955432] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 4.957615] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 4.970723] [<003f62c7>] ? error_code+0x77/0x80
[ 4.972857] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.975039] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.977230] [<003f62cb>] error_code+0x7b/0x80
[ 4.979280] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 4.981579] [<0008be83>] ? free_hot_cold_page+0xbb/0x218
[ 4.983938] [<00010046>] ? mce_disable_cpu+0x49/0x68
[ 4.986189] [<0008c017>] free_hot_page+0xa/0xc
[ 4.988276] [<0008f450>] put_page+0xcb/0xd1
[ 4.990310] [<000a74d9>] free_page_and_swap_cache+0x38/0x3b
[ 4.992837] [<0009b128>] unmap_vmas+0x4a2/0x7b8
[ 4.994941] [<003eefb4>] ? init_amd+0x278/0x559
[ 4.997048] [<00b5e040>] ? iret_exc+0x764550/0xa06510
[ 4.999310] [<000a0498>] exit_mmap+0x9d/0x11c
[ 5.001424] [<0002f0c4>] mmput+0x24/0x81
[ 5.003354] [<000bb798>] flush_old_exec+0x57a/0x82e
[ 5.005567] [<000b624d>] ? vfs_read+0x165/0x1a7
[ 5.007668] [<00203459>] ? set_fs+0xf/0x11
[ 5.009639] [<000ba26d>] ? kernel_read+0x3c/0x45
[ 5.011846] [<000e525f>] load_elf_binary+0x34e/0x17b7
[ 5.014150] [<0009867b>] ? page_address+0x77/0x98
[ 5.016350] [<0009867b>] ? page_address+0x77/0x98
[ 5.018506] [<000b9aaf>] ? put_arg_page+0x8/0xa
[ 5.020647] [<000b9d47>] ? copy_strings+0x209/0x21f
[ 5.022899] [<000b9e41>] search_binary_handler+0xa5/0x21c
[ 5.025270] [<000baea8>] do_execve+0x32d/0x481
[ 5.027379] [<00800000>] ? iret_exc+0x406510/0xa06510
[ 5.029671] [<00010000>] ? mce_disable_cpu+0x3/0x68
[ 5.031945] [<00010000>] ? mce_disable_cpu+0x3/0x68
[ 5.034166] [<000c8000>] ? inode_change_ok+0x68/0x119
[ 5.036429] [<000c8000>] ? inode_change_ok+0x68/0x119
[ 5.038722] [<00002ea1>] sys_execve+0x26/0x45
[ 5.040899] [<000047a2>] syscall_call+0x7/0xb
[ 5.042989] [<000b007b>] ? __kmalloc+0x20/0x18a
[ 5.045090] [<003f0000>] ? mce_create_device+0x30/0x14a
[ 5.047409] [<000bea8c>] ? putname+0x2d/0x2f
[ 5.049426] [<00002dec>] ? per_cpu__ftrace_cpu_disabled+0xc/0x15
[ 5.052119] [<002031f0>] ? trace_hardirqs_on_thunk+0xc/0x10
[ 5.054550] [<000047d5>] ? restore_all_notrace+0x0/0x18
[ 5.059215] ------------[ cut here ]------------
[ 5.061360] kernel BUG at arch/x86/mm/highmem_32.c:45!
[ 5.062104] invalid opcode: 0000 [#3] SMP
[ 5.062104] last sysfs file:
[ 5.062104] Modules linked in:
[ 5.062104]
[ 5.062104] Pid: 1, comm: init Tainted: G D (2.6.31.5-vsc #1)
[ 5.062104] EIP: 0060:[<00021c10>] EFLAGS: 00010286 CPU: 0
[ 5.062104] EIP is at kmap_atomic_prot+0x75/0x124
[ 5.062104] EAX: 3a987163 EBX: c4aa8060 ECX: ba987163 EDX: 80000000
[ 5.062104] ESI: 3a983067 EDI: 003f7cf2 EBP: f7449d7c ESP: f7449d48
[ 5.062104] DS: 0068 ES: 0068 FS: 00d8 GS: 0033 SS: 0068
[ 5.062104] Process init (pid: 1, ti=f7448000 task=f7440000 task.ti=f7448000)
[ 5.062104] Stack:
[ 5.062104] 0001abc9 015b7d90 00000000 00000001 3a98a067 015b7d90 00000000 fffff000
[ 5.062104] <0> 00000038 00000007 00000007 3a983067 003f7cf2 f7449d8c 00021cd3 00000163
[ 5.062104] <0> 80000000 f7449ef4 0009beec f7449dc8 0001abc9 015b7d98 00000000 00000001
[ 5.062104] Call Trace:
[ 5.062104] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.062104] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.062104] [<00021cd3>] ? kmap_atomic+0x14/0x16
[ 5.062104] [<0009beec>] ? handle_mm_fault+0x51c/0x153f
[ 5.062104] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.062104] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.062104] [<00021acd>] ? kunmap_atomic+0x5d/0x6d
[ 5.062104] [<0001b1f3>] ? paravirt_leave_lazy_mmu+0xd/0xf
[ 5.062104] [<0009ed46>] ? copy_page_range+0x5ac/0x642
[ 5.062104] [<00003c2f>] ? do_notify_resume+0x6e/0x31f
[ 5.062104] [<00042e66>] ? alloc_pid+0x19/0x2f2
[ 5.062104] [<0001b82a>] ? pvclock_clocksource_read+0x49/0xcd
[ 5.062104] [<000038cd>] ? sys_sigaction+0x67/0x199
[ 5.062104] [<00024e60>] ? __enqueue_entity+0x9c/0xa4
[ 5.062104] [<000038cd>] ? sys_sigaction+0x67/0x199
[ 5.062104] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.062104] [<0001b0bc>] ? native_load_tls+0x35/0x39
[ 5.062104] [<00003359>] ? __switch_to+0x100/0x182
[ 5.062104] [<00027990>] ? finish_task_switch+0x58/0xd8
[ 5.062104] [<00006280>] ? per_cpu__runqueues+0x0/0x500
[ 5.062104] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.062104] [<003f82e8>] ? do_page_fault+0x5f6/0x6d6
[ 5.062104] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.062104] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.062104] [<003f62cb>] ? error_code+0x7b/0x80
[ 5.062104] [<00200000>] ? radix_tree_gang_lookup+0x98/0xa7
[ 5.062104] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.062104] [<00010207>] ? mce_read+0x1a2/0x313
[ 5.062104] Code: 19 00 00 6b c0 14 01 c2 a1 84 c4 e2 c1 89 55 f0 c1 e2 03 89 55 ec 89 45 e8 a1 0c a2 06 c2 29 d0 8b 50 04 8b 00 89 d1 09 c1 74 04 <0f> 0b eb fe ff 15 2c 52 5b c1 8b 45 08 8b 0d 80 21 0b c2 8b 75
[ 5.062104] EIP: [<00021c10>] kmap_atomic_prot+0x75/0x124 SS:ESP 0068:f7449d48
[ 5.192737] ---[ end trace 44bcce69b8e42735 ]---
[ 5.194820] note: init[1] exited with preempt_count 1
[ 5.197034] ------------[ cut here ]------------
[ 5.199122] kernel BUG at arch/x86/mm/highmem_32.c:45!
[ 5.201433] invalid opcode: 0000 [#4] SMP
[ 5.202723] last sysfs file:
[ 5.202723] Modules linked in:
[ 5.202723]
[ 5.202723] Pid: 1, comm: init Tainted: G D (2.6.31.5-vsc #1)
[ 5.202723] EIP: 0060:[<00021c10>] EFLAGS: 00010286 CPU: 0
[ 5.202723] EIP is at kmap_atomic_prot+0x75/0x124
[ 5.202723] EAX: 3a987163 EBX: c4aa8080 ECX: ba987163 EDX: 80000000
[ 5.202723] ESI: 00000007 EDI: c4355000 EBP: f7449ad4 ESP: f7449aa0
[ 5.202723] DS: 0068 ES: 0068 FS: 00d8 GS: 0000 SS: 0068
[ 5.202723] Process init (pid: 1, ti=f7448000 task=f7440000 task.ti=f7448000)
[ 5.202723] Stack:
[ 5.202723] 00000004 f7449ac0 00000000 f7420000 c00bca60 f7449ac4 003f842e fffff000
[ 5.202723] <0> 00000038 00000007 00000000 00000007 c4355000 f7449ae4 00021cd3 00000163
[ 5.202723] <0> 80000000 f7449b84 0009af40 3a984067 00000000 3a984067 00000000 c4355000
[ 5.202723] Call Trace:
[ 5.202723] [<003f842e>] ? __atomic_notifier_call_chain+0x11/0x13
[ 5.202723] [<00021cd3>] ? kmap_atomic+0x14/0x16
[ 5.202723] [<0009af40>] ? unmap_vmas+0x2ba/0x7b8
[ 5.202723] [<00400000>] ? iret_exc+0x6510/0xa06510
[ 5.202723] [<000a0498>] ? exit_mmap+0x9d/0x11c
[ 5.202723] [<0002f0c4>] ? mmput+0x24/0x81
[ 5.202723] [<00032963>] ? exit_mm+0xdc/0xe4
[ 5.202723] [<00033dd4>] ? do_exit+0x182/0x59c
[ 5.202723] [<00031ca1>] ? printk+0x15/0x17
[ 5.202723] [<00034255>] ? do_group_exit+0x67/0x90
[ 5.202723] [<003f6d3a>] ? oops_end+0x9b/0xa3
[ 5.202723] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.202723] [<0000763a>] ? die+0x54/0x5a
[ 5.202723] [<003f6573>] ? do_trap+0x89/0xae
[ 5.202723] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.202723] [<00005ad2>] ? do_invalid_op+0x7b/0x85
[ 5.202723] [<00030002>] ? copy_process+0xb3d/0x1084
[ 5.202723] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.202723] [<0001aaef>] ? kvm_deferred_mmu_op+0x33/0x7a
[ 5.202723] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.202723] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.202723] [<003f62c7>] ? error_code+0x77/0x80
[ 5.202723] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.202723] [<00203200>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 5.202723] [<003f62cb>] ? error_code+0x7b/0x80
[ 5.202723] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.202723] [<00010068>] ? mce_read+0x3/0x313
[ 5.202723] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.202723] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.202723] [<00010286>] ? mce_read+0x221/0x313
[ 5.202723] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.202723] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.202723] [<00021cd3>] ? kmap_atomic+0x14/0x16
[ 5.202723] [<0009beec>] ? handle_mm_fault+0x51c/0x153f
[ 5.202723] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.202723] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.202723] [<00021acd>] ? kunmap_atomic+0x5d/0x6d
[ 5.202723] [<0001b1f3>] ? paravirt_leave_lazy_mmu+0xd/0xf
[ 5.202723] [<0009ed46>] ? copy_page_range+0x5ac/0x642
[ 5.202723] [<00003c2f>] ? do_notify_resume+0x6e/0x31f
[ 5.202723] [<00042e66>] ? alloc_pid+0x19/0x2f2
[ 5.202723] [<0001b82a>] ? pvclock_clocksource_read+0x49/0xcd
[ 5.202723] [<000038cd>] ? sys_sigaction+0x67/0x199
[ 5.202723] [<00024e60>] ? __enqueue_entity+0x9c/0xa4
[ 5.202723] [<000038cd>] ? sys_sigaction+0x67/0x199
[ 5.202723] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.202723] [<0001b0bc>] ? native_load_tls+0x35/0x39
[ 5.202723] [<00003359>] ? __switch_to+0x100/0x182
[ 5.202723] [<00027990>] ? finish_task_switch+0x58/0xd8
[ 5.202723] [<00006280>] ? per_cpu__runqueues+0x0/0x500
[ 5.202723] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.202723] [<003f82e8>] ? do_page_fault+0x5f6/0x6d6
[ 5.202723] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.202723] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.202723] [<003f62cb>] ? error_code+0x7b/0x80
[ 5.202723] [<00200000>] ? radix_tree_gang_lookup+0x98/0xa7
[ 5.202723] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.202723] [<00010207>] ? mce_read+0x1a2/0x313
[ 5.202723] Code: 19 00 00 6b c0 14 01 c2 a1 84 c4 e2 c1 89 55 f0 c1 e2 03 89 55 ec 89 45 e8 a1 0c a2 06 c2 29 d0 8b 50 04 8b 00 89 d1 09 c1 74 04 <0f> 0b eb fe ff 15 2c 52 5b c1 8b 45 08 8b 0d 80 21 0b c2 8b 75
[ 5.202723] EIP: [<00021c10>] kmap_atomic_prot+0x75/0x124 SS:ESP 0068:f7449aa0
[ 5.395799] ---[ end trace 44bcce69b8e42736 ]---
[ 5.408749] Fixing recursive fault but reboot is needed!
[ 5.411162] BUG: scheduling while atomic: init/1/0x00000002
[ 5.413557] Modules linked in:
[ 5.415342] Pid: 1, comm: init Tainted: G D 2.6.31.5-vsc #1
[ 5.417987] Call Trace:
[ 5.419404] [<0002aa70>] __schedule_bug+0x5b/0x62
[ 5.421634] [<003f43b0>] schedule+0x71/0x8ea
[ 5.423668] [<00031ca1>] ? printk+0x15/0x17
[ 5.425664] [<00033cf1>] do_exit+0x9f/0x59c
[ 5.427661] [<00031ca1>] ? printk+0x15/0x17
[ 5.429657] [<00034255>] do_group_exit+0x67/0x90
[ 5.431853] [<003f6d3a>] oops_end+0x9b/0xa3
[ 5.433851] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.436145] [<0000763a>] die+0x54/0x5a
[ 5.438002] [<003f6573>] do_trap+0x89/0xae
[ 5.439966] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.442216] [<00005ad2>] do_invalid_op+0x7b/0x85
[ 5.444357] [<00030002>] ? copy_process+0xb3d/0x1084
[ 5.446641] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.448939] [<003f5ff1>] ? _spin_unlock_irqrestore+0x1d/0x30
[ 5.451455] [<003f842e>] ? __atomic_notifier_call_chain+0x11/0x13
[ 5.454083] [<003f843c>] ? atomic_notifier_call_chain+0xc/0xe
[ 5.456562] [<003f62c7>] ? error_code+0x77/0x80
[ 5.458663] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.460871] [<00203200>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 5.463399] [<003f62cb>] error_code+0x7b/0x80
[ 5.465448] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.467614] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.469905] [<00010286>] ? mce_read+0x221/0x313
[ 5.472096] [<003f842e>] ? __atomic_notifier_call_chain+0x11/0x13
[ 5.474723] [<00021cd3>] kmap_atomic+0x14/0x16
[ 5.476800] [<0009af40>] unmap_vmas+0x2ba/0x7b8
[ 5.478912] [<00400000>] ? iret_exc+0x6510/0xa06510
[ 5.481181] [<000a0498>] exit_mmap+0x9d/0x11c
[ 5.483263] [<0002f0c4>] mmput+0x24/0x81
[ 5.485186] [<00032963>] exit_mm+0xdc/0xe4
[ 5.487157] [<00033dd4>] do_exit+0x182/0x59c
[ 5.489189] [<00031ca1>] ? printk+0x15/0x17
[ 5.491251] [<00034255>] do_group_exit+0x67/0x90
[ 5.493382] [<003f6d3a>] oops_end+0x9b/0xa3
[ 5.495393] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.497733] [<0000763a>] die+0x54/0x5a
[ 5.499591] [<003f6573>] do_trap+0x89/0xae
[ 5.501643] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.503853] [<00005ad2>] do_invalid_op+0x7b/0x85
[ 5.505995] [<00030002>] ? copy_process+0xb3d/0x1084
[ 5.508239] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.510569] [<0001aaef>] ? kvm_deferred_mmu_op+0x33/0x7a
[ 5.512970] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.515197] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.517655] [<003f62c7>] ? error_code+0x77/0x80
[ 5.519765] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.521991] [<00203200>] ? trace_hardirqs_off_thunk+0xc/0x10
[ 5.524471] [<003f62cb>] error_code+0x7b/0x80
[ 5.526523] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.528703] [<00010068>] ? mce_read+0x3/0x313
[ 5.530780] [<00005a57>] ? do_invalid_op+0x0/0x85
[ 5.532974] [<00021c10>] ? kmap_atomic_prot+0x75/0x124
[ 5.535264] [<00010286>] ? mce_read+0x221/0x313
[ 5.537364] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.539571] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.541941] [<00021cd3>] kmap_atomic+0x14/0x16
[ 5.544021] [<0009beec>] handle_mm_fault+0x51c/0x153f
[ 5.546265] [<0001abc9>] ? kvm_mmu_write+0x70/0x78
[ 5.548422] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.550878] [<00021acd>] ? kunmap_atomic+0x5d/0x6d
[ 5.553081] [<0001b1f3>] ? paravirt_leave_lazy_mmu+0xd/0xf
[ 5.555457] [<0009ed46>] ? copy_page_range+0x5ac/0x642
[ 5.557719] [<00003c2f>] ? do_notify_resume+0x6e/0x31f
[ 5.559983] [<00042e66>] ? alloc_pid+0x19/0x2f2
[ 5.562142] [<0001b82a>] ? pvclock_clocksource_read+0x49/0xcd
[ 5.564594] [<000038cd>] ? sys_sigaction+0x67/0x199
[ 5.566779] [<00024e60>] ? __enqueue_entity+0x9c/0xa4
[ 5.569015] [<000038cd>] ? sys_sigaction+0x67/0x199
[ 5.571266] [<0001b212>] ? native_pax_close_kernel+0x1d/0x23
[ 5.573698] [<0001b0bc>] ? native_load_tls+0x35/0x39
[ 5.575938] [<00003359>] ? __switch_to+0x100/0x182
[ 5.578095] [<00027990>] ? finish_task_switch+0x58/0xd8
[ 5.580415] [<00006280>] ? per_cpu__runqueues+0x0/0x500
[ 5.582756] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.584916] [<003f82e8>] do_page_fault+0x5f6/0x6d6
[ 5.587074] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.589232] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.591453] [<003f62cb>] error_code+0x7b/0x80
[ 5.593486] [<00200000>] ? radix_tree_gang_lookup+0x98/0xa7
[ 5.595886] [<003f7cf2>] ? do_page_fault+0x0/0x6d6
[ 5.598042] [<00010207>] ? mce_read+0x1a2/0x313
[ 5.600211] input: ImExPS/2 Generic Explorer Mouse as /class/input/input1

[PaX Team]

[ 4.501231] BUG: unable to handle kernel paging request at fffa7000
[ 4.503796] IP: [<0008be83>] free_hot_cold_page+0xbb/0x218
[ 4.505948] *pdpt = 00000000015ec001 *pde = 00000000015b7067 *pte = 0000000000000000

this is weird, it seems as if kmap_atomic hadn't actually established a mapping for sanitize_highpage. i can't tell if it's a kvm related bug or something else yet. can you send me your config/vmlinux/etc?