Kernel caused hang with grsecurity-3.1-4.2.6-201511282239

[peetaur]

Kernel caused a hang... my virtual machines kept running, and I could ssh to the host, but the video on the host was unusable (and possibly the keyboard and mouse too). Xorg wasn't responding, and kill -9 wouldn't even work. I'm using the open radeon driver. The hang happened while idling for a long time (almost 1.23 hours if 4429 seconds is correct) on a text tty.

grsecurity-3.1-4.2.6-201511282239

Code: Select all

[    0.000000] Linux version 4.2.6-1-grsec-kvm-host (peter@peter) (gcc version 5.2.0 (GCC) ) #30 SMP PREEMPT Wed Dec 2 19:49:05 CET 2015
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.2.6-1-grsec-kvm-host-x86_64 root=UUID=dc395127-6336-448f-a950-137c100420c9 rw pcie_acs_override=downstream apparmor=1 security=apparmor vfio-pci.pci-ids=00:13.0,00:13.2,00:14.2,00:16.0,00:16.2,01:00.1,04:00.0,04:00.1,05:00.0,05:00.1 softlevel=kvm
[...]
[ 4429.146971] PAX: size overflow detected in function set_selection drivers/tty/vt/selection.c:175 cicus.210_438 max, count: 39, decl: sel_end; num: 0; context: vardecl_selection.c;
[ 4429.146974] CPU: 2 PID: 2052 Comm: gpm Not tainted 4.2.6-1-grsec-kvm-host #30
[ 4429.146974] Hardware name: Gigabyte Technology Co., Ltd. GA-990FXA-UD5/GA-990FXA-UD5, BIOS F11 10/26/2012
[ 4429.146978]  ffffffff81884883 b571f7587211844e 0000000000000000 ffffffff817ae4be
[ 4429.146979]  ffffc900046b3b18 ffffffff815ab32c 0000000000000000 ffffffff817ae4b0
[ 4429.146981]  ffffc900046b3b48 ffffffff811a73c4 ffff880457fbb000 ffff880447b60021
[ 4429.146981] Call Trace:
[ 4429.146988]  [<ffffffff815ab32c>] dump_stack+0x4c/0x79
[ 4429.146991]  [<ffffffff811a73c4>] report_size_overflow+0x34/0x40
[ 4429.146994]  [<ffffffff813aeb78>] set_selection+0xa08/0xcc0
[ 4429.146996]  [<ffffffff813b7f35>] tioclinux+0x85/0x230
[ 4429.146998]  [<ffffffff81072ae9>] ? capable+0x19/0x20
[ 4429.147000]  [<ffffffff813aa5ee>] vt_ioctl+0x1be/0x1d60
[ 4429.147003]  [<ffffffff8139c81a>] tty_ioctl+0x45a/0xe70
[ 4429.147005]  [<ffffffff811b6290>] do_vfs_ioctl+0x3f0/0x770
[ 4429.147007]  [<ffffffff811b6687>] SyS_ioctl+0x77/0x90
[ 4429.147009]  [<ffffffff815b0beb>] entry_SYSCALL_64_fastpath+0x12/0x85
[ 4490.443568] usb 1-1: USB disconnect, device number 2
[ 4490.620112] usb 1-2: USB disconnect, device number 3
[ 4492.659250] usb 1-2: new low-speed USB device number 4 using xhci_hcd
[ 4492.814560] usb 1-2: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[ 4492.814565] usb 1-2: ep 0x82 - rounding interval to 1024 microframes, ep desc says 2040 microframes
[ 4492.842209] input: Logitech USB Keyboard as /devices/pci0000:00/0000:00:09.0/0000:02:00.0/usb1/1-2/1-2:1.0/0003:046D:C31D.0008/input/input10
[ 4492.893096] hid-generic 0003:046D:C31D.0008: input,hidraw0: USB HID v1.10 Keyboard [Logitech USB Keyboard] on usb-0000:02:00.0-2/input0
[ 4492.933734] input: Logitech USB Keyboard as /devices/pci0000:00/0000:00:09.0/0000:02:00.0/usb1/1-2/1-2:1.1/0003:046D:C31D.0009/input/input11
[ 4492.986394] hid-generic 0003:046D:C31D.0009: input,hidraw1: USB HID v1.10 Device [Logitech USB Keyboard] on usb-0000:02:00.0-2/input1
[ 4494.801617] usb 1-1: new low-speed USB device number 5 using xhci_hcd
[ 4494.955010] usb 1-1: ep 0x81 - rounding interval to 64 microframes, ep desc says 80 microframes
[ 4494.970654] input: DELL DELL USB Laser Mouse as /devices/pci0000:00/0000:00:09.0/0000:02:00.0/usb1/1-1/1-1:1.0/0003:046D:C063.000A/input/input12
[ 4494.971265] hid-generic 0003:046D:C063.000A: input,hidraw3: USB HID v1.10 Mouse [DELL DELL USB Laser Mouse] on usb-0000:02:00.0-1/input0
[ 4566.395153] INFO: task kworker/7:1:102 blocked for more than 120 seconds.
[ 4566.395155]       Not tainted 4.2.6-1-grsec-kvm-host #30
[ 4566.395155] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4566.395158] kworker/7:1     D 0000000000010e00     0   102      2 0x00000000
[ 4566.395161] Workqueue: events ffffffff813b81a0
[ 4566.395162]  ffffc90001e9bc48 0000000000000046 0000000000000001 ffff88045aa01600
[ 4566.395163]  ffffc90001e9bc68 ffff88045aa01d20 ffffffff819c3350 ffff88045aa01600
[ 4566.395164]  0000000000000000 ffff88046fdd0600 ffffc90001e9bc68 ffffffff815acd5d
[ 4566.395165] Call Trace:
[ 4566.395170]  [<ffffffff815acd5d>] schedule+0x3d/0x90
[ 4566.395172]  [<ffffffff815af800>] schedule_timeout+0x190/0x210
[ 4566.395175]  [<ffffffff8109ba97>] ? account_entity_dequeue+0x97/0xd0
[ 4566.395177]  [<ffffffff812df054>] ? __pax_list_add+0x24/0x50
[ 4566.395178]  [<ffffffff815ae808>] __down+0x68/0xb0
[ 4566.395180]  [<ffffffff810a0700>] ? throttle_cfs_rq+0x170/0x170
[ 4566.395182]  [<ffffffff810b3b2f>] down+0x3f/0x50
[ 4566.395184]  [<ffffffff810b8dd0>] console_lock+0x10/0x30
[ 4566.395186]  [<ffffffff813b81bd>] console_callback+0x1d/0x180
[ 4566.395187]  [<ffffffff810850c5>] process_one_work+0x125/0x370
[ 4566.395189]  [<ffffffff8108535c>] worker_thread+0x4c/0x460
[ 4566.395190]  [<ffffffff81085310>] ? process_one_work+0x370/0x370
[ 4566.395191]  [<ffffffff81085310>] ? process_one_work+0x370/0x370
[ 4566.395192]  [<ffffffff8108b2d7>] kthread+0xd7/0xf0
[ 4566.395193]  [<ffffffff8108b200>] ? kthread_worker_fn+0x160/0x160
[ 4566.395195]  [<ffffffff815b0ffe>] ret_from_fork+0x3e/0x70
[ 4566.395196]  [<ffffffff8108b200>] ? kthread_worker_fn+0x160/0x160
[ 4641.662558] kactivitymanage[2849]: segfault at 71ae3695dd10 ip 000071ae36baf881 sp 000074586efce708 error 4 in libQt5Sql.so.5.5.1[71ae36b99000+41000]
[ 4641.662575] grsec: Segmentation fault occurred at 000071ae3695dd10 in /usr/bin/kactivitymanagerd[kactivitymanage:2849] uid/euid:1000/1000 gid/egid:100/100, parent /usr/bin/init[init:1] uid/euid:0/0 gid/egid:0/0
[ 4646.753066] synergys[6852]: segfault at 0 ip 0000000000455960 sp 0000778b9390cb30 error 4 in synergys[400000+de000]
[ 4646.753078] grsec: Segmentation fault occurred at            (nil) in /usr/lib/synergy/synergys[synergys:6852] uid/euid:1000/1000 gid/egid:100/100, parent /usr/bin/bash[bash:6844] uid/euid:1000/1000 gid/egid:100/100
[ 4686.535377] INFO: task kworker/7:1:102 blocked for more than 120 seconds.
[ 4686.535378]       Not tainted 4.2.6-1-grsec-kvm-host #30
[ 4686.535378] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4686.535381] kworker/7:1     D 0000000000010e00     0   102      2 0x00000000
[ 4686.535384] Workqueue: events ffffffff813b81a0
[ 4686.535386]  ffffc90001e9bc48 0000000000000046 0000000000000001 ffff88045aa01600
[ 4686.535387]  ffffc90001e9bc68 ffff88045aa01d20 ffffffff819c3350 ffff88045aa01600
[ 4686.535388]  0000000000000000 ffff88046fdd0600 ffffc90001e9bc68 ffffffff815acd5d
[ 4686.535388] Call Trace:
[ 4686.535394]  [<ffffffff815acd5d>] schedule+0x3d/0x90
[ 4686.535396]  [<ffffffff815af800>] schedule_timeout+0x190/0x210
[ 4686.535398]  [<ffffffff8109ba97>] ? account_entity_dequeue+0x97/0xd0
[ 4686.535400]  [<ffffffff812df054>] ? __pax_list_add+0x24/0x50
[ 4686.535401]  [<ffffffff815ae808>] __down+0x68/0xb0
[ 4686.535403]  [<ffffffff810a0700>] ? throttle_cfs_rq+0x170/0x170
[ 4686.535405]  [<ffffffff810b3b2f>] down+0x3f/0x50
[ 4686.535406]  [<ffffffff810b8dd0>] console_lock+0x10/0x30
[ 4686.535408]  [<ffffffff813b81bd>] console_callback+0x1d/0x180
[ 4686.535410]  [<ffffffff810850c5>] process_one_work+0x125/0x370
[ 4686.535411]  [<ffffffff8108535c>] worker_thread+0x4c/0x460
[ 4686.535412]  [<ffffffff81085310>] ? process_one_work+0x370/0x370
[ 4686.535413]  [<ffffffff81085310>] ? process_one_work+0x370/0x370
[ 4686.535414]  [<ffffffff8108b2d7>] kthread+0xd7/0xf0
[ 4686.535415]  [<ffffffff8108b200>] ? kthread_worker_fn+0x160/0x160
[ 4686.535417]  [<ffffffff815b0ffe>] ret_from_fork+0x3e/0x70
[ 4686.535418]  [<ffffffff8108b200>] ? kthread_worker_fn+0x160/0x160
[ 4686.535442] INFO: task kworker/2:2:6031 blocked for more than 120 seconds.
[ 4686.535443]       Not tainted 4.2.6-1-grsec-kvm-host #30
[ 4686.535443] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 4686.535444] kworker/2:2     D 0000000000010e00     0  6031      2 0x00000080
[ 4686.535446] Workqueue: events ffffffff813a6340
[ 4686.535447]  ffffc9000343bb28 0000000000000046 0000000000000000 ffff880425bed800
[ 4686.535448]  0000000000000000 ffff880425bedf20 ffffffff819c3350 ffff880425bed800
[ 4686.535449]  0000000000000001 0000000000000000 ffffc9000343bb48 ffffffff815acd5d
[ 4686.535449] Call Trace:
[ 4686.535450]  [<ffffffff815acd5d>] schedule+0x3d/0x90
[ 4686.535451]  [<ffffffff815af800>] schedule_timeout+0x190/0x210
[ 4686.535453]  [<ffffffff812df054>] ? __pax_list_add+0x24/0x50
[ 4686.535454]  [<ffffffff815ae808>] __down+0x68/0xb0
[ 4686.535455]  [<ffffffff810b3b2f>] down+0x3f/0x50
[ 4686.535456]  [<ffffffff810b8dd0>] console_lock+0x10/0x30
[ 4686.535457]  [<ffffffff813b7336>] con_flush_chars+0x26/0x50
[ 4686.535459]  [<ffffffff813a149b>] n_tty_receive_buf_common+0x64b/0xaf0
[ 4686.535460]  [<ffffffff8109c94b>] ? set_next_entity+0xab/0x460
[ 4686.535461]  [<ffffffff813a196e>] n_tty_receive_buf2+0x2e/0x40
[ 4686.535462]  [<ffffffff813a643a>] flush_to_ldisc+0xfa/0x1a0
[ 4686.535463]  [<ffffffff810850c5>] process_one_work+0x125/0x370
[ 4686.535465]  [<ffffffff8108535c>] worker_thread+0x4c/0x460
[ 4686.535465]  [<ffffffff81085310>] ? process_one_work+0x370/0x370
[ 4686.535466]  [<ffffffff81085310>] ? process_one_work+0x370/0x370
[ 4686.535467]  [<ffffffff8108b2d7>] kthread+0xd7/0xf0
[ 4686.535468]  [<ffffffff8108b200>] ? kthread_worker_fn+0x160/0x160
[ 4686.535469]  [<ffffffff815b0ffe>] ret_from_fork+0x3e/0x70
[ 4686.535470]  [<ffffffff8108b200>] ? kthread_worker_fn+0x160/0x160
[ 4706.863081] synergys[6903]: segfault at 0 ip 0000000000455960 sp 00007e2d1cfc3a70 error 4 in synergys[400000+de000]
[ 4706.863090] grsec: Segmentation fault occurred at            (nil) in /usr/lib/synergy/synergys[synergys:6903] uid/euid:1000/1000 gid/egid:100/100, parent /usr/bin/bash[bash:6896] uid/euid:1000/1000 gid/egid:100/100


[peetaur]

and just as I posted that I thought you'd probably want me to test vanilla 4.2.6, right...?

[PaX Team]

thanks, can you try the following patch:

Code: Select all

--- a/arch/x86/include/asm/uaccess.h      2015-11-28 02:11:28.429618495 +0100
+++ b/arch/x86/include/asm/uaccess.h      2015-12-05 02:27:51.893199069 +0100
@@ -459,7 +459,7 @@
 #define __get_user_nocheck(x, ptr, size)                               \
 ({                                                                     \
        int __gu_err;                                                   \
-       unsigned long __gu_val;                                         \
+       __inttype(*(ptr)) __gu_val;                                     \
        __get_user_size(__gu_val, (ptr), (size), __gu_err, -EFAULT);    \
        (x) = (__typeof__(*(ptr)))__gu_val;                             \
        __gu_err;                                                       \

[peetaur]

Thanks, I'm building now and will test soon.

[peetaur]

Still fails. (this time tested in a virtual machine, also with radeon with a dedicated graphics card passed through).

By the way, I am using gpm, which is for mouse support in text ttys, and 'highlights' text if you click and drag, and also the same highlighting effect for the mouse cursor, and that's what I read "set_selection" does, and gpn appears in the trace "Comm: gpm". I don't have to click and drag for it to hang.

Code: Select all

[ 2222.389996] PAX: size overflow detected in function set_selection drivers/tty/vt/selection.c:174 cicus.218_444 max, count: 45, decl: sel_end; num: 0; context: vardecl_selection.c;
[ 2222.389999] CPU: 0 PID: 1556 Comm: gpm Not tainted 4.2.6-1-grsec-kvm-host #31
[ 2222.390000] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.8.2-20150617_082717-anatol 04/01/2014
[ 2222.390003]  ffffffff81884883 309ba678cd6ce0dd 0000000000000000 ffffffff817ae13e
[ 2222.390005]  ffffc9000657bba8 ffffffff815aae2c 0000000000000000 ffffffff817ae130
[ 2222.390007]  ffffc9000657bbd8 ffffffff811a73b4 000078a844260073 ffff88015b808800
[ 2222.390008] Call Trace:
[ 2222.390014]  [<ffffffff815aae2c>] dump_stack+0x4c/0x79
[ 2222.390017]  [<ffffffff811a73b4>] report_size_overflow+0x34/0x40
[ 2222.390020]  [<ffffffff813ae6d7>] set_selection+0x977/0xbe0
[ 2222.390022]  [<ffffffff813b7a45>] tioclinux+0x85/0x230
[ 2222.390025]  [<ffffffff81072ae9>] ? capable+0x19/0x20
[ 2222.390027]  [<ffffffff813aa29b>] vt_ioctl+0x1bb/0x1ca0
[ 2222.390029]  [<ffffffff8139c4ca>] tty_ioctl+0x45a/0xe70
[ 2222.390032]  [<ffffffff811b6280>] do_vfs_ioctl+0x3f0/0x770
[ 2222.390033]  [<ffffffff811b6677>] SyS_ioctl+0x77/0x90
[ 2222.390036]  [<ffffffff815b06eb>] entry_SYSCALL_64_fastpath+0x12/0x85


[ephox]

Could you please send me your kernel .config?

[peetaur]

@ephox, sure, here it is

[ephox]

I can't reproduce the problem. Could you please send me the results (drivers/tty/vt/selection.* and arch/x86/include/asm/uaccess.h) of make drivers/tty/vt/selection.o EXTRA_CFLAGS="-fdump-tree-all -fdump-ipa-all"?

[peetaur]

This is with your patch applied still.

Code: Select all

make drivers/tty/vt/selection.o EXTRA_CFLAGS="-fdump-tree-all -fdump-ipa-all"

Code: Select all

  CHK     include/config/kernel.release
  UPD     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  UPD     include/generated/utsrelease.h
  CC      kernel/bounds.s
  CHK     include/generated/bounds.h
  CHK     include/generated/timeconst.h
  CC      arch/x86/kernel/asm-offsets.s
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CC      scripts/mod/empty.o
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  CC      scripts/mod/devicetable-offsets.s
  GEN     scripts/mod/devicetable-offsets.h
  HOSTCC  scripts/mod/file2alias.o
scripts/mod/file2alias.c: In function ‘do_pci_entry’:
scripts/mod/file2alias.c:432:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  ADD(alias, "v", vendor != PCI_ANY_ID, vendor);
                         ^
scripts/mod/file2alias.c:118:13: note: in definition of macro ‘ADD’
         if (cond)                                               \
             ^
scripts/mod/file2alias.c:433:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  ADD(alias, "d", device != PCI_ANY_ID, device);
                         ^
scripts/mod/file2alias.c:118:13: note: in definition of macro ‘ADD’
         if (cond)                                               \
             ^
scripts/mod/file2alias.c:434:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  ADD(alias, "sv", subvendor != PCI_ANY_ID, subvendor);
                             ^
scripts/mod/file2alias.c:118:13: note: in definition of macro ‘ADD’
         if (cond)                                               \
             ^
scripts/mod/file2alias.c:435:29: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  ADD(alias, "sd", subdevice != PCI_ANY_ID, subdevice);
                             ^
scripts/mod/file2alias.c:118:13: note: in definition of macro ‘ADD’
         if (cond)                                               \
             ^
scripts/mod/file2alias.c: In function ‘do_vmbus_entry’:
scripts/mod/file2alias.c:917:16: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  for (i = 0; i < (sizeof(*guid) * 2); i += 2)
                ^
scripts/mod/file2alias.c: In function ‘do_ipack_entry’:
scripts/mod/file2alias.c:1074:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  ADD(alias, "v", vendor != IPACK_ANY_ID, vendor);
                         ^
scripts/mod/file2alias.c:118:13: note: in definition of macro ‘ADD’
         if (cond)                                               \
             ^
scripts/mod/file2alias.c:1075:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
  ADD(alias, "d", device != IPACK_ANY_ID, device);
                         ^
scripts/mod/file2alias.c:118:13: note: in definition of macro ‘ADD’
         if (cond)                                               \
             ^
  HOSTCC  scripts/mod/sumversion.o
  HOSTLD  scripts/mod/modpost
  CC      drivers/tty/vt/selection.o

Do you think maybe my sysctls are relevant? Do you want those?

Code: Select all

kernel.grsecurity.chroot_caps=0
kernel.grsecurity.chroot_deny_chmod=0
kernel.grsecurity.chroot_deny_chroot=1
kernel.grsecurity.chroot_deny_fchdir=1
kernel.grsecurity.chroot_deny_mknod=1
kernel.grsecurity.chroot_deny_mount=1
kernel.grsecurity.chroot_deny_pivot=1
kernel.grsecurity.chroot_deny_shmat=1
kernel.grsecurity.chroot_deny_sysctl=1
kernel.grsecurity.chroot_deny_unix=1
kernel.grsecurity.chroot_enforce_chdir=1
kernel.grsecurity.chroot_execlog=0
kernel.grsecurity.chroot_findtask=1
kernel.grsecurity.chroot_restrict_nice=1
kernel.grsecurity.audit_chdir=0
kernel.grsecurity.audit_mount=0
kernel.grsecurity.consistent_setxid=1
kernel.grsecurity.deny_new_usb=0
kernel.grsecurity.deter_bruteforce=1
kernel.grsecurity.dmesg=1
kernel.grsecurity.fifo_restrictions=1
kernel.grsecurity.harden_ptrace=1
kernel.grsecurity.ip_blackhole=1
kernel.grsecurity.lastack_retries=4
kernel.grsecurity.linking_restrictions=1
kernel.grsecurity.ptrace_readexec=1
kernel.grsecurity.romount_protect=0
kernel.grsecurity.harden_ipc=0
kernel.grsecurity.exec_logging=0
kernel.grsecurity.resource_logging=1
kernel.grsecurity.rwxmap_logging=1
kernel.grsecurity.signal_logging=1
kernel.grsecurity.timechange_logging=1
kernel.grsecurity.tpe=0
kernel.grsecurity.tpe_gid=778
kernel.grsecurity.tpe_invert=1
kernel.grsecurity.tpe_restrict_all=1
kernel.pax.softmode=0


And did you try with gpm? Start gpm and Xorg on boot, and then switch to a text tty and maybe move the mouse, and it seems to reliably hang. (sometimes it didn't, but I think then I didn't move the mouse)

[ephox]

Thanks for the report, it will be fixed in the next grsec patch so I don't need the drivers/tty/vt/selection.* and arch/x86/include/asm/uaccess.h files