2.6.37.3 crashes when btrfs is used for root partition

[nannou9]

Well i am not a kernel developer and i've never been analyzing kernel code (with some exceptions like webcam drivers) but i have found that there is no config GRKERNSEC_RBAC_DEBUG in Kconfig file. Is this the reason of the problem i have faced?

[nannou9]

I believe this will fix the problem with enabling the debug option

===============
--- grsecurity/Kconfig 2011-03-21 22:39:10.000000000 +0100
+++ grsecurity/Kconfig 2011-03-21 22:35:42.000000000 +0100
@@ -1012,6 +1012,11 @@
many of your logs are being interpreted as flooding, you may want to
raise this value.

+config GRKERNSEC_RBAC_DEBUG
+ bool "Debugging RBAC System (EXPERIMENTAL)"
+ help
+ Yet another DEBUG option for RBAC system.
+
endmenu

endmenu

===============

[spender]

In grsecurity/Kconfig, below:

Code: Select all

menu "Grsecurity"

can you add:

Code: Select all

config GRKERNSEC_RBAC_DEBUG
         bool


and then try compiling with the modified config?

-Brad

[nannou9]

I already did it. I have even posted a diff in my previous post

Here you are current oops:

Mar 21 22:02:49 localhost kernel: [ 86.947291] Obtained real root device=14, inode=256
Mar 21 22:02:49 localhost kernel: [ 86.947295] Copying special role admin
Mar 21 22:02:49 localhost kernel: [ 86.947424] ------------[ cut here ]------------
Mar 21 22:02:49 localhost kernel: [ 86.947562] kernel BUG at grsecurity/gracl.c:1844!
Mar 21 22:02:49 localhost kernel: [ 86.947563] invalid opcode: 0000 [#1] SMP
Mar 21 22:02:49 localhost kernel: [ 86.947565] last sysfs file: /sys/devices/pci0000:00/0000:00:1f.2/host0/target0:0:0/0:0:0:0/block/sda/dev
Mar 21 22:02:49 localhost kernel: [ 86.947567] CPU 2
Mar 21 22:02:49 localhost kernel: [ 86.947568] Modules linked in: fuse xfs exportfs jfs ext4 jbd2 ext2 scsi_wait_scan
Mar 21 22:02:49 localhost kernel: [ 86.947572]
Mar 21 22:02:49 localhost kernel: [ 86.947574] Pid: 3816, comm: gradm Not tainted 2.6.37.4-grsec #1
Mar 21 22:02:49 localhost kernel: [ 86.947575] System manufacturer System Product Name/P7P55 LX
Mar 21 22:02:49 localhost kernel: [ 86.947577] RIP: 0010:[<ffffffff8126fa2b>] [<ffffffff8126fa2b>] __chk_obj_label+0x11e/0x167
Mar 21 22:02:49 localhost kernel: [ 86.947582] RSP: 0018:ffff88021b961d58 EFLAGS: 00010246
Mar 21 22:02:49 localhost kernel: [ 86.947584] RAX: 0000000000000000 RBX: ffff88021d4bf240 RCX: 000000000000003d
Mar 21 22:02:49 localhost kernel: [ 86.947585] RDX: ffff88021e9bd200 RSI: 000000000000000e RDI: 000000000000010e
Mar 21 22:02:49 localhost kernel: [ 86.947587] RBP: ffff88021b961da8 R08: ffff88021b961d70 R09: 0000000000000001
Mar 21 22:02:49 localhost kernel: [ 86.947588] R10: 0000000000000002 R11: 00000000000000d0 R12: ffff88021b961d70
Mar 21 22:02:49 localhost kernel: [ 86.947590] R13: ffff88021fc4a800 R14: ffff88021d4bf0c0 R15: 0000000000000001
Mar 21 22:02:49 localhost kernel: [ 86.947592] FS: 00006b00da11b700(0000) GS:ffff8800df480000(0000) knlGS:0000000000000000
Mar 21 22:02:49 localhost kernel: [ 86.947594] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Mar 21 22:02:49 localhost kernel: [ 86.947595] CR2: 00006b00da121000 CR3: 000000021ece9000 CR4: 00000000000006f0
Mar 21 22:02:49 localhost kernel: [ 86.947597] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 21 22:02:49 localhost kernel: [ 86.947598] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Mar 21 22:02:49 localhost kernel: [ 86.947600] Process gradm (pid: 3816, threadinfo ffff88021b960000, task ffff88021e8742f0)
Mar 21 22:02:49 localhost kernel: [ 86.947601] Stack:
Mar 21 22:02:49 localhost kernel: [ 86.947602] ffff88021fc4a800 0000000000000000 ffff88021dcdbc00 0000000000000000
Mar 21 22:02:49 localhost kernel: [ 86.947604] ffff88021b961d88 ffff88021fc80000 ffff88021b961de0 ffff88021fc80000
Mar 21 22:02:49 localhost kernel: [ 86.947606] 0000000000000000 0000000000000000 ffff88021b961db8 ffffffff8126fa85
Mar 21 22:02:49 localhost kernel: [ 86.947609] Call Trace:
Mar 21 22:02:49 localhost kernel: [ 86.947611] [<ffffffff8126fa85>] chk_obj_label+0x11/0x13
Mar 21 22:02:49 localhost kernel: [ 86.947614] [<ffffffff812717fc>] gr_set_acls+0x168/0x315
Mar 21 22:02:49 localhost kernel: [ 86.947616] [<ffffffff812733b1>] gracl_init+0x764/0x7cd
Mar 21 22:02:49 localhost kernel: [ 86.947618] [<ffffffff81273699>] write_grsec_handler+0x27f/0x9a6
Mar 21 22:02:49 localhost kernel: [ 86.947621] [<ffffffff810df9fb>] vfs_write+0x111/0x171
Mar 21 22:02:49 localhost kernel: [ 86.947623] [<ffffffff810dfb11>] sys_write+0x45/0x69
Mar 21 22:02:49 localhost kernel: [ 86.947626] [<ffffffff810029fb>] system_call_fastpath+0x16/0x1b
Mar 21 22:02:49 localhost kernel: [ 86.947628] Code: 86 00 45 89 f9 4d 89 e0 4c 89 ee 4c 89 f7 e8 1c fe ff ff 48 89 45 b8 e8 fe 68 e8 ff fe 05 5e 38 63 00 48 8b 45 b8 48 85 c0 75 04 <0f> 0b eb fe 48 83 c4 28 5b 41 5c 41 5d 41 5e 41 5f c9 c3 4c 3b
Mar 21 22:02:49 localhost kernel: [ 86.947640] RIP [<ffffffff8126fa2b>] __chk_obj_label+0x11e/0x167
Mar 21 22:02:49 localhost kernel: [ 86.947643] RSP <ffff88021b961d58>
Mar 21 22:02:49 localhost kernel: [ 86.947645] ---[ end trace e4d7d7793736bb54 ]---

[spender]

Ok, so now the question is why the device for the / mount for the init running on your system is different than that of other processes running. For gradm, the device number was 15, but it's reporting the one for init as 14. Is there anything unusual about your init/initramfs process? Could I see your /proc/mounts and /proc/1/mounts(strip any sensitive info)? The 14/15 for a device number is strange as well (we'd normally expect something > 0xff, like 0x801 for /dev/sda).

-Brad

[nannou9]

This is output for: /proc/mounts and /proc/1/mounts (which seems to be exactly same)

rootfs / rootfs rw 0 0
/dev/sda7 / btrfs rw,noatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
rc-svcdir /libexec/rc/init.d tmpfs rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,nosuid,relatime,size=10240k,mode=755 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620 0 0
shm /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
/dev/sda6 /boot reiserfs rw,relatime 0 0
/dev/sda5 /home btrfs rw,relatime 0 0
/dev/sda3 /mnt/vbox btrfs rw,relatime 0 0
usbfs /proc/bus/usb usbfs rw,nosuid,noexec,relatime,devgid=85,devmode=664 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0

Unfortunately i am not doing anything unusual. It is even worse as it is happening no matter to kernel config as I've been trying the default (gentoo recommended rescuecd- like) one and my own customized.

So what now?

[spender]

Can you show me a stat of "/libexec/rc/init.d" and "/libexec/rc/init.d/." ?

-Brad

[nannou9]

stat /libexec/rc/init.d
File: `/libexec/rc/init.d'
Size: 340 Blocks: 0 IO Block: 4096 directory
Device: dh/13d Inode: 2695 Links: 14
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-03-23 18:47:34.201010134 +0000
Modify: 2011-03-23 18:46:36.704999999 +0000
Change: 2011-03-23 18:46:36.704999999 +0000
Birth: -

stat /libexec/rc/init.d/.
File: `/libexec/rc/init.d/.'
Size: 340 Blocks: 0 IO Block: 4096 directory
Device: dh/13d Inode: 2695 Links: 14
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2011-03-23 18:47:34.201010134 +0000
Modify: 2011-03-23 18:46:36.704999999 +0000
Change: 2011-03-23 18:46:36.704999999 +0000
Birth: -

[spender]

I think your init may be chrooted or pivoted. I'm working on a new 2.6.38 patch that I'd like you to try when I release it -- it will use the root of the fs namespace for init instead of whatever its current filesystem root is. The fs namespace root should be closer in all cases to the real root, and hopefully will match up in your case.

-Brad

[spender]

I've uploaded the new patch:

http://grsecurity.net/~spender/grsec-2. ... test.patch

-Brad

[am7op5]

I've been experiencing this problem too. Just tested the patch and I still get an oops at grsecurity/gracl.c:1952.

My root filesystem is btrfs on an LVM volume, which is itself inside a dm-crypt encrypted partition.

[spender]

What distro are you running? I'll try to reproduce this.

-Brad

[nannou9]

Funtoo.
If you already have a boot partition with grsec customized kernel, all you have to do is (assuming you already have the newest btrfs-progs):
1. Create btrfs partition (at least 2GB size)
2. Optionally you can create subvolumes, snapshots etc.
3. Downlaod http://ftp.osuosl.org/pub/funtoo/funtoo ... ent.tar.xz (for intel Core2 or better 64bit cpus)
4. Unpack it to your brand new btrfs partition mounted to /mnt (or anywhere else)
5. Chroot to /mnt
6. change root pass by # passwd
7. copy gradm2 source to /usr/src
8. alter your grub.conf (lilo.conf or anything else you have) to boot system from btrfs part.
9. reboot current and boot to your brand new Funtoo
10. compile gradm and start using
11. CRASH

Please note that kernel is crashig with and without subvolumes/snapshots.

[nannou9]

Please also note that i'm not using encrypted fs, nor raid fs, nor lvm fs.

[spender]

am7op5: Did you ever have RBAC working with btrfs before? If so, what was the first patch/kernel version that caused this problem for you?

-Brad