arm boot failure

[fabled]

Hi,

Finally tried booting 3.10.7 with grsecurity-2.9.1-3.10.7-201308171249.patch on a Wandboard Quad core ARM box. I've tested successful boot on it with 3.10.9 kernel and identical config, so seems to be grsec issue.

The log:
[ 0.353900] devtmpfs: unable to create devtmpfs -14

Would seem to imply -EFAULT.

The full dmesg after smp is initialized:
[ 0.341946] SMP: Total of 4 processors activated (6324.22 BogoMIPS).
[ 0.348414] CPU: All CPU(s) started in SVC mode.
[ 0.353900] devtmpfs: unable to create devtmpfs -14
[ 0.388460] Unable to handle kernel NULL pointer dereference at virtual address 00000080
[ 0.396660] pgd = c0004000
[ 0.399460] [00000080] *pgd=00000000
[ 0.403146] Internal error: Oops: 5 [#1] SMP ARM
[ 0.407860] Modules linked in:
[ 0.411027] CPU: 0 PID: 27 Comm: kdevtmpfs Not tainted 3.10.7-grsec #7
[ 0.417660] task: ef0d0540 ti: ef106000 task.ti: ef106000
[ 0.423178] PC is at __queue_work+0x40/0x28c
[ 0.427550] LR is at queue_work_on+0x40/0x4c
[ 0.431924] pc : [<c0033014>] lr : [<c00332a0>] psr: 20000193
[ 0.431924] sp : ef107ed8 ip : 00000000 fp : 00000000
[ 0.443601] r10: 00000004 r9 : ef107f38 r8 : 00000001
[ 0.448925] r7 : 00000004 r6 : c041cf8c r5 : 00000000 r4 : 60000113
[ 0.455557] r3 : 60000193 r2 : c041cf8c r1 : 00000000 r0 : 00000004
[ 0.462188] Flags: nzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
[ 0.469519] Control: 10c53c7d Table: 1000404a DAC: 00000015
[ 0.475365] Process kdevtmpfs (pid: 27, stack limit = 0xef1061c0)
[ 0.481560] Stack: (0xef107ed8 to 0xef108000)
[ 0.486015] 7ec0: 00000004 00000000
[ 0.494304] 7ee0: c041cf8c 60000113 c041cf8c 00000000 00000004 00000001 ef107f38 00000000
[ 0.502590] 7f00: 00000000 c00332a0 00000000 ef109000 00000000 ef055f54 ef106028 c003de14
[ 0.510879] 7f20: ef0d0540 c001ff60 ef055f28 a0000113 c01bdd4c 00050005 00000000 00000000
[ 0.519164] 7f40: ef107f64 ef055f18 00000000 ef055f54 c01bdd4c 00000000 00000000 00000000
[ 0.527451] 7f60: 00000000 c00399b8 9f7eedff 00000000 ffffefff ef055f54 00000000 00000000
[ 0.535737] 7f80: ef107f80 ef107f80 00000001 00010001 ef107f90 ef107f90 ffffffff ef055f18
[ 0.544025] 7fa0: c0039914 00000000 00000000 c0008f18 00000000 00000000 00000000 00000000
[ 0.552308] 7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 0.560596] 7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff ffffbf76
[ 0.568899] [<c0033014>] (__queue_work+0x40/0x28c) from [<c00332a0>] (queue_work_on+0x40/0x4c)
[ 0.577631] [<c00332a0>] (queue_work_on+0x40/0x4c) from [<c003de14>] (free_nsproxy+0xc4/0xdc)
[ 0.586287] [<c003de14>] (free_nsproxy+0xc4/0xdc) from [<c001ff60>] (do_exit+0x414/0x984)
[ 0.594588] [<c001ff60>] (do_exit+0x414/0x984) from [<c00399b8>] (kthread+0xa4/0xac)
[ 0.602453] [<c00399b8>] (kthread+0xa4/0xac) from [<c0008f18>] (ret_from_fork+0x14/0x3c)
[ 0.610655] Code: e59f123c ebffa379 e3a03001 e5c43011 (e5953080)
[ 0.616885] ---[ end trace af5b256dd27bb7ab ]---
[ 0.621601] Fixing recursive fault but reboot is needed!
[ 0.662214] pinctrl core: initialized pinctrl subsystem
[ 0.667872] regulator-dummy: no parameters
[ 0.672552] NET: Registered protocol family 16
[ 0.678780] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.693212] syscon 20c8000.anatop: regmap [mem 0x020c8000-0x020c8fff] registered
[ 0.700990] vdd1p1: 800 <--> 1375 mV at 1100 mV
[ 0.705993] vdd3p0: 2800 <--> 3150 mV at 3000 mV
[ 0.711119] vdd2p5: 2000 <--> 2750 mV at 2400 mV
[ 0.716204] cpu: 725 <--> 1450 mV at 1150 mV
[ 0.720979] vddpu: 725 <--> 1450 mV at 1150 mV
[ 0.725879] vddsoc: 725 <--> 1450 mV at 1200 mV
[ 0.731685] syscon 20e0000.iomuxc-gpr: regmap [mem 0x020e0000-0x020e0037] registered
[ 0.741142] hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
[ 0.749273] hw-breakpoint: maximum watchpoint size is 4 bytes.
[ 0.755854] imx6q-pinctrl 20e0000.iomuxc: initialized IMX pinctrl driver
[ 0.762996] Serial: AMBA PL011 UART driver
[ 0.769831] bio: create slab <bio-0> at 0
[ 0.774948] pps_core: LinuxPPS API ver. 1 registered
[ 0.780036] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.789309] PTP clock support registered
[ 0.794164] Switching to clocksource mxc_timer1
[ 0.799276] Unable to handle kernel NULL pointer dereference at virtual address 00000008
[ 0.807476] pgd = c0004000
[ 0.810305] [00000008] *pgd=00000000
[ 0.813994] Internal error: Oops: 5 [#2] SMP ARM
[ 0.818708] Modules linked in:
[ 0.821872] CPU: 0 PID: 27 Comm: kdevtmpfs Tainted: G D 3.10.7-grsec #7
[ 0.829463] task: ef0d0540 ti: ef106000 task.ti: ef106000
[ 0.834978] PC is at exit_shm+0x8/0x50
[ 0.838838] LR is at do_exit+0x3fc/0x984
[ 0.842863] pc : [<c0127678>] lr : [<c001ff48>] psr: 60000113
[ 0.842863] sp : ef107c90 ip : 00000000 fp : 00000000
[ 0.854541] r10: c0033014 r9 : ef107cb0 r8 : 00000000
[ 0.859866] r7 : ef106020 r6 : ef106000 r5 : 00000000 r4 : ef0d0540
[ 0.866496] r3 : 00000000 r2 : ef0d09e8 r1 : 00000000 r0 : ef0d0540
[ 0.873130] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
[ 0.880369] Control: 10c53c7d Table: 1000404a DAC: 00000015
[ 0.886215] Process kdevtmpfs (pid: 27, stack limit = 0xef1061c0)
[ 0.892411] Stack: (0xef107c90 to 0xef108000)
[ 0.896868] 7c80: 00000000 ef0d0540 00000000 c001ff48
[ 0.905157] 7ca0: 00000001 c0033014 00000000 00070007 c035291a ef107ccc d27bb7ab c0422004
[ 0.913443] 7cc0: 0000000b ef106000 ef107d2a c0406a10 00000001 c0033014 00000000 c000c080
[ 0.921732] 7ce0: ef1061c0 0000000b c0033014 bf000000 00000008 60000193 65000000 31663935
[ 0.930019] 7d00: 20633332 66666265 39373361 61336520 30303330 35652031 30333463 28203131
[ 0.938307] 7d20: 35393565 30383033 00002029 c02a2f64 c034d70b 00000080 00000005 00000000
[ 0.946594] 7d40: ef107e90 00000005 00000000 00000028 00000000 c02a2d20 ef0d0540 c02ab0c4
[ 0.954880] 7d60: ffff8b54 c25d0b00 00000003 c03fc080 00000001 ef107d8c 00000004 c004e798
[ 0.963166] 7d80: 00000000 00000000 0000df8b 00000001 ef09c01c ef0d0540 00000000 c03f7b00
[ 0.971453] 7da0: c0403384 ef106038 ef106000 c040fa50 ef107e5c c02a7dd8 0000005d 00000000
[ 0.979739] 7dc0: 00000000 00000000 151847b5 00000005 00000080 c0407388 ef107e90 00000001
[ 0.988025] 7de0: ef107f38 00000004 00000000 c00083dc c000ac68 00000001 00000000 00000000
[ 0.996310] 7e00: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1.004594] 7e20: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1.012880] 7e40: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1.021166] 7e60: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 c0033014
[ 1.029453] 7e80: 20000193 ffffffff ef107ec4 c02a9700 00000004 00000000 c041cf8c 60000193
[ 1.037740] 7ea0: 60000113 00000000 c041cf8c 00000004 00000001 ef107f38 00000004 00000000
[ 1.046026] 7ec0: 00000000 ef107ed8 c00332a0 c0033014 20000193 ffffffff 00000004 00000000
[ 1.054312] 7ee0: c041cf8c 60000113 c041cf8c 00000000 00000004 00000001 ef107f38 00000000
[ 1.062599] 7f00: 00000000 c00332a0 00000000 ef109000 00000000 ef055f54 ef106028 c003de14
[ 1.070886] 7f20: ef0d0540 c001ff60 ef055f28 a0000113 c01bdd4c 00050005 00000000 00000000
[ 1.079173] 7f40: ef107f64 ef055f18 00000000 ef055f54 c01bdd4c 00000000 00000000 00000000
[ 1.087459] 7f60: 00000000 c00399b8 9f7eedff 00000000 ffffefff ef055f54 00000000 00000000
[ 1.095747] 7f80: ef107f80 ef107f80 00000001 00010001 ef107f90 ef107f90 ffffffff ef055f18
[ 1.104032] 7fa0: c0039914 00000000 00000000 c0008f18 00000000 00000000 00000000 00000000
[ 1.112317] 7fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 1.120602] 7fe0: 00000000 00000000 00000000 00000000 00000013 00000000 ffffffff ffffbf76
[ 1.128906] [<c0127678>] (exit_shm+0x8/0x50) from [<c001ff48>] (do_exit+0x3fc/0x984)
[ 1.136778] [<c001ff48>] (do_exit+0x3fc/0x984) from [<c000c080>] (die+0x2b4/0x3b8)
[ 1.144481] [<c000c080>] (die+0x2b4/0x3b8) from [<c02a2d20>] (__do_kernel_fault.part.10+0x54/0x74)
[ 1.153571] [<c02a2d20>] (__do_kernel_fault.part.10+0x54/0x74) from [<c02ab0c4>] (do_page_fault+0x348/0x3b0)
[ 1.163517] [<c02ab0c4>] (do_page_fault+0x348/0x3b0) from [<c00083dc>] (do_DataAbort+0x4c/0xb0)
[ 1.172333] [<c00083dc>] (do_DataAbort+0x4c/0xb0) from [<c02a9700>] (__dabt_svc+0x40/0x60)
[ 1.180702] Exception stack(0xef107e90 to 0xef107ed8)
[ 1.185855] 7e80: 00000004 00000000 c041cf8c 60000193
[ 1.194143] 7ea0: 60000113 00000000 c041cf8c 00000004 00000001 ef107f38 00000004 00000000
[ 1.202427] 7ec0: 00000000 ef107ed8 c00332a0 c0033014 20000193 ffffffff
[ 1.209162] [<c02a9700>] (__dabt_svc+0x40/0x60) from [<c0033014>] (__queue_work+0x40/0x28c)
[ 1.217631] [<c0033014>] (__queue_work+0x40/0x28c) from [<c00332a0>] (queue_work_on+0x40/0x4c)
[ 1.226363] [<c00332a0>] (queue_work_on+0x40/0x4c) from [<c003de14>] (free_nsproxy+0xc4/0xdc)
[ 1.235005] [<c003de14>] (free_nsproxy+0xc4/0xdc) from [<c001ff60>] (do_exit+0x414/0x984)
[ 1.243303] [<c001ff60>] (do_exit+0x414/0x984) from [<c00399b8>] (kthread+0xa4/0xac)
[ 1.251161] [<c00399b8>] (kthread+0xa4/0xac) from [<c0008f18>] (ret_from_fork+0x14/0x3c)
[ 1.259362] Code: eafc5953 c01274dc e92d4038 e59033b0 (e5934008)
[ 1.265600] ---[ end trace af5b256dd27bb7ac ]---
[ 1.270336] Fixing recursive fault but reboot is needed!

[PaX Team]

assuming you have UDEREF enabled, can you try without it?

[spender]

Can you also try booting without CONFIG_DEVTMPFS? Also please post your .config.

-Brad

[fabled]

assuming you have UDEREF enabled, can you try without it?

No, UDEREF was not enabled. Now that I noticed it, I tried enabling it and got compile error:

Code: Select all

  CC      arch/arm/kernel/suspend.o
In file included from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/idmap.h:5:0,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/kernel/suspend.c:3:
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/pgtable.h: In function 'test_domain':
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/pgtable.h:74:2: error: implicit declaration of function 'current_thread_info' [-Werror=implicit-function-declaration]
  return ((current_thread_info()->cpu_domain) & domain_val(domain, 3)) == domain_val(domain, domaintype);
  ^
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/pgtable.h:74:32: error: invalid type argument of '->' (have 'int')
  return ((current_thread_info()->cpu_domain) & domain_val(domain, 3)) == domain_val(domain, domaintype);
                                ^
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/pgtable.h:74:2: error: implicit declaration of function 'domain_val' [-Werror=implicit-function-declaration]
  return ((current_thread_info()->cpu_domain) & domain_val(domain, 3)) == domain_val(domain, domaintype);
  ^
In file included from /home/tteras/aports/main/linux-grsec/src/linux-3.10/include/linux/thread_info.h:54:0,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/include/linux/preempt.h:9,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/include/linux/spinlock.h:50,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/include/linux/mm_types.h:8,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/include/asm-generic/pgtable.h:7,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/pgtable.h:363,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/idmap.h:5,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/kernel/suspend.c:3:
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/thread_info.h: At top level:
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/thread_info.h:94:35: error: conflicting types for 'current_thread_info'
 static inline struct thread_info *current_thread_info(void) __attribute_const__;
                                   ^
In file included from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/idmap.h:5:0,
                 from /home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/kernel/suspend.c:3:
/home/tteras/aports/main/linux-grsec/src/linux-3.10/arch/arm/include/asm/pgtable.h:74:11: note: previous implicit declaration of 'current_thread_info' was here
  return ((current_thread_info()->cpu_domain) & domain_val(domain, 3)) == domain_val(domain, domaintype);
           ^
cc1: some warnings being treated as errors


Can you also try booting without CONFIG_DEVTMPFS? Also please post your .config.

That makes the original error go away - but now it fails to mount root, which is strange since I use initramfs, and the boot works just fine with vanilla kernel. Perhaps devtmpfs is required to work with initramfs.

dmesg is:
[ 0.496294] Trying to unpack rootfs image as initramfs...
[ 0.958517] Freeing initrd memory: 4744K (c1100000 - c15a2000)
[ 0.964713] hw perfevents: enabled with ARMv7 Cortex-A9 PMU driver, 7 counters available
[ 1.067294] bounce pool size: 64 pages
[ 1.071602] VFS: Disk quotas dquot_6.5.2
[ 1.075849] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
[ 1.082753] msgmni has been set to 1477
[ 1.087821] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
[ 1.095356] io scheduler noop registered
[ 1.099429] io scheduler cfq registered (default)
[ 1.104398] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 1.111625] Serial: AMBA driver
[ 1.114944] Serial: IMX driver
[ 1.118420] 2020000.serial: ttymxc0 at MMIO 0x2020000 (irq = 58) is a IMX
[ 1.125370] console [ttymxc0] enabled, bootconsole disabled
[ 1.125370] console [ttymxc0] enabled, bootconsole disabled
[ 1.143751] brd: module loaded
[ 1.152042] libphy: fec_enet_mii_bus: probed
[ 1.156864] fec 2188000.ethernet eth0: registered PHC device 0
[ 1.163150] cpuidle: using governor ladder
[ 1.167294] cpuidle: using governor menu
[ 1.171522] gre: GRE over IPv4 demultiplexor driver
[ 1.176645] TCP: cubic registered
[ 1.180021] Key type dns_resolver registered
[ 1.184377] VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
[ 1.192331] ThumbEE CPU extension supported.
[ 1.196940] registered taskstats version 1
[ 1.201457] /home/tteras/aports/main/linux-grsec/src/linux-3.10/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
[ 1.212567] Warning: unable to open an initial console.
[ 1.217877] md: Skipping autodetection of RAID arrays. (raid=autodetect will force)
[ 1.225684] List of all partitions:
[ 1.229189] No filesystem could mount root, tried:
[ 1.234121] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)

The config I tried with now is available at: http://dev.alpinelinux.org/~tteras/config-grsec-arm

[spender]

To fix compilation, apply http://grsecurity.net/~spender/arm_comp_fix.diff
It'll be included in the next patch.

Can you boot a kernel with both KERNEXEC and UDEREF enabled?

-Brad

[fabled]

To fix compilation, apply http://grsecurity.net/~spender/arm_comp_fix.diff
It'll be included in the next patch.

Yes, fixes compilation. Thanks.

Can you boot a kernel with both KERNEXEC and UDEREF enabled?

With KERNEXEC and UDEREF and DEVTMPFS enabled it hangs early in the boot. Right after the initial:

Code: Select all

Uncompressing Linux... done, booting the kernel.

Output. I see none of the other early console debug outputs, not even the first "[ 0.000000] Booting Linux on physical CPU 0x0".

[spender]

The same I assume for a kernel with KERNEXEC enabled but UDEREF disabled? I'm trying to establish a baseline here as KERNEXEC/UDEREF works fine on my Arndale.

-Brad

[fabled]

The same I assume for a kernel with KERNEXEC enabled but UDEREF disabled? I'm trying to establish a baseline here as KERNEXEC/UDEREF works fine on my Arndale.

This was the original configuration. Then it boots, but oopses in kdevtmpfs; backtrace in first post.

As you can see from config, I'm compiling a multiplatform kernel, using flattened device tree and initramfs. I wonder if any of these causes the issues as they are data passed from bootloader.

[spender]

The config you linked had UDEREF enabled but KERNEXEC disabled. You're certain the original config had KERNEXEC enabled? Could you try to boot a kernel with UDEREF and KERNEXEC both disabled then? Can you also try a kernel not compiled with CONFIG_ARM_THUMB/CONFIG_ARM_THUMBEE?

-Brad

[N8Fear]

FYI: on my Raspberry Pi KERNEXEC and UDEREF seem to work fine (as in they are enabled and everything's working).