grsecurity-3.1-4.3.3-201601051958 fails to boot on arm

[fabled]

grsecurity-3.1-4.3.3-201601051958 built for arm does not seem to boot.

Tested using on qemu/vexpress and wandboard (real hardware).

The regular generic build does not print anything. But with lowlevel debugging and earlyprintk I get the following:

Code: Select all

Uncompressing Linux... done, booting the kernel.
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.3.3-0-grsec (tteras@ttdev-edge-armhf) (gcc version 5.3.0 (Alpine 5.3.0) ) #1-Alpine SMP Tue Jan 12 10:01:35 GMT 2016
[    0.000000] CPU: ARMv7 Processor [412fc0f1] revision 1 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[    0.000000] Machine model: V2P-CA15
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] PERCPU: Embedded 11 pages/cpu @eeed4000 s15744 r8192 d21120 u45056
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 260434
[    0.000000] Kernel command line: earlyprintk console=ttyAMA0 secure=off
[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Memory: 1026288K/1048576K available (3621K kernel code, 422K rwdata, 1700K rodata, 656K init, 334K bss, 22288K reserved, 0K cma-reserved, 270336K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xf0000000 - 0xff000000   ( 240 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xef800000   ( 760 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0591a04   (3623 kB)
[    0.000000]       .init : 0xc0760000 - 0xc0804000   ( 656 kB)
[    0.000000]       .data : 0xc0804000 - 0xc086da40   ( 423 kB)
[    0.000000]        .bss : 0xc0870000 - 0xc08c38b4   ( 335 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]    Build-time adjustment of leaf fanout to 32.
[    0.000000]    RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=2
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] L2C: failed to init: -19
[    0.000000] Architected cp15 timer(s) running at 62.50MHz (virt).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
[    0.000172] sched_clock: 56 bits at 62MHz, resolution 16ns, wraps every 4398046511096ns
[    0.000730] Switching to timer-based delay loop, resolution 16ns
[    0.005132] clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns


After the last log message the system just hangs, and nothing happens after that.
Similar config on vanilla kernel works.

Any suggestions how to debug further?

[PaX Team]

can you use qemu's debugging facilities (gdb and/or logging) to see what goes wrong? presumably there's some early unexpected exception, should be easy to see what triggers it.

[fabled]

It seems to hang with:

Code: Select all

(gdb) where
#0  0xffff000c in ?? ()
#1  0xffff0010 in ?? ()

(gdb) info registers
r0             0xc0870180   -1064894080
r1             0x0   0
r2             0xc0804000   -1065336832
r3             0x0   0
r4             0x0   0
r5             0xc0870000   -1064894464
r6             0xffffffff   -1
r7             0xc0806480   -1065327488
r8             0xef7fedc0   -276828736
r9             0xc07b8e38   -1065644488
r10            0x0   0
r11            0x0   0
r12            0x0   0
sp             0xc087018c   0xc087018c <stacks+12>
lr             0xffff0010   -65520
pc             0xffff000c   0xffff000c
cpsr           0x200001d7   536871383


Based on some stepping at least clocksource_of_init() seems to have completed successfully.

[fabled]

Based on some stepping at least clocksource_of_init() seems to have completed successfully.

I single stepped this and it seems to hang at local_irq_enable() on instruction "cpsie i". So basically when interrupts get enabled. qemu does not really show me which interrupt is triggered.

[PaX Team]

it looks like the vector page, i guess DOMAIN_VECTORS isn't defined correctly in one of the configurations.

[fabled]

it looks like the vector page, i guess DOMAIN_VECTORS isn't defined correctly in one of the configurations.

Do you need any additional information?
I also have CONFIG_VDSO=y which is relatively new feature for ARM.

[spender]

Can you post your full kernel .config somewhere?

-Brad

[fabled]

Can you post your full kernel .config somewhere?

http://dev.alpinelinux.org/~tteras/grse ... mhf-config

Do note that to get the boot message earlyprintk support is turned on, so the kernel with this config works only on vexpress (I used it on Qemu).

[spender]

Can you disable CONFIG_CPU_SW_DOMAIN_PAN? You already have UDEREF enabled.

Thanks,
-Brad

[fabled]

Can you disable CONFIG_CPU_SW_DOMAIN_PAN? You already have UDEREF enabled.

No. It's not editable in menuconfig.

Selected by: PAX_MEMORY_UDEREF [=y] && GRKERNSEC [=y] && (X86 || ARM [=y] && (CPU_V6 [=n] || CPU_V6K [=n] || CPU_V7 [=y]) && !ARM_LPAE [=n]) && !UML_X86 && !XEN [=n] && ARM [=y]

[PaX Team]

you can remove "select CPU_SW_DOMAIN_PAN if ARM" from security/Kconfig to test.

[fabled]

Based on dmesg output it still hangs in similar manner even with CONFIG_CPU_SW_DOMAIN_PAN=n.

[fabled]

Seems the latest patch (grsecurity-3.1-4.3.5-201602092235) fixes the above hanging issue.

However, now it crashes with the following error:

Code: Select all

[    2.385953] Freeing unused kernel memory: 660K (c0771000 - c0816000)
[    2.413427] grsec: Segmentation fault occurred at 0000001c in /init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper/0:0] uid/euid:0/0 gid/egid:0/0
[    2.415419] grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper/0:0] uid/euid:0/0 gid/egid:0/0
[    2.420638] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    2.420638]
[    2.421115] CPU: 0 PID: 1 Comm: init Not tainted 4.3.5-0-grsec #1-Alpine
[    2.421347] Hardware name: ARM-Versatile Express
[    2.422846] [<c02180d8>] (unwind_backtrace+0x0/0xe0) from [<c02144c0>] (show_stack+0x10/0x14)
[    2.423428] [<c02144c0>] (show_stack+0x10/0x14) from [<c03c95f0>] (dump_stack+0x74/0x90)
[    2.423891] [<c03c95f0>] (dump_stack+0x74/0x90) from [<c02cca10>] (panic+0x84/0x1e0)
[    2.424365] [<c02cca10>] (panic+0x84/0x1e0) from [<c023e714>] (do_exit+0x51c/0x914)
[    2.424728] [<c023e714>] (do_exit+0x51c/0x914) from [<c023eb80>] (do_group_exit+0x48/0xcc)
[    2.425072] [<c023eb80>] (do_group_exit+0x48/0xcc) from [<c0249b0c>] (get_signal+0x4d8/0x53c)
[    2.425434] [<c0249b0c>] (get_signal+0x4d8/0x53c) from [<c0213b40>] (do_signal+0x8c/0x4bc)
[    2.425785] [<c0213b40>] (do_signal+0x8c/0x4bc) from [<c02140d0>] (do_work_pending+0x54/0xa4)
[    2.426143] [<c02140d0>] (do_work_pending+0x54/0xa4) from [<c020f6c0>] (slow_work_pending+0xc/0x20)
[    2.426793] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    2.426793]


This happens with CONFIG_VDSO=y. Compiling with CONFIG_VDSO=n things seems to work again.