grsecurity forums

grsecurity-3.1-4.3.3-201601051958 built for arm does not seem to boot.

Tested using on qemu/vexpress and wandboard (real hardware).

The regular generic build does not print anything. But with lowlevel debugging and earlyprintk I get the following:

Code: Select all

Uncompressing Linux... done, booting the kernel.
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 4.3.3-0-grsec (tteras@ttdev-edge-armhf) (gcc version 5.3.0 (Alpine 5.3.0) ) #1-Alpine SMP Tue Jan 12 10:01:35 GMT 2016
[    0.000000] CPU: ARMv7 Processor [412fc0f1] revision 1 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[    0.000000] Machine model: V2P-CA15
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] PERCPU: Embedded 11 pages/cpu @eeed4000 s15744 r8192 d21120 u45056
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 260434
[    0.000000] Kernel command line: earlyprintk console=ttyAMA0 secure=off
[    0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes)
[    0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
[    0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Memory: 1026288K/1048576K available (3621K kernel code, 422K rwdata, 1700K rodata, 656K init, 334K bss, 22288K reserved, 0K cma-reserved, 270336K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xf0000000 - 0xff000000   ( 240 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xef800000   ( 760 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0591a04   (3623 kB)
[    0.000000]       .init : 0xc0760000 - 0xc0804000   ( 656 kB)
[    0.000000]       .data : 0xc0804000 - 0xc086da40   ( 423 kB)
[    0.000000]        .bss : 0xc0870000 - 0xc08c38b4   ( 335 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000]    Build-time adjustment of leaf fanout to 32.
[    0.000000]    RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=32, nr_cpu_ids=2
[    0.000000] NR_IRQS:16 nr_irqs:16 16
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] GIC CPU mask not found - kernel will fail to boot.
[    0.000000] L2C: failed to init: -19
[    0.000000] Architected cp15 timer(s) running at 62.50MHz (virt).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1cd42e208c, max_idle_ns: 881590405314 ns
[    0.000172] sched_clock: 56 bits at 62MHz, resolution 16ns, wraps every 4398046511096ns
[    0.000730] Switching to timer-based delay loop, resolution 16ns
[    0.005132] clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275 ns


After the last log message the system just hangs, and nothing happens after that.
Similar config on vanilla kernel works.

Any suggestions how to debug further?

can you use qemu's debugging facilities (gdb and/or logging) to see what goes wrong? presumably there's some early unexpected exception, should be easy to see what triggers it.

It seems to hang with:

Code: Select all

(gdb) where
#0  0xffff000c in ?? ()
#1  0xffff0010 in ?? ()

(gdb) info registers
r0             0xc0870180   -1064894080
r1             0x0   0
r2             0xc0804000   -1065336832
r3             0x0   0
r4             0x0   0
r5             0xc0870000   -1064894464
r6             0xffffffff   -1
r7             0xc0806480   -1065327488
r8             0xef7fedc0   -276828736
r9             0xc07b8e38   -1065644488
r10            0x0   0
r11            0x0   0
r12            0x0   0
sp             0xc087018c   0xc087018c <stacks+12>
lr             0xffff0010   -65520
pc             0xffff000c   0xffff000c
cpsr           0x200001d7   536871383


Based on some stepping at least clocksource_of_init() seems to have completed successfully.

fabled wrote:Based on some stepping at least clocksource_of_init() seems to have completed successfully.

I single stepped this and it seems to hang at local_irq_enable() on instruction "cpsie i". So basically when interrupts get enabled. qemu does not really show me which interrupt is triggered.

it looks like the vector page, i guess DOMAIN_VECTORS isn't defined correctly in one of the configurations.

PaX Team wrote:it looks like the vector page, i guess DOMAIN_VECTORS isn't defined correctly in one of the configurations.

Do you need any additional information?
I also have CONFIG_VDSO=y which is relatively new feature for ARM.

Can you post your full kernel .config somewhere?

-Brad

spender wrote:Can you post your full kernel .config somewhere?

http://dev.alpinelinux.org/~tteras/grse ... mhf-config

Do note that to get the boot message earlyprintk support is turned on, so the kernel with this config works only on vexpress (I used it on Qemu).

Can you disable CONFIG_CPU_SW_DOMAIN_PAN? You already have UDEREF enabled.

Thanks,
-Brad

spender wrote:Can you disable CONFIG_CPU_SW_DOMAIN_PAN? You already have UDEREF enabled.

No. It's not editable in menuconfig.

Selected by: PAX_MEMORY_UDEREF [=y] && GRKERNSEC [=y] && (X86 || ARM [=y] && (CPU_V6 [=n] || CPU_V6K [=n] || CPU_V7 [=y]) && !ARM_LPAE [=n]) && !UML_X86 && !XEN [=n] && ARM [=y]

you can remove "select CPU_SW_DOMAIN_PAN if ARM" from security/Kconfig to test.

Based on dmesg output it still hangs in similar manner even with CONFIG_CPU_SW_DOMAIN_PAN=n.

Seems the latest patch (grsecurity-3.1-4.3.5-201602092235) fixes the above hanging issue.

However, now it crashes with the following error:

Code: Select all

[    2.385953] Freeing unused kernel memory: 660K (c0771000 - c0816000)
[    2.413427] grsec: Segmentation fault occurred at 0000001c in /init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper/0:0] uid/euid:0/0 gid/egid:0/0
[    2.415419] grsec: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /init[init:1] uid/euid:0/0 gid/egid:0/0, parent /[swapper/0:0] uid/euid:0/0 gid/egid:0/0
[    2.420638] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    2.420638]
[    2.421115] CPU: 0 PID: 1 Comm: init Not tainted 4.3.5-0-grsec #1-Alpine
[    2.421347] Hardware name: ARM-Versatile Express
[    2.422846] [<c02180d8>] (unwind_backtrace+0x0/0xe0) from [<c02144c0>] (show_stack+0x10/0x14)
[    2.423428] [<c02144c0>] (show_stack+0x10/0x14) from [<c03c95f0>] (dump_stack+0x74/0x90)
[    2.423891] [<c03c95f0>] (dump_stack+0x74/0x90) from [<c02cca10>] (panic+0x84/0x1e0)
[    2.424365] [<c02cca10>] (panic+0x84/0x1e0) from [<c023e714>] (do_exit+0x51c/0x914)
[    2.424728] [<c023e714>] (do_exit+0x51c/0x914) from [<c023eb80>] (do_group_exit+0x48/0xcc)
[    2.425072] [<c023eb80>] (do_group_exit+0x48/0xcc) from [<c0249b0c>] (get_signal+0x4d8/0x53c)
[    2.425434] [<c0249b0c>] (get_signal+0x4d8/0x53c) from [<c0213b40>] (do_signal+0x8c/0x4bc)
[    2.425785] [<c0213b40>] (do_signal+0x8c/0x4bc) from [<c02140d0>] (do_work_pending+0x54/0xa4)
[    2.426143] [<c02140d0>] (do_work_pending+0x54/0xa4) from [<c020f6c0>] (slow_work_pending+0xc/0x20)
[    2.426793] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    2.426793]


This happens with CONFIG_VDSO=y. Compiling with CONFIG_VDSO=n things seems to work again.