I am on Gentoo Hardened, ARMv7. For the last 6 (or so) major Firefox releases - I think since 38, I have been unable to get Firefox to work on grsecurity on ARM, (even) with PaX entirely disabled. Noteworthy is that I am also unable to get Chromium to work. Firefox (and chromium) are compiled on the device itself, as are the kernels, and I don't run into problems on vanilla linux. So while I cannot fully rule out a bad compiler, it seems unlikely.
On linux-4.5.2-hardened-r1 (again, Hardened Gentoo), Firefox segfaults early on. The problem doesn't show on vanilla 4.5.0.
I have two logs, generated by the following command: strace -f -o firefox-log.txt -- firefox --help
This is the log with Linux 4.5.2 + grsec: https://wizzup.org/firefox-grsec.txt
This is the log with Linux 4.5, vanilla: https://wizzup.org/firefox-vanilla.txt
If you strip off the pid that strace prints, the logs are easily compared with a tool like vimdiff.
Kernel config for Linux 4.5.2 + grsec: https://wizzup.org/grsec-4.5.2-config.txt
I had a brief discussion with pipacs on IRC, where he asked me to debug a few things. I hope I'm allowed to paste the (small) log here.
23:45 <Wizzup> This is most interesting. Even with PAX completely disabled, firefox segfaults on start.
23:45 <pipacs> and works on vanilla?
23:45 <Wizzup> Yes
23:45 <pipacs> interesting
23:45 <Wizzup> It seems chromium has the same problems
23:45 <pipacs> could it be a userland issue, like those syscall stub emulations that we disable?
23:45 <Wizzup> (I know it's not useful debug info): http://sprunge.us/HLNQ
23:46 <pipacs> any kernel logs on it?
23:46 <Wizzup> No kernel logs (could be because pax is disabled)
23:46 <Wizzup> I can turn on ASLR and some other PaX options and boot again, PaX at least gives some error messages.
23:47 <pipacs> try to get a coredump or attach with gdb and see where the sigill is from
23:47 <Wizzup> OK
23:48 <Wizzup> It may require recompiling firefox with debug info.
23:48 <Wizzup> http://sprunge.us/UhbV
23:49 <pipacs> aha can you look at /proc/pid/maps
23:49 <pipacs> and see what's mapped there
23:49 <Wizzup> fair point.
23:49 <pipacs> does x/8i $pc show anything?
23:49 <Wizzup> Seems to be: b3f3a000-b668b000 r-xp 00000000 fe:00 832394 /usr/lib/firefox/libxul.so
23:50 <Wizzup> (gdb) x/8i $pc
23:50 <Wizzup> => 0x30800000:ICannot access memory at address 0x30800000
23:51 <Wizzup> there doesn't seem to be anything mapped at that address, it seems
23:51 <Wizzup> Right, to my paste wrt libxul was stack 1, stack 0 doesn't seem to be mapped at all
23:57 <Wizzup> If I can help debugging this issue, please let me know what I can do
Day changed to 04 May 2016
00:10 <pipacs> well, try to strace it
00:10 <pipacs> and compare the mmap/munmap/mprotect calls in the trace with the faulting address
00:10 <pipacs> (turn off aslr for this)
00:11 <pipacs> maybe you'll see what happened before the failing access to that map
00:11 <Wizzup> I tried a quick strace and saw the last system call before the sigsegv was a futex, but I don't think that was the cause.
00:11 <Wizzup> I'll note to compare the traces. It's getting late here.
00:12 <pipacs> nah, it's something that happens to that mmap
00:12 <pipacs> maybe it's a race or some use-after-free
00:12 <pipacs> but i don't immediately see how any of our features would cause this
00:12 <Wizzup> That is why I also specifically turned off CONFIG_PAX, to further pin down the problem
00:13 <Wizzup> I'll try to get some useful info tomorrow - I rebooted back into vanilla to get a browser
As suggested I made a trace of the system calls that happen on vanilla and on the hardened kernel, but I cannot figure out what the problem might be. Any ideas?
I'm happy to do provide additional information or debug the problem further given some instructions.