kernel panic - grsecurity-3.1-4.5.2-201604290633

[audiocricket]

Hello there,

just wanted to try RAP-enabled module-less kernel with grsecurity-3.1-4.5.2-201604290633 test patch, however the kernel panics.

It's a DO-based KVM VPS and I've recorded the boot process, but it panics too fast that I think I have only a portion of the panic message:



This kernel config for 4.5.2 is 99.9% the same as the previous perfectly stable working 4.4.8 kernel (grsecurity-3.1-4.4.8-201604252206.patch),
only PARAVIRT options are completely turned off and PAX_RAP is turned on here. Also to mention, the 4.5.2 kernel has ZFS/SPL patches to be
able to use ZFSonLinux.

Maybe the spl_vmem_fini on the screenshot may have something to with SPL/ZFS stuff? But it works with 3.1-4.4.8-201604252206 ...

How can I be helpful in debugging this? What more info do you need? I can PM you my 4.5.2 kernel config and other relevant info you may need.

Many thanks for any reply!

Cheers,
Audiocricket

[PaX Team]

Also to mention, the 4.5.2 kernel has ZFS/SPL patches to be able to use ZFSonLinux.

Maybe the spl_vmem_fini on the screenshot may have something to with SPL/ZFS stuff?

it's not a maybe, it's pretty much a certainty . RAP detects mismatched types on indirect function calls where the function pointer doesn't exactly match the type of the target function. these are all violations of the C standard and must be fixed before enabling RAP on a codebase. i did that work for linux itself (the size increase of the patch is mostly due to that) and someone else will have to do the same work for every out-of-tree module.

[audiocricket]

Thanks for the reply.

I'd pretty happy to help and make this work correctly, do you have any suggestion/hint where to start?

[PaX Team]

it's basically a two step process. first you have to find all the bad function pointer casts, second you have to fix them properly (removing the cast itself won't be enough, programmers use that only to shut up the compiler after all). the first step can be mostly automated (not part of the public RAP offering) or done by manual auditing (analysing RAP reports from runtime detection can help pinpoint bad code constructs, e.g., it often happens that RAP reports a problem on a function pointer loaded from an ops structure and by looking at the initializer of the given ops structure one finds even more bad casts, just look at what i had to fix in the linux NFS code for an example ).

fixing a bad fptr cast involves finding out what type is used on the indirect call and the types of all possible target functions. once you have that data you'll have to settle on a common type (one typical case i ran across a lot is that the fptr passes a void* whereas the target functions take a specialized pointer) and fix up all the target functions accordingly. again looking at some of the fixes i made in linux would probably help explain best what is needed to do here.

there's also some finesse possible in choosing the common type, this is what i referred to as 'type diversification' in my H2HC slides. going with the void* vs. specialized pointer example, the obvious and less tedious way of fixing it is to choose the void* as the common type for the given function parameter and do a type cast to the specialized pointer inside each target function. the better way (and i'm guilty of not doing it myself, but i'd already spent 2 months just to fix up linux the lazy way) is to create a new union type with all the possible pointer types as fields in it and choose the union type as the common type, not void*. this will in turn improve the diversity of the type hash at no expense of performance and perhaps even provide better documentation for the given function pointer and its target functions.

another, much less likely cause for a type hash mismatch is when you have indirectly called functions implemented in assembly (e.g., the linux arch specific crypto code). in such cases you'll have to modify the prologue of functions to emit the type hash before the first instruction (again, you'll find examples in PaX).