XEN PVHVM + PAX_MEMORY_UDEREF + PAX_KERNEXEC

[jorgus]

Hi,

There is an old topic regarding XEN and PAX (viewtopic.php?f=3&t=2954&hilit=plugin) but it's not exactly what I'm trying to do (I'm not talking dom0 but domU), so forgive me opening a new one. What I'm trying to do is run 3.2.33 with grsecurity patch and CONFIG_PAX_MEMORY_UDEREF + CONFIG_PAX_KERNEXEC options set (among many others) in a XEN HVM domU with PV drivers, also known as PVHVM. There is no way to enable paravirtualised XEN drivers without enabling CONFIG_XEN, which disables CONFIG_PAX_MEMORY_UDEREF and CONFIG_PAX_KERNEXEC. However, after a little make-up to security/Kconfig (removing !XEN from option dependencies) and enabling the two aforementioned options I successfully booted the resulting kernel (built for AMD64 architecture).

We noticed some slowdown in tests, but it seems to be working. Has anyone from the PAX team tested this in such setup? Indeed XEN "pure PV" domains do not boot with either of these options set, but HVM on a VT-enabled CPU is a different story. As there is no point in using "pure HVM" domains with ever so slow emulated disk, network etc., CONFIG_XEN needs to be enabled for an optimised PVHVM kernel.

P.S.
How much (if any) of PAX_KERNEXEC works for AMD64 when compiling with gcc 4.4 with no plugin support? Obviously it does something, because "pure PV" domain does not boot with this option set, but "select PAX_KERNEXEC_PLUGIN if X86_64" under PAX_KERNEXEC makes me think. I will be migrating to gcc 4.7 soon, but I would be grateful if someone clarified this.

[PaX Team]

However, after a little make-up to security/Kconfig (removing !XEN from option dependencies) and enabling the two aforementioned options I successfully booted the resulting kernel (built for AMD64 architecture).

We noticed some slowdown in tests, but it seems to be working. Has anyone from the PAX team tested this in such setup? Indeed XEN "pure PV" domains do not boot with either of these options set, but HVM on a VT-enabled CPU is a different story. As there is no point in using "pure HVM" domains with ever so slow emulated disk, network etc., CONFIG_XEN needs to be enabled for an optimised PVHVM kernel.

the short answer is that Xen has never really been on our radar, so whatever users try and report back on is what we'll rely on for further support. in the past i made some judgement calls when i knew that by design a certain feature wouldn't work with Xen/PV and debugged some early boot problems with help from users but beyond that no testing/experimentation happened. now with your input i could relax the dependence on !XEN but first i need to know how to tell PVHVM from pure PV Xen apart . and it has to be a compile time decision, as at least KERNEXEC does things (CR0.WP flipping) that PV Xen will never support. i'll also have tell the difference between dom0/domU because the former is always PV if i'm not mistaken.

How much (if any) of PAX_KERNEXEC works for AMD64 when compiling with gcc 4.4 with no plugin support?

everything works as it should since KERNEXEC predates plugin support by many years . what you will miss is some extra security where the KERNEXEC plugin instruments function pointer dereferences to prevent trivial hijacking attacks (but it's not bulletproof at all, especially if you have a powerful enough write bug). so if you want a justification for moving to a newer compiler, i'd say that the other plugins are much more important .

Obviously it does something, because "pure PV" domain does not boot with this option set, but "select PAX_KERNEXEC_PLUGIN if X86_64" under PAX_KERNEXEC makes me think.

pure PV fails due to the CR0.WP manipulation mentioned above as, by design, Xen doesn't allow it because it doesn't/cannot let a guest kernel directly modify page tables (coincidentally, some of the KERNEXEC code would do just that, but the more general issue is the general lack of CR0.WP support in PV domains).

[jorgus]

Hi PaX Team,

I'm afraid you cannot tell whether the kernel being compiled is going to be used as PV or PVHVM (the drivers are the same) or even dom0 (as I said, CONFIG_XEN_DOM0 is automatically selected by CONFIG_XEN for unknown reason). I don't mind tweaking the grsecurity patch but maybe for the rest of the world it would be better just to warn users in option descriptions instead of disabling them. Anyway I think that PV is going to die. I have been an old-time PV lover since the times there was no Intel VT until today. After making some crude performance tests I found out that indeed PVHVM works faster than PV on modern processors with features such as VT-x, EPT etc. We are talking amd64 architecture - on i386 PV may defend itself more.

By the way is it normal that KERNEXEC alone has negilble impact on PVHVM performance, UDEREF has moderate impact but the two options together result in some kind of synergy and the impact is higher than it would be expected from simply adding the overhead times? Still talking amd64. Here are some examples of this crude test which was a full build of a custom 3.2.33 grsec-enabled kernel (consisting of removing old source tree, unpacking, patching and compiling using 2 cores). The overhead of KERNEXEC alone seems to be a very slight percentage, UDEREF alone adds some 11-12% but the two together result in nearly 30% overhead.


PVHVM, PAX_MEMORY_UDEREF=n, CONFIG_PAX_KERNEXEC=n
real 7m7.691s
user 11m6.578s
sys 1m6.344s


PVHVM, PAX_MEMORY_UDEREF=n, CONFIG_PAX_KERNEXEC=y
real 7m12.977s
user 11m4.722s
sys 1m11.440s

real 7m11.884s
user 11m6.410s
sys 1m11.700s


PVHVM, PAX_MEMORY_UDEREF=y, CONFIG_PAX_KERNEXEC=n
real 7m55.779s
user 12m39.071s
sys 1m13.305s


PVHVM, PAX_MEMORY_UDEREF=y, CONFIG_PAX_KERNEXEC=y
real 9m3.745s
user 13m19.322s
sys 2m14.356s

real 9m3.255s
user 13m22.230s
sys 2m13.876s


By the way the test on PV obviously with PAX_MEMORY_UDEREF=n, CONFIG_PAX_KERNEXEC=n was

real 8m27.569s
user 11m39.548s
sys 3m0.483s

real 8m31.056s
user 11m42.328s
sys 3m2.295s

real 8m28.681s
user 11m41.000s
sys 3m0.359s

which proves its obsolescence, at least in kernel compilation world