xen + pax + 2.6.27

[PaX Team]

What specify info are you after? This is the back trace:

Code: Select all

pin_pagetable_pfn (cmd=2154807176, pfn=4294967274) at arch/x86/xen/enlighten.c:854
854         BUG();
(gdb) bt
#0  pin_pagetable_pfn (cmd=2154807176, pfn=4294967274) at arch/x86/xen/enlighten.c:854
#1  0xffffffff80711d67 in xen_start_kernel () at arch/x86/xen/enlighten.c:1557


i think this is due to another change i made recently, to map up to 4GB initially in the identity map (all these pre-allocated page table games are part of a bigger effort btw) and it seems Xen has its own idea about these early identity mapped page tables. i'll have to dig into that code a bit to see how they can be fixed for Xen, or someone already familiar with Xen could help me out here .

[bplant]

i think this is due to another change i made recently, to map up to 4GB initially in the identity map (all these pre-allocated page table games are part of a bigger effort btw) and it seems Xen has its own idea about these early identity mapped page tables. i'll have to dig into that code a bit to see how they can be fixed for Xen, or someone already familiar with Xen could help me out here .

I noticed that you uploaded some new test patches. I gave #18 a shot, but still no go. Did test patches #16-18 contain any fixes for booting a xen kernel?

[PaX Team]

I noticed that you uploaded some new test patches. I gave #18 a shot, but still no go. Did test patches #16-18 contain any fixes for booting a xen kernel?

i only fixed kvm/KERNEXEC/i386 there, no time for Xen yet. i'd really appreciate if someone looked at the dom0 side and told me what exactly it expects in domU page tables...

[bplant]

i'd really appreciate if someone looked at the dom0 side and told me what exactly it expects in domU page tables...

Unfortunately kernel programming is not my forte. I'm more than happy to test your patches in my dev xen environment though.

What specify info are you after? This is the back trace:

Code: Select all

pin_pagetable_pfn (cmd=2154807176, pfn=4294967274) at arch/x86/xen/enlighten.c:854
854         BUG();
(gdb) bt
#0  pin_pagetable_pfn (cmd=2154807176, pfn=4294967274) at arch/x86/xen/enlighten.c:854
#1  0xffffffff80711d67 in xen_start_kernel () at arch/x86/xen/enlighten.c:1557


i think this is due to another change i made recently, to map up to 4GB initially in the identity map (all these pre-allocated page table games are part of a bigger effort btw) and it seems Xen has its own idea about these early identity mapped page tables. i'll have to dig into that code a bit to see how they can be fixed for Xen, or someone already familiar with Xen could help me out here .

Care to share (or point me to another post of) what the bigger effort is and what the initial identity map is? I'm curious as to what it is and why you want to map up to 4GB initially as apposed to however much you were mapping previously.

[PaX Team]

Care to share (or point me to another post of) what the bigger effort is and what the initial identity map is? I'm curious as to what it is and why you want to map up to 4GB initially as apposed to however much you were mapping previously.

the bigger effort is kernel self-protection that involves a lot of things, here i was meaning the protection of page tables in general, and those allocated by the kernel for its own use in particular. the identity map is just that, it maps virtual addresses to physical ones in a one-to-one manner, that is, virtual address X is mapped to physical address X. it's used during early init but unfortunately some of these page tables survive and remain in active use later as well. that's a problem because all these early page tables are allocated from the first 1MB of RAM, something that even the vanilla kernel's own /dev/mem 'protection' fails to protect, rendering it quite useless as by owning kernel page tables you really own the kernel. now beyond fixing up this 'protection' i also went further and tried to make sure that these page tables are allocated statically inside the kernel's image (which is read-only under KERNEXEC), and that meant that i had to increase the number of such page tables. this is where Xen complains as apparently these statically allocated page tables must be somehow registered with the hypervisor, except for the identity mapping it's not that straightforward, that's where i'd need to know what exactly Xen expects there.

[bplant]

I asked for help on the xen-devel mailing list and got a fast reply (http://lists.xensource.com/archives/htm ... 00731.html) from Jeremy Fitzhardinge (he's been merging all the paravirt_ops stuff into mainline). Here is my email and his reply:

Code: Select all

> Hi list,
>
> I was hoping some people "in the know" about how the hypervisor works etc could take a look at the thread below and hopefully offer some advice.
>
> http://forums.grsecurity.net/viewtopic.php?f=1&t=2063&p=8759#p8745
>
> The PaX developers are trying to get their patch to work on a 2.6.28 paravirt_ops kernel. They already have PaX working with a 2.6.27 paravirt_ops kernel, but a recent change to the PaX patch, specifically to map up to 4GB initially in the identity map is causing the kernel to panic when booted under xen.
>
> Any help on the matter would be greatly appreciated by all.

Looks like what they're trying to do is pretty redundant; Xen is pretty
good at protecting the kernel's pagetables for it.  From just looking at
that piece of thread, I'm guessing they're creating writeable aliases of
the pagetable pages, which Xen won't allow.

Yeah, this one:
|(XEN) mm.c:794:d35 Attempt to create linear p.t. with write perms

While I'm happy to forward messages back and forth between these forums and the xen-devel mailing list, it might be easier if you join the mailing list and post your question(s) directly as things are bound to get lost in translation and be much slower otherwise. You can join the xen-devel mailing list here: http://lists.xensource.com/mailman/listinfo/xen-devel

[PaX Team]

Looks like what they're trying to do is pretty redundant; Xen is pretty good at protecting the kernel's pagetables for it.

ehem, not quite. PaX is protecting its *own* page tables, neither Xen nor anything else on the market does that today, AFAIK.

While I'm happy to forward messages back and forth between these forums and the xen-devel mailing list

how about CC'ing me next time ? . anyway, as things stand, xen dom0 support is scheduled for mainline very soon, i'll take a look at it then.

[bplant]

Looks like what they're trying to do is pretty redundant; Xen is pretty good at protecting the kernel's pagetables for it.

ehem, not quite. PaX is protecting its *own* page tables, neither Xen nor anything else on the market does that today, AFAIK.

I'm just the messenger

While I'm happy to forward messages back and forth between these forums and the xen-devel mailing list

how about CC'ing me next time ? . anyway, as things stand, xen dom0 support is scheduled for mainline very soon, i'll take a look at it then.

Yeah sorry, I should CC'd you on the email. Looking forward to it.

[amos]

Hi, i tried the latest grsec patch (http://grsecurity.org/test/grsecurity-2 ... 1758.patch) with 2.6.31. However i'm not able to boot the resulting image as guest in xen (Debian Lenny, xen 3.2.1, X86_64). I don't have to enable grsec/pax, its enough to patch the source tree. The domU crashes immediately after start:

#xm create -c /etc/xen/vms/xyz
Using config file "/etc/xen/vms/xyz".
Started domain toronly
#

from /var/log/xen/xend.log:

[2009-09-20 09:47:21 3531] INFO (XendDomain:1165) Domain xyz (211) unpaused.
[2009-09-20 09:47:21 3531] WARNING (XendDomainInfo:1258) Domain has crashed: name=xyz id=211.
[2009-09-20 09:47:21 3531] DEBUG (XendDomainInfo:1897) XendDomainInfo.destroy: domid=211

xm dmesg:

(XEN) Unhandled page fault in domain 211 on VCPU 0 (ec=0002)
(XEN) Pagetable walk from 0000000000004018:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 211 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.2-1 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e033:[<ffffffff816f0a5f>]
(XEN) RFLAGS: 0000000000000202 CONTEXT: guest
(XEN) rax: 0000000000004018 rbx: ffffffff81a51000 rcx: ffffffff8177a410
(XEN) rdx: 0000000000000000 rsi: ffffffff81625fdc rdi: 0000000000001d01
(XEN) rbp: 0000000000000000 rsp: ffffffff81625fa0 r8: 0000000000000000
(XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000
(XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000
(XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000006f0
(XEN) cr3: 0000000092dfa000 cr2: 0000000000004018
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
(XEN) Guest stack trace from rsp=ffffffff81625fa0:
(XEN) ffffffff8177a410 0000000000000000 0000000000000002 ffffffff816f0a5f
(XEN) 000000010000e030 0000000000010002 ffffffff81625fe8 000000000000e02b
(XEN) ffffffff816f0a4a 0000000000000000 0000000000000000 0000000000000000
(XEN) ffffffff81a00000 ffffffff81a01000 ffffffff81a02000 ffffffff81a03000
(XEN) ffffffff81a04000 ffffffff81a05000 ffffffff81a06000 ffffffff81a07000
(XEN) ffffffff81a08000 ffffffff81a09000 ffffffff81a0a000 ffffffff81a0b000
(XEN) ffffffff81a0c000 ffffffff81a0d000 ffffffff81a0e000 ffffffff81a0f000
(XEN) ffffffff81a10000 ffffffff81a11000 ffffffff81a12000 ffffffff81a13000
(XEN) ffffffff81a14000 ffffffff81a15000 ffffffff81a16000 ffffffff81a17000
(XEN) ffffffff81a18000 ffffffff81a19000 ffffffff81a1a000 ffffffff81a1b000
(XEN) ffffffff81a1c000 ffffffff81a1d000 ffffffff81a1e000 ffffffff81a1f000
(XEN) ffffffff81a20000 ffffffff81a21000 ffffffff81a22000 ffffffff81a23000
(XEN) ffffffff81a24000 ffffffff81a25000 ffffffff81a26000 ffffffff81a27000
(XEN) ffffffff81a28000 ffffffff81a29000 ffffffff81a2a000 ffffffff81a2b000
(XEN) ffffffff81a2c000 ffffffff81a2d000 ffffffff81a2e000 ffffffff81a2f000
(XEN) ffffffff81a30000 ffffffff81a31000 ffffffff81a32000 ffffffff81a33000
(XEN) ffffffff81a34000 ffffffff81a35000 ffffffff81a36000 ffffffff81a37000
(XEN) ffffffff81a38000 ffffffff81a39000 ffffffff81a3a000 ffffffff81a3b000
(XEN) ffffffff81a3c000 ffffffff81a3d000 ffffffff81a3e000 ffffffff81a3f000
(XEN) ffffffff81a40000 ffffffff81a41000 ffffffff81a42000 ffffffff81a43000

It would be great if someone could look into that, so maybe it will work with the next releases of the kernel. Getting it to work with 2.6.31 isn't that important, since spender found a way to exploit it, too. I'm happy to provide more information if you tell me what you need.

And sorry for the bad formatting, but somehow bbcode is off and i can't find a way to turn it on.

Regards, amos

[PaX Team]

xm dmesg:

(XEN) Unhandled page fault in domain 211 on VCPU 0 (ec=0002)
(XEN) Pagetable walk from 0000000000004018:

i've been working on xen support on and off for a few weeks now, so it'll be definitely fixed.

And sorry for the bad formatting, but somehow bbcode is off and i can't find a way to turn it on.

it's our spam protection, new users don't get to post URLs, etc right away .

[bplant]

i've been working on xen support on and off for a few weeks now, so it'll be definitely fixed.

That's good news. Let us know when you want some testers.

[Chojin]

I am also interested with xen + pax + 2.6.31 for security purpose.

But even with the last test patch published yesterday I did not succeed to run the kernel yet.

[john_anderson_ii]

Pax Team,

I've also tried the latest patchset/kernel (2.6.31.5) and have received this page fault:

Code: Select all

(XEN) Unhandled page fault in domain 26 on VCPU 0 (ec=0000)
(XEN) Pagetable walk from 0000000000000028:
(XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 26 (vcpu#0) crashed on cpu#4:
(XEN) ----[ Xen-3.1.3  x86_64  debug=y  Not tainted ]----
(XEN) CPU:    4
(XEN) RIP:    e033:[<ffffffff81018496>]
(XEN) RFLAGS: 0000000000000282   CONTEXT: guest
(XEN) rax: 0000000000521109   rbx: 0000000000000000   rcx: 0000000000000020
(XEN) rdx: ffffffff82ba6000   rsi: 00000000deadbeef   rdi: 0000000000000000
(XEN) rbp: 0000000000000000   rsp: ffffffff81601f50   r8:  0000000000000000
(XEN) r9:  ffffffff81817283   r10: ffffffff8102f528   r11: ffffffff81004280
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000006b0
(XEN) cr3: 0000000503189000   cr2: 0000000000000028
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
(XEN) Guest stack trace from rsp=ffffffff81601f50:
(XEN)    0000000000000020 ffffffff81004280 0000000000000000 ffffffff81018496
(XEN)    000000010000e030 0000000000010082 ffffffff81601f98 000000000000e02b
(XEN)    0000000000000007 ffffffff81004890 ffffffff8181719e 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffffffff81816c47 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000


Do you want me to put the 2.6.31.5 sources, build, etc on that rPath-Xen test box we setup for you to debug this?

[Chojin]

For information I finally tried HVM Xen guest with a standard kernel patched with grsecurity, no problem anymore :)

[PaX Team]

Do you want me to put the 2.6.31.5 sources, build, etc on that rPath-Xen test box we setup for you to debug this?

yes, please, i was going to do it myself soon now that i got all the free time i need .