xen + pax + 2.6.27

[john_anderson_ii]

yes, please, i was going to do it myself soon now that i got all the free time i need .

Ok, it's done. I'm pre-building the source tree right now so (hopefully) not too many objects will need to be rebuilt as you make changes.

[elbryan]

[quote="Chojin"]
For information I finally tried HVM Xen guest with a standard kernel patched with grsecurity, no problem anymore
[/quote]

Which version did you use? (both kernel and patchset)

[bplant]

Do you want me to put the 2.6.31.5 sources, build, etc on that rPath-Xen test box we setup for you to debug this?

yes, please, i was going to do it myself soon now that i got all the free time i need .

Was 2.6.31.* ever made to work? I've just tried 2.6.32.6 since it's going to be supported long term, but it crashes straight away.

Back trace:

Code: Select all

0xffffffff81814aff in xen_start_kernel () at arch/x86/xen/enlighten.c:1133
1133      per_cpu(xen_vcpu, 0) = &HYPERVISOR_shared_info->vcpu_info[0];

xm dmesg:

Code: Select all

(XEN) d16:v0: unhandled page fault (ec=0002)
(XEN) Pagetable walk from 0000000000004018:
(XEN)  L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 16 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.3.1  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    1
(XEN) RIP:    e033:[<ffffffff81814aff>]
(XEN) RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest
(XEN) rax: 0000000000004018   rbx: ffffffff82284000   rcx: ffffffff8189e410
(XEN) rdx: 0000000000000000   rsi: 0000000000000007   rdi: 0000000000000003
(XEN) rbp: ffffffff81601ff8   rsp: ffffffff81601fa0   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000026b0
(XEN) cr3: 000000016dd7b000   cr2: 0000000000004018
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: e02b   cs: e033
(XEN) Guest stack trace from rsp=ffffffff81601fa0:
(XEN)    ffffffff8189e410 0000000000000000 0000000000000002 ffffffff81814aff
(XEN)    000000010000e030 0000000000010046 ffffffff81601fe8 000000000000e02b
(XEN)    ffffffff81814adf 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffffffff82201000 ffffffff82202000 ffffffff82203000 ffffffff82204000
(XEN)    ffffffff82205000 ffffffff82206000 ffffffff82207000 ffffffff82208000
(XEN)    ffffffff82209000 ffffffff8220a000 ffffffff8220b000 ffffffff8220c000
(XEN)    ffffffff8220d000 ffffffff8220e000 ffffffff8220f000 ffffffff82210000
(XEN)    ffffffff82211000 ffffffff82212000 ffffffff82213000 ffffffff82214000
(XEN)    ffffffff82215000 ffffffff82216000 ffffffff82217000 ffffffff82218000
(XEN)    ffffffff82219000 ffffffff8221a000 ffffffff8221b000 ffffffff8221c000
(XEN)    ffffffff8221d000 ffffffff8221e000 ffffffff8221f000 ffffffff82220000
(XEN)    ffffffff82221000 ffffffff82222000 ffffffff82223000 ffffffff82224000
(XEN)    ffffffff82225000 ffffffff82226000 ffffffff82227000 ffffffff82228000
(XEN)    ffffffff82229000 ffffffff8222a000 ffffffff8222b000 ffffffff8222c000
(XEN)    ffffffff8222d000 ffffffff8222e000 ffffffff8222f000 ffffffff82230000
(XEN)    ffffffff82231000 ffffffff82232000 ffffffff82233000 ffffffff82234000
(XEN)    ffffffff82235000 ffffffff82236000 ffffffff82237000 ffffffff82238000
(XEN)    ffffffff82239000 ffffffff8223a000 ffffffff8223b000 ffffffff8223c000
(XEN)    ffffffff8223d000 ffffffff8223e000 ffffffff8223f000 ffffffff82240000
(XEN)    ffffffff82241000 ffffffff82242000 ffffffff82243000 ffffffff82244000


More than happy to help resolve this issue in any way that I can.

[PaX Team]

Was 2.6.31.* ever made to work?

no, i moved to .32 and am still working on it.

I've just tried 2.6.32.6 since it's going to be supported long term, but it crashes straight away.

hmm, that's not where it dies for me, can you send me your vmlinux (not bzImage) please that corresponds to this report?

[bplant]

I've just tried 2.6.32.6 since it's going to be supported long term, but it crashes straight away.

hmm, that's not where it dies for me, can you send me your vmlinux (not bzImage) please that corresponds to this report?

I have emailed it to you.

[PaX Team]

thanks to John's and Brad's help i managed to fix the problem (it's in xen's vmlinux loader that ignores read-only ELF segments), 32.6-test10 should work now.

[cormander]

Thank, I am able to boot a grsecurity and pax enabled xen domU under x86_64.

I am, however, not able to boot the 32bit version of the kernel. Here is the workspace of the said kernel:

http://build.cormander.com/job/linux-2. ... -grsec/ws/

I haven't even been able to boot the kernel with the pax only patch applied, with no config options enabled. Here is the corresponding workspace:

http://build.cormander.com/job/linux-2. ... config/ws/

The vanilla kernel boots just fine.

It's linux 2.6.32.8 with pax-linux-2.6.32.7-test13.patch applied (there was no patch fuzz so I'm assuming there isn't a problem).

I just noticed you uploaded pax-linux-2.6.32.7-test15.patch today ... I can give that a try tomorrow. Let me know if you see anything out of the ordinary or if I have a bad config.

Thanks

[PaX Team]

I am, however, not able to boot the 32bit version of the kernel. Here is the workspace of the said kernel:

can you post any logs you get please? i have yet to figure out how to test 32 bit domU, so you'll have to help me out here .

[cormander]

Code: Select all

# xm create -c grsec32.cormander.com
Using config file "/etc/xen/grsec32.cormander.com".
Started domain grsec32.cormander.com
#


Output simply ends when it should show booting. This produces in the following in "xm dmesg":

(XEN) traps.c:405:d72 Unhandled general protection fault fault/trap [#13] in domain 72 on VCPU 0 [ec=0000]
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 72 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-3.1.2-128.1.6.el5 x86_64 debug=n Not tainted ]----
(XEN) CPU: 0
(XEN) RIP: e019:[<00000000c1005913>]
(XEN) RFLAGS: 0000000000000246 CONTEXT: guest
(XEN) rax: 000000000000000c rbx: 00000000c1a43000 rcx: 00000000c17fd008
(XEN) rdx: 00000000c1783000 rsi: 00000000c17c2610 rdi: 00000000c170cae8
(XEN) rbp: 00000000c16f1fd0 rsp: 00000000c16f1fc0 r8: 0000000000000000
(XEN) r9: 0000000000000000 r10: 0000000000000000 r11: 0000000000000000
(XEN) r12: 0000000000000000 r13: 0000000000000000 r14: 0000000000000000
(XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000006f0
(XEN) cr3: 000000008f812000 cr2: 0000000000000000
(XEN) ds: e021 es: e021 fs: 00d8 gs: 0000 ss: e021 cs: e019
(XEN) Guest stack trace from esp=c16f1fc0:
(XEN) 00000000 c1005913 0001e019 00010046 c16f1ffc c10058a6 c17fd008 c178d1fb
(XEN) 00000000 00000000 00000000 00000000 00000000 c1a40000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

You should be able to download any binaries and configs from the workspace links in my previous post.

If it helps, this is the log output of the kernel being built: http://build.cormander.com/job/linux-2. ... /6/console

Let me know if there is anything else you need.

Thanks!

[PaX Team]

Output simply ends when it should show booting. This produces in the following in "xm dmesg":

thanks, it seems that the per-cpu segment register is incorrectly set up, i'll take a look.

[gaima]

[quote="PaX Team"]thanks to John's and Brad's help i managed to fix the problem (it's in xen's vmlinux loader that ignores read-only ELF segments), 32.6-test10 should work now.[/quote]

I too have been able to get a 2.6.32 x86_64 kernel booted as a domU, using 2.1.14-2.6.32.7-201002041705. Thanks for the hard work!

This didn't work so well at first though, with the "xm dmesg" included below. After 21 attempts I've isolated the CONFIG directive that causes it to crash. CONFIG_PAX_MPROTECT, on it crashes, off it doesn't.
I was using a variation on CONFIG_GRKERNSEC_HIGH, which forces CONFIG_PAX_MPROTECT on.

[code]
(XEN) d118:v0: unhandled page fault (ec=0010)
(XEN) Pagetable walk from 0000000000000000:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 118 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.4.2 x86_64 debug=n Not tainted ]----
(XEN) CPU: 1
(XEN) RIP: e033:[<0000000000000000>]
(XEN) RFLAGS: 0000000000000206 EM: 1 CONTEXT: pv guest
(XEN) rax: ffff88000156bbd8 rbx: ffffffffff57b000 rcx: ffffffffff57b000
(XEN) rdx: 00003ffffffff000 rsi: ffffffffff57b000 rdi: 0000000806280067
(XEN) rbp: ffffffff81601e78 rsp: ffffffff81601e10 r8: 0000000000000040
(XEN) r9: ffffffff8171d700 r10: 0000000000000002 r11: 0000000000040000
(XEN) r12: ffff88000156bbd8 r13: 80000000dff98063 r14: ffffffffffffffff
(XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000000006f0
(XEN) cr3: 000000080643a000 cr2: 0000000000000000
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
(XEN) Guest stack trace from rsp=ffffffff81601e10:
(XEN) ffffffffff57b000 0000000000040000 0000000000000010 0000000000000000
(XEN) 000000010000e030 0000000000010006 ffffffff81601e50 000000000000e02b
(XEN) ffffffff81027620 0000000000000040 0000000000000884 80000000dff98063
(XEN) 0000000000040000 ffffffff81601e88 ffffffff810276a6 ffffffff81601e98
(XEN) ffffffff8102ba8e ffffffff81601eb8 ffffffff81004502 0000000040000000
(XEN) 0000000000000000 ffffffff81601ec8 ffffffff81003e5b ffffffff81601ed8
(XEN) ffffffff818140d5 ffffffff81601f78 ffffffff818158d1 ffffffff81005eab
(XEN) ffffffff81845830 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 0000000000000000 ffffffff81601f78 ffffffff8138b584 0000000000000010
(XEN) ffffffff81601f88 0000000000000000 0000000000000000 0000000000000000
(XEN) ffffffff81845830 0000000000000000 0000000000000000 ffffffffffffffff
(XEN) 0000000000000000 ffffffff81601fb8 ffffffff81810968 ffffffff81601fb8
(XEN) ffffffff818482e0 000000000193fd08 0000000000000000 0000000000000000
(XEN) 0000000000000000 ffffffff81601fd8 ffffffff81810281 ffffffff81805590
(XEN) ffffffff81c03000 ffffffff81601ff8 ffffffff81813a78 0000000000000000
(XEN) 0000000000000000 0000000000000000 ffffffff81a00000 ffffffff81a01000
(XEN) ffffffff81a02000 ffffffff81a03000 ffffffff81a04000 ffffffff81a05000
(XEN) ffffffff81a06000 ffffffff81a07000 ffffffff81a08000 ffffffff81a09000
(XEN) ffffffff81a0a000 ffffffff81a0b000 ffffffff81a0c000 ffffffff81a0d000
(XEN) ffffffff81a0e000 ffffffff81a0f000 ffffffff81a10000 ffffffff81a11000
[/code]

[cormander]

For what it's worth, CONFIG_GRKERNSEC_HIGH set CONFIG_PAX_KERNEXEC, which causes the xen domU to not even boot. When I set it to custom, CONFIG_PAX_KERNEXEC got correctly removed (via !XEN in its kconfig), and was able to boot.

[PaX Team]

This didn't work so well at first though, with the "xm dmesg" included below. After 21 attempts I've isolated the CONFIG directive that causes it to crash. CONFIG_PAX_MPROTECT, on it crashes, off it doesn't.

that should work too, i tested with all PaX features enabled (for MPROTECT to work you have to clean your system of GNU_STACK breakage though). this crash looks like some NULL function ptr dereference in the guest, not sure why it was handled by the hypervisor though. could you send me the vmlinux that corresponds to this log? or compile one again, crash it and send them both my way?

[PaX Team]

For what it's worth, CONFIG_GRKERNSEC_HIGH set CONFIG_PAX_KERNEXEC, which causes the xen domU to not even boot.

that's weird because your crash log above was from a kernel without KERNEXEC (check the .config).

[cormander]

For what it's worth, CONFIG_GRKERNSEC_HIGH set CONFIG_PAX_KERNEXEC, which causes the xen domU to not even boot.

that's weird because your crash log above was from a kernel without KERNEXEC (check the .config).

Yes, you're right. I figured the KERNEXEC config problem before I discovered the problem with the 32bit kernel. I talked to spender about this and he said that he'd "fix it in the next patch". I'll be sure to check for this too.