Kernel Panic on udp networking

[coredumb]

Hello,

I'm running dnsmasq as my DNS servers on EL6 with grsecurity and I'm being hit quite often by kernel panics, both on 3.2.55 and 3.14.13.
No RBAC used.
Here's the best trace i could get.

Hope it's sufficient.

-C.

[PaX Team]

can you resolve the RIP address to a symbol in vmlinux? something like addr2line -e vmlinux -fip ffffffff815b6667 should be enough.

[coredumb]

With debug enabled, i get only this:

udp_sendmsg
/root/rpmbuild/BUILD/kernel-3.14.13.el6/linux-3.14.13-100.el6.x86_64/net/ipv4/udp.c:869

[coredumb]

Any pointers? I've not yet tried vanilla, but i've seen this issue on last 3.2.55 grsec as well.

[coredumb]

I've finally been able to capture the full panic


Hope this helps

[spender]

Hi,

This is a classic use-after-free condition caught by PAX_MEMORY_SANITIZE. Notice that R12 is set to a poisoned fefefe value, causing a GPF on dereference. The faulting instruction is:
cmp r13, [r12-28h]
I'm quite sure we don't cause this problem -- if you were to disable PAX_MEMORY_SANITIZE, the "problem" would disappear. It's an upstream bug, possibly related to the following:
http://sourceforge.net/p/e1000/mailman/ ... /31312382/

I don't see that the issues raised in those posts have been resolved.

If you can send your vmlinux file we can investigate it further.

Thanks,
-Brad

[coredumb]

Hi Brad,

Thanks for your reply, here's the vmlinux of this version recompiled with enough debug flags. Last time on IRC pipacs seemed to say that the vmlinux wasn't so correct but could find his way in the disasm code.

Hope this helps,
Olivier

[coredumb]

Seems i have the exact same issue with vmxnet3 driver

[PaX Team]

do you still have this problem? if i'm not mistaken you had SANITIZE enabled where we fixed a slab/rcu problem a while ago and i think this is what you experienced as well.

[coredumb]

Oh my sorry i actually did patch my kernel for this specific case and forgot about this thread
I've a fresh 3.14.33 build i can stress test right now, i'll let you know.

By patch i mean:

Code: Select all

diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index cdc3258..984427d 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -2220,7 +2220,7 @@ struct proto udp_prot = {
.sysctl_wmem = &sysctl_udp_wmem_min,
.sysctl_rmem = &sysctl_udp_rmem_min,
.obj_size = sizeof(struct udp_sock),
- .slab_flags = SLAB_DESTROY_BY_RCU,
+ .slab_flags = SLAB_DESTROY_BY_RCU | SLAB_NO_SANITIZE,
.h.udp_table = &udp_table,
#ifdef CONFIG_COMPAT
.compat_setsockopt = compat_udp_setsockopt,

[spender]

You shouldn't need that patch anymore as we exempt slabs with SLAB_DESTROY_BY_RCU from sanitization.

-Brad

[coredumb]

Yes i'm validating that right now with a fresh 3.14.33 build

[coredumb]

All good after 4 days of stress testing.