Page 1 of 1

PAX: refcount overflow detected (2.6.32.16)

PostPosted: Fri Jul 16, 2010 2:49 pm
by mnalis
Hi, a day or so after installing the vanilla 2.6.32 patched with kernel grsecurity-2.2.0-2.6.32.16-201007101507.patch on x86 machine, it crashed with panic on reboot. Few days after, it crashed again, this time I've managed to copy the logs on time. The call trace does not look very helpful, though. I've included it anyways.

It used to work with grsecurity-2.1.14-2.6.32.11-201004071936.patch for months without any problems. Both kernels have CONFIG_PAX_REFCOUNT=y (I can provide full configs if needed)

Jul 16 10:06:44 newaxe kernel: PAX: refcount overflow detected in: md13_raid1:623, uid/euid: 0/0
Jul 16 10:06:44 newaxe kernel:
Jul 16 10:06:44 newaxe kernel: Pid: 623, comm: md13_raid1 Tainted: G W (2.6.32.16-grsec #4)
Jul 16 10:06:44 newaxe kernel: EIP: 0060:[<0009853d>] EFLAGS: 00000806 CPU: 0
Jul 16 10:06:44 newaxe kernel: EAX: c1cac840 EBX: f320000c ECX: 00000025 EDX: f320061c
Jul 16 10:06:44 newaxe kernel: ESI: 00000017 EDI: f2c1a7c0 EBP: f2685bfc ESP: f2685bec
Jul 16 10:06:44 newaxe kernel: DS: 0068 ES: 0068 FS: 00d8 GS: 00e0 SS: 0068
Jul 16 10:06:44 newaxe kernel: CR0: 8005003b CR2: aabbb078 CR3: 01805000 CR4: 000006f0
Jul 16 10:06:44 newaxe kernel: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Jul 16 10:06:44 newaxe kernel: DR6: ffff0ff0 DR7: 00000400
Jul 16 10:06:44 newaxe kernel: Call Trace:
Jul 16 10:06:44 newaxe kernel: [<00098612>]
Jul 16 10:06:44 newaxe kernel: [<0008c95c>]
Jul 16 10:06:44 newaxe kernel: [<00003960>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<003fffff>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<0008cdde>]
Jul 16 10:06:44 newaxe kernel: [<00051200>] ?
Jul 16 10:06:44 newaxe kernel: [<0014c558>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<000891c6>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011210>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<000af809>]
Jul 16 10:06:44 newaxe kernel: [<00011210>] ?
Jul 16 10:06:44 newaxe kernel: [<0008914a>]
Jul 16 10:06:44 newaxe kernel: [<00089434>]
Jul 16 10:06:44 newaxe kernel: [<000a7d0b>]
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<001414cf>]
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<00140076>]
Jul 16 10:06:44 newaxe kernel: [<00036b24>] ?
Jul 16 10:06:44 newaxe kernel: [<00002344>] ?
Jul 16 10:06:44 newaxe kernel: [<0000ffff>] ?
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<00016e14>] ?
Jul 16 10:06:44 newaxe kernel: [<00004cb2>] ?
Jul 16 10:06:44 newaxe kernel: [<0000a801>] ?
Jul 16 10:06:44 newaxe kernel: [<0021a588>]
Jul 16 10:06:44 newaxe kernel: [<0021a676>]
Jul 16 10:06:44 newaxe kernel: [<0002bf9d>] ?
Jul 16 10:06:44 newaxe kernel: [<002c37bf>] ?
Jul 16 10:06:44 newaxe kernel: [<0000863b>] ?
Jul 16 10:06:44 newaxe kernel: [<00036b24>] ?
Jul 16 10:06:44 newaxe kernel: [<00004140>] ?
Jul 16 10:06:44 newaxe kernel: [<00004140>] ?
Jul 16 10:06:44 newaxe kernel: [<0000219c>] ?
Jul 16 10:06:44 newaxe kernel: [<0003d5ee>] ?
Jul 16 10:06:44 newaxe kernel: [<0003da13>] ?
Jul 16 10:06:44 newaxe kernel: [<0003da28>] ?
Jul 16 10:06:44 newaxe kernel: [<002c3b52>] ?
Jul 16 10:06:44 newaxe kernel: [<0003db02>] ?
Jul 16 10:06:44 newaxe kernel: [<00230099>]
Jul 16 10:06:44 newaxe kernel: [<00047922>] ?
Jul 16 10:06:44 newaxe kernel: [<0022ffc0>] ?
Jul 16 10:06:44 newaxe kernel: [<00047711>]
Jul 16 10:06:44 newaxe kernel: [<000476b0>] ?
Jul 16 10:06:44 newaxe kernel: [<00004fe7>]
Jul 16 10:06:44 newaxe kernel: PAX: refcount overflow detected in: md13_raid1:623, uid/euid: 0/0
Jul 16 10:06:44 newaxe kernel:
Jul 16 10:06:44 newaxe kernel: Pid: 623, comm: md13_raid1 Tainted: G W (2.6.32.16-grsec #4)
Jul 16 10:06:44 newaxe kernel: EIP: 0060:[<0009853d>] EFLAGS: 00000806 CPU: 0
Jul 16 10:06:44 newaxe kernel: EAX: c1cac850 EBX: f320000c ECX: 00000025 EDX: f320062c
Jul 16 10:06:44 newaxe kernel: ESI: 0000001b EDI: f2c1a7c0 EBP: f2685bfc ESP: f2685bec
Jul 16 10:06:44 newaxe kernel: DS: 0068 ES: 0068 FS: 00d8 GS: 00e0 SS: 0068
Jul 16 10:06:44 newaxe kernel: CR0: 8005003b CR2: aabbb078 CR3: 01805000 CR4: 000006f0
Jul 16 10:06:44 newaxe kernel: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Jul 16 10:06:44 newaxe kernel: DR6: ffff0ff0 DR7: 00000400
Jul 16 10:06:44 newaxe kernel: Call Trace:
Jul 16 10:06:44 newaxe kernel: [<00098638>]
Jul 16 10:06:44 newaxe kernel: [<0008c95c>]
Jul 16 10:06:44 newaxe kernel: [<00003960>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<003fffff>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<0008cdde>]
Jul 16 10:06:44 newaxe kernel: [<00051200>] ?
Jul 16 10:06:44 newaxe kernel: [<0014c558>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<000891c6>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011210>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<000af809>]
Jul 16 10:06:44 newaxe kernel: [<00011210>] ?
Jul 16 10:06:44 newaxe kernel: [<0008914a>]
Jul 16 10:06:44 newaxe kernel: [<00089434>]
Jul 16 10:06:44 newaxe kernel: [<000a7d0b>]
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<001414cf>]
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<00140076>]
Jul 16 10:06:44 newaxe kernel: [<00036b24>] ?
Jul 16 10:06:44 newaxe kernel: [<00002344>] ?
Jul 16 10:06:44 newaxe kernel: [<0000ffff>] ?
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<00016e14>] ?
Jul 16 10:06:44 newaxe kernel: [<00004cb2>] ?
Jul 16 10:06:44 newaxe kernel: [<0000a801>] ?
Jul 16 10:06:44 newaxe kernel: [<0021a588>]
Jul 16 10:06:44 newaxe kernel: [<0021a676>]
Jul 16 10:06:44 newaxe kernel: [<0002bf9d>] ?
Jul 16 10:06:44 newaxe kernel: [<002c37bf>] ?
Jul 16 10:06:44 newaxe kernel: [<0000863b>] ?
Jul 16 10:06:44 newaxe kernel: [<00036b24>] ?
Jul 16 10:06:44 newaxe kernel: [<00004140>] ?
Jul 16 10:06:44 newaxe kernel: [<00004140>] ?
Jul 16 10:06:44 newaxe kernel: [<0000219c>] ?
Jul 16 10:06:44 newaxe kernel: [<0003d5ee>] ?
Jul 16 10:06:44 newaxe kernel: [<0003da13>] ?
Jul 16 10:06:44 newaxe kernel: [<0003da28>] ?
Jul 16 10:06:44 newaxe kernel: [<002c3b52>] ?
Jul 16 10:06:44 newaxe kernel: [<0003db02>] ?
Jul 16 10:06:44 newaxe kernel: [<00230099>]
Jul 16 10:06:44 newaxe kernel: [<00047922>] ?
Jul 16 10:06:44 newaxe kernel: [<0022ffc0>] ?
Jul 16 10:06:44 newaxe kernel: [<00047711>]
Jul 16 10:06:44 newaxe kernel: [<000476b0>] ?
Jul 16 10:06:44 newaxe kernel: [<00004fe7>]
Jul 16 10:06:44 newaxe kernel: 000000
Jul 16 10:06:44 newaxe kernel: DR6: ffff0ff0 DR7: 00000400
Jul 16 10:06:44 newaxe kernel: Call Trace:
Jul 16 10:06:44 newaxe kernel: [<00098612>]
Jul 16 10:06:44 newaxe kernel: [<0008c95c>]
Jul 16 10:06:44 newaxe kernel: [<00003960>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<003fffff>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<0008cdde>]
Jul 16 10:06:44 newaxe kernel: [<001c9890>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<000891c6>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011210>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?
Jul 16 10:06:44 newaxe kernel: [<000af809>]
Jul 16 10:06:44 newaxe kernel: [<00011210>] ?
Jul 16 10:06:44 newaxe kernel: [<0008914a>]
Jul 16 10:06:44 newaxe kernel: [<00089434>]
Jul 16 10:06:44 newaxe kernel: [<000a7d0b>]
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<001414cf>]
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<00140076>]
Jul 16 10:06:44 newaxe kernel: [<00036b24>] ?
Jul 16 10:06:44 newaxe kernel: [<00002344>] ?
Jul 16 10:06:44 newaxe kernel: [<0000ffff>] ?
Jul 16 10:06:44 newaxe kernel: [<00800000>] ?
Jul 16 10:06:44 newaxe kernel: [<00016e14>] ?
Jul 16 10:06:44 newaxe kernel: [<00004cb2>] ?
Jul 16 10:06:44 newaxe kernel: [<0000a801>] ?
Jul 16 10:06:44 newaxe kernel: [<0021a588>]
Jul 16 10:06:44 newaxe kernel: [<0021a676>]
Jul 16 10:06:44 newaxe kernel: [<0002bf9d>] ?
Jul 16 10:06:44 newaxe kernel: [<002c37bf>] ?
Jul 16 10:06:44 newaxe kernel: [<0000863b>] ?
Jul 16 10:06:44 newaxe kernel: [<00036b24>] ?
Jul 16 10:06:44 newaxe kernel: [<00004140>] ?
Jul 16 10:06:44 newaxe kernel: [<00004140>] ?
Jul 16 10:06:44 newaxe kernel: [<0000219c>] ?
Jul 16 10:06:44 newaxe kernel: [<0003d5ee>] ?
Jul 16 10:06:44 newaxe kernel: [<0003da13>] ?
Jul 16 10:06:44 newaxe kernel: [<0003da28>] ?
Jul 16 10:06:44 newaxe kernel: [<002c3b52>] ?
Jul 16 10:06:44 newaxe kernel: [<0003db02>] ?
Jul 16 10:06:44 newaxe kernel: [<00230099>]
Jul 16 10:06:44 newaxe kernel: [<00047922>] ?
Jul 16 10:06:44 newaxe kernel: [<0022ffc0>] ?
Jul 16 10:06:44 newaxe kernel: [<00047711>]
Jul 16 10:06:44 newaxe kernel: [<000476b0>] ?
Jul 16 10:06:44 newaxe kernel: [<00004fe7>]
Jul 16 10:06:44 newaxe kernel: PAX: refcount overflow detected in: md13_raid1:623, uid/euid: 0/0
Jul 16 10:06:44 newaxe kernel:
Jul 16 10:06:44 newaxe kernel: Pid: 623, comm: md13_raid1 Tainted: G W (2.6.32.16-grsec #4)
Jul 16 10:06:44 newaxe kernel: EIP: 0060:[<0009853d>] EFLAGS: 00000806 CPU: 0
Jul 16 10:06:44 newaxe kernel: EAX: c1cac850 EBX: f320000c ECX: 00000025 EDX: f320062c
Jul 16 10:06:44 newaxe kernel: ESI: 0000001b EDI: f2c1a7c0 EBP: f2685bfc ESP: f2685bec
Jul 16 10:06:44 newaxe kernel: DS: 0068 ES: 0068 FS: 00d8 GS: 00e0 SS: 0068
Jul 16 10:06:44 newaxe kernel: CR0: 8005003b CR2: aabbb078 CR3: 01805000 CR4: 000006f0
Jul 16 10:06:44 newaxe kernel: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Jul 16 10:06:44 newaxe kernel: DR6: ffff0ff0 DR7: 00000400
Jul 16 10:06:44 newaxe kernel: Call Trace:
Jul 16 10:06:44 newaxe kernel: [<00098638>]
Jul 16 10:06:44 newaxe kernel: [<0008c95c>]
Jul 16 10:06:44 newaxe kernel: [<00003960>] ?
Jul 16 10:06:44 newaxe kernel: [<00031200>] ?
Jul 16 10:06:44 newaxe kernel: [<00011200>] ?

[... etc ... it repeats quite a few times, and then eventually system decides to reboot with panic on reboot]

Jul 16 10:06:46 newaxe kernel: PAX: refcount overflow detected in: md13_raid1:623, uid/euid: 0/0
Jul 16 10:06:46 newaxe kernel:
Jul 16 10:06:46 newaxe kernel: Pid: 623, comm: md13_raid1 Tainted: G W (2.6.32.16-grsec #4)
Jul 16 10:06:46 newaxe kernel: EIP: 0060:[<0009853d>] EFLAGS: 00000802 CPU: 0
Jul 16 10:06:46 newaxe kernel: EAX: c1cac850 EBX: f320000c ECX: 00000025 EDX: f320062c
Jul 16 10:06:46 newaxe kernel: ESI: 0000001b EDI: f2c1a7c0 EBP: f2685bfc ESP: f2685bec
Jul 16 10:06:46 newaxe kernel: DS: 0068 ES: 0068 FS: 00d8 GS: 00e0 SS: 0068
Jul 16 10:06:46 newaxe kernel: CR0: 8005003b CR2: 0806c082 CR3: 01805000 CR4: 000006f0
Jul 16 10:06:46 newaxe kernel: DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Jul 16 10:06:46 newaxe kernel: DR6: ffff0ff0 DR7: 00000400
Jul 16 10:06:46 newaxe kernel: Call Trace:
Jul 16 10:06:46 newaxe kernel: [<00098638>]
Jul 16 10:06:46 newaxe kernel: [<0008c95c>]
Jul 16 10:06:46 newaxe kernel: [<00003960>] ?
Jul 16 10:06:46 newaxe kernel: [<00031200>] ?
Jul 16 10:06:46 newaxe kernel: [<00011200>] ?
Jul 16 10:06:46 newaxe kernel: [<003fffff>] ?
Jul 16 10:16:14 newaxe syslogd 1.5.0#5: restart (remote reception).

Any ideas? if not I'll try putting kernel with disabled CONFIG_PAX_REFCOUNT so it boots into it on next try, but as this feature used to work for me, I was hoping I could keep it. :-)

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Fri Jul 16, 2010 5:44 pm
by PaX Team
mnalis wrote:Hi, a day or so after installing the vanilla 2.6.32 patched with kernel grsecurity-2.2.0-2.6.32.16-201007101507.patch on x86 machine, it crashed with panic on reboot. Few days after, it crashed again, this time I've managed to copy the logs on time. The call trace does not look very helpful, though. I've included it anyways.

It used to work with grsecurity-2.1.14-2.6.32.11-201004071936.patch for months without any problems. Both kernels have CONFIG_PAX_REFCOUNT=y (I can provide full configs if needed)
i'll need the corresponding vmlinux (the uncompressed) image to tell whether it's a new false positive or a real overflow ;).

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Fri Jul 16, 2010 7:01 pm
by mnalis
I've put it up at http://voyager.hr/~mnalis/tmp/vmlinux
md5sum is 083448e99959317a3b96c8bc13e194dd

Thanks

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Fri Jul 16, 2010 10:01 pm
by PaX Team
mnalis wrote:I've put it up at http://voyager.hr/~mnalis/tmp/vmlinux
thanks, it's a false positive, fixed in the latest patches ;).

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Sat Jul 17, 2010 7:24 am
by mnalis
Thanks a lot!

Errr, where one I download a patch for this fix (which I can apply against grsec patched kernel) ?
I guess it is not included in http://www.grsecurity.net/stable/grsecu ... 2107.patch ?

CVS mentioned on grsecurity.net download page ( http://cvsweb.grsecurity.net/index.cgi/ ) seems grossly outdated.

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Sat Jul 17, 2010 10:33 am
by PaX Team
mnalis wrote:I guess it is not included in http://www.grsecurity.net/stable/grsecu ... 2107.patch ?
yet it is ;).

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Sat Jul 17, 2010 12:46 pm
by mnalis
oh, great then :) Applied and compiling; I'll let you know if it crashes again...

Re: PAX: refcount overflow detected (2.6.32.16)

PostPosted: Wed Sep 01, 2010 4:45 am
by AleFranz
is a download link to 2.6.32.16 patch still available? thanks!