PAX: size overflow in function environ_read fs/proc/base.c

[jotik]

Using Gentoo-s hardened-sources (4.2.6-r8 and 4.3.3-r4):

Code: Select all

PAX: size overflow detected in function environ_read fs/proc/base.c:1067 cicus.472_265 min, count: 58, decl: access_remote_vm; num: 4; context: fndecl; /usr/bin/ldd[ldd:24710] uid/euid:0/0 gid/egid:0/0
CPU: 4 PID: 2522 Comm: ps Not tainted 4.2.6-hardened-r8 #1tr.so.1.1.0.debug by /lib64/ld-2.21.so[ld-linux-x86-64:24718] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/ldd[ldd:24716] uid/euid:0/0 gid/egid:0/0
 ffffffff81805c83 b24ca701ca6321a0 0000000000000000 ffffffff816723cf.0.debug by /lib64/ld-2.21.so[ld-linux-x86-64:24724] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/ldd[ldd:24722] uid/euid:0/0 gid/egid:0/0
 ffffc90007ffbc48 ffffffff8152feb7 0000000000000097 ffffffff8167246d-2.21.so.debug by /lib64/ld-2.21.so[ld-linux-x86-64:24730] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/ldd[ldd:24728] uid/euid:0/0 gid/egid:0/0
 ffffc90007ffbc78 ffffffff81139f96 0000000000000000 0000000000000000
Call Trace:
 [<ffffffff8152feb7>] dump_stack+0x45/0x5d
 [<ffffffff81139f96>] report_size_overflow+0x36/0x40
 [<ffffffff811938f5>] environ_read+0x3a5/0x4f0
 [<ffffffff811306dd>] __vfs_read+0x5d/0x140
 [<ffffffff8126518b>] ? security_file_permission+0xbb/0xd0
 [<ffffffff81130893>] vfs_read+0xd3/0x240
 [<ffffffff81130f7c>] SyS_read+0x5c/0xe0
 [<ffffffff81534eb0>] entry_SYSCALL_64_fastpath+0x12/0x8a


Code: Select all

PAX: size overflow detected in function environ_read fs/proc/base.c:1064 cicus.503_265 min, count: 54, decl: access_remote_vm; num: 4; context: fndecl;
CPU: 0 PID: 18410 Comm: ps Not tainted 4.3.3-hardened-r4 #1
 ffffffff00000002 14080fbf382ba2a6 0000000000000000 0000000000000428
 ffffc90003bcbc20 ffffffff812c27b8 ffffffff816912de ffffc90003bcbc50
 ffffffff8114397e 0000000000000000 0000000000000000 fffffc68e816d818
Call Trace:
 [<ffffffff812c27b8>] dump_stack+0x44/0x5c
 [<ffffffff8114397e>] report_size_overflow+0x6e/0x80
 [<ffffffff811a00d3>] environ_read+0x3b3/0x510
 [<ffffffff81139e96>] __vfs_read+0x56/0x130
 [<ffffffff81276ff8>] ? security_file_permission+0xc8/0xe0
 [<ffffffff8113a044>] vfs_read+0xd4/0x250
 [<ffffffff8113a735>] SyS_read+0x55/0xd0
 [<ffffffff81550ff0>] entry_SYSCALL_64_fastpath+0x12/0x8a


[ephox]

Could you please send me the results (fs/proc/base.*) of make fs/proc/base.o EXTRA_CFLAGS="-fdump-tree-all -fdump-ipa-all" and your kernel .config?
Which gcc version did you use?

[jotik]

You can find the fs/proc/base.* files here, no .config. GCC version 4.9.3.

[ephox]

Could you please apply this patch (4.3.3) and send me the result from dmesg?

Code: Select all

--- fs/proc/base.c.orig 2016-01-19 22:01:14.699210722 +0100
+++ fs/proc/base.c      2016-01-19 22:05:22.467199676 +0100
@@ -1061,6 +1061,7 @@
                this_len = mm->env_end - (mm->env_start + src);
 
                max_len = min_t(size_t, PAGE_SIZE, count);
+               printk(KERN_ERR "PAX environ_read: env_end: %lx, mm->env_start: %lx, src: %lx, count: %lx\n", mm->env_end, mm->env_start, src, count);
                this_len = min(max_len, this_len);
 
                retval = access_remote_vm(mm, (mm->env_start + src),


[jotik]

Last time this overflow happened was after 12 days of uptime for that VM. I usually reboot it more frequently, so patching and trying to reproduce the bug will likely take too much time for me. Sorry!

[ephox]

Last time this overflow happened was after 12 days of uptime for that VM. I usually reboot it more frequently, so patching and trying to reproduce the bug will likely take too much time for me. Sorry!

I see. But I didn't find the bug so I don't know how to fix it. It could be a real bug not a false positive

[jotik]

Happened again with 4.4.4, for which I didn't to apply the patch...

Code: Select all

[343779.647418] PAX: size overflow detected in function environ_read /usr/src/linux-4.4.4-hardened/fs/proc/base.c:1064 cicus.477_287 min, count: 54, decl: access_remote_vm; num: 4; context: fndecl;
[343779.647926] CPU: 5 PID: 7976 Comm: ps Not tainted 4.4.4-hardened #1
[343779.647929]  797c830700000002 797c8307d0c10909 0000000000000286 0000000000000000
[343779.647934]  ffffc90001aebc10 ffffffff812ca75a 0000000000000000 797c8307d0c10909
[343779.647937]  ffffffff816aa870 0000000000000428 ffffc90001aebc40 ffffffff8114a8ce
[343779.647941] Call Trace:
[343779.647948]  [<ffffffff812ca75a>] dump_stack+0x76/0xbc
[343779.647953]  [<ffffffff8114a8ce>] report_size_overflow+0x6e/0x80
[343779.647957]  [<ffffffff811a7041>] environ_read+0x3b1/0x510
[343779.647961]  [<ffffffff81140d77>] __vfs_read+0x57/0x140
[343779.647965]  [<ffffffff8127ddb8>] ? security_file_permission+0xc8/0xe0
[343779.647969]  [<ffffffff81140f23>] vfs_read+0xc3/0x250
[343779.647972]  [<ffffffff8114163c>] SyS_read+0x5c/0xd0
[343779.647977]  [<ffffffff8155cd30>] entry_SYSCALL_64_fastpath+0x12/0x8a

[ephox]

Thanks for the report, but I need the printk output before I can fix this.

[jotik]

Code: Select all

... after many previous similar debug messages ...
[37315.188078] PAX environ_read: env_end: 38b2728fbf8, mm->env_start: 38b2728f4ed, src: 0, count: 7ff
[37315.188218] PAX environ_read: env_end: 382858f104b, mm->env_start: 382858f0929, src: 0, count: 7ff
[37315.188715] PAX environ_read: env_end: 0, mm->env_start: 39cf59b19ca, src: 0, count: 7ff
[37315.188717] PAX: size overflow detected in function environ_read fs/proc/base.c:1065 cicus.479_290 min, count: 54, decl: access_remote_vm; num: 4; context: fndecl;
[37315.189082] CPU: 5 PID: 20991 Comm: ps Not tainted 4.4.6-hardened-r1 #2
[37315.189083]  28c16bdf00000002 28c16bdf5c6fdc95 0000000000000286 0000000000000000
[37315.189085]  ffffc90003aa3c80 ffffffff812c9e2a 0000039cf59b19ca 28c16bdf5c6fdc95
[37315.189087]  ffffffff816a08ef 0000000000000429 ffffc90003aa3cb0 ffffffff8114a47e
[37315.189088] Call Trace:
[37315.189092]  [<ffffffff812c9e2a>] dump_stack+0x76/0xbc
[37315.189095]  [<ffffffff8114a47e>] report_size_overflow+0x6e/0x80
[37315.189097]  [<ffffffff811a6dac>] environ_read+0x38c/0x5b0
[37315.189100]  [<ffffffff81140a07>] __vfs_read+0x57/0x130
[37315.189102]  [<ffffffff8127d4db>] ? security_file_permission+0xbb/0xd0
[37315.189104]  [<ffffffff81140ba3>] vfs_read+0xc3/0x240
[37315.189108]  [<ffffffff81141279>] SyS_read+0x59/0xd0
[37315.189111]  [<ffffffff8155c670>] entry_SYSCALL_64_fastpath+0x12/0x8a
[45519.802369] PAX environ_read: env_end: 39b3e125fd5, mm->env_start: 39b3e1258ca, src: 0, count: 7ff
[45519.802531] PAX environ_read: env_end: 3ae9efb655f, mm->env_start: 3ae9efb5e3d, src: 0, count: 7ff
[45519.802722] PAX environ_read: env_end: 3cdd8d85f12, mm->env_start: 3cdd8d857ef, src: 0, count: 7ff
... more of these debug messages follow ...


[ephox]

I think it is a real bug, env_end can't be 0. Could you please report it to upstream?

[jotik]

Reported to upstream Kernel Bug Tracker as Bug 116461. I won't waste my time on their mailing lists.

[minipli]

I guess your application is reading /proc/PID/environ of a not yet fully initialized process so env_end can, in fact, be zero in this case (see fs/binfmt_elf.c:create_elf_tables(), it sets env_end last).

Other places facing the same problem simply test the variable to be non-zero to test if 'mm' is ready for inspection, e.g. fs/proc/base.c:proc_pid_cmdline_read() or mm/util.c:get_cmdline()). So, I'd say, just do so here as well.

Can you try the following patch, please?:

Code: Select all

diff --git a/fs/proc/base.c b/fs/proc/base.c
index 417bafef0012..c1235e2e6b63 100644
--- a/fs/proc/base.c
+++ b/fs/proc/base.c
@@ -1034,7 +1034,7 @@ static ssize_t environ_read(struct file *file, char __user *buf,
        int ret = 0;
        struct mm_struct *mm = file->private_data;
 
-       if (!mm)
+       if (!mm || !mm->env_end)
                return 0;
 
 #ifdef CONFIG_GRKERNSEC_PROC_MEMMAP


But Emese is right, this looks like an upstream bug that should probably be fixed there as well.

[jotik]

*shrugs*

I don't have a reliable means to trigger this code path. If I apply the patch and the PAX size overflow plugin doesn't trigger for N days, then what do you suppose that would tell us? That it compiled?

[minipli]

It'll ensure the condition that triggers the size overflow won't happen -- assuming it compiles

[minipli]

Btw., here's a way to reproduce the size overflow fast:

Code: Select all

$ while : ; do cat /proc/*/environ >/dev/null 2>&1; done &
$ for i in $(seq 16); do export FOO$i="$(dd if=/dev/urandom bs=16k count=1 2>&1 | hexdump -C)"; done
$ while : ; do /bin/true; done


Without the patch the size overflow triggered in the second loop already. With the patch it survived for around 15 minutes so far