glibc-2.3 pthreads broken

[goodbyte]

Hi
I've been trying to upgrade my system to glibc-2.3.1 for a while now, and I've tracked down the bug to something I think is a grsecurity/PaX bug/incompatibility. Some background: I'm runnig a linux-2.4.19 box with grsecurity-1.9.7d, NOEXEC, SEGMEXEC, MPROTECT, ASLR, RANDKSTACK, RANDUSTACK, RANDMMAP, RANDEXEC, KMEM are set. Everything is compied with gcc-3.2.

After compiling glibc-2.3.1 (same error with 2.3) and running make check I get failure in two tests linking statically against the pthread library. When I install, all binaries linked against pthread exit with a SEGFAULT. I ran a strace on one of these programs (ls), and found that the last call is to modify_ldt, which fails. I tried disabling SEGMEXEC on the binary and it started working! (func = 1, so it should be the block in arch/i386/kernel/ldt.c:write_ldt that fails.)

Any suggestions on how this can be fixed in the kernel, or is it the pthreads implementation that wants to access memory above 1.5GB?

[goodbyte]

I reviewed the linuxtrhreads code. All calls to modify_ldt are done similar to this:

Code: Select all

/* from linuxthreads/sysdeps/i386/useldt.h, continue bakslashes removed */
#define DO_MODIFY_LDT(descr, nr)
({
  struct modify_ldt_ldt_s ldt_entry =
    { nr, (unsigned long int) (descr), 0xfffff /* 4GB in pages */,
      1, 0, 0, 1, 0, 1 };
  if (__modify_ldt (1, &ldt_entry, sizeof (ldt_entry)) != 0)
    abort ();
  asm ("movw %w0, %%gs" : : "q" (nr * 8 + 7));
})


The thing here is the 4GB in pages. From the ChangeLog:

2002-09-17 Roland McGrath <roland@redhat.com>

* sysdeps/i386/tls.h (TLS_DO_MODIFY_LDT, TLS_DO_SET_THREAD_AREA):
Set the descriptor limit to the full 4GB, so %gs:OFFSET works for any
offset (positive or negative) relative to the thread struct.
* sysdeps/i386/useldt.h (DO_MODIFY_LDT, DO_SET_THREAD_AREA): Likewise.

But I can't see where in the code this is used...

[PaX Team]

Any suggestions on how this can be fixed in the kernel, or is it the pthreads implementation that wants to access memory above 1.5GB?

there are a few issues raised with this new glibc/TLS support.

1. the new TLS mechanism requires kernel support which has not yet been backported to the 2.4 series, therefore you should not configure your glibc with --with-tls at all (if you do, your applications will not work regardless if PaX is enabled or not).

2. SEGMEXEC reduces the available address space to 1.5GB and declares the rest off limits, although technically i could allow access to the 1.5-3GB range as well (the executable mappings are read-only). to be honest, i don't like this idea of glibc abusing the segmentation mechanism for accessing 'negative' offsets, that doesn't work for 'any offset' as they claim (think about 'mov eax,[gs:0xffffffff]' and the like).

[goodbyte]

1. the new TLS mechanism requires kernel support which has not yet been backported to the 2.4 series, therefore you should not configure your glibc with --with-tls at all (if you do, your applications will not work regardless if PaX is enabled or not).

Yes, but according to the glibc release anouncment, tls is off by default and I even use --without-tls, and checked config.h - HAVE_TLS_SUPPORT is undefined.

2. SEGMEXEC reduces the available address space to 1.5GB and declares the rest off limits, although technically i could allow access to the 1.5-3GB range as well (the executable mappings are read-only). to be honest, i don't like this idea of glibc abusing the segmentation mechanism for accessing 'negative' offsets, that doesn't work for 'any offset' as they claim (think about 'mov eax,[gs:0xffffffff]' and the like).

So it would be possible to just skip the block that checks the limit? Even in a "normal" situation, pthreads would request 4GB, when only 3GB are available, right?

[PaX Team]

Yes, but according to the glibc release anouncment, tls is off by default and I even use --without-tls, and checked config.h - HAVE_TLS_SUPPORT is undefined.

hmm, then your modify_ldt() calls with 4GB limit came from somewhere else, would be nice if you could find out where from (may pinpoint a real bug in glibc, i don't see where else they would need 4G limits).

So it would be possible to just skip the block that checks the limit?

not skip it altogether, just remove the checks against TASK_SIZE/2. the other check in (ldt_info.contents & 2) is needed to prevent code segments getting into the LDT.

Even in a "normal" situation, pthreads would request 4GB, when only 3GB are available, right?

yes, they need the 4GB limit not to address memory above 3GB (in linear address space), but to be able to abuse the address wraparound and address memory below the base address of GS (actually i wonder what the heck there is, the pthread thread descriptor structure begins at gs:0 already, so no member has a negative offset...).

[goodbyte]

Thanks for the help, I removed the TASK_SIZE/2 checks, and now everything seems to be fine!

From what I've seen in the source code, this should affect glibc, when compiled on linux >2.3.99 on a i686 or higher. Probably it's safe to exclude the calls to modify_ldt from glibc (undefine __ASUME_LDT_WORKS, or remove #include "../useldt.h" from linuxthreads/sysdeps/i386/i686/pt-machine.h) but since I got it working I'll leave it as it is. (Note, this is when not using tls.)

I don't think this is a bug, all tls related functions reside in linuxthreads/sysdeps/i386/tls.h and that file is #ifdef HAVE_TLS, but does not seem critical for pthreads to make the call to modify_ldt either. (I don't know too much about the inner workings of linuxthreads though.)

At least I got it woking! Thanks again.