Xorg fails to fork therefore cannot invoke xkbcomp

[Dwokfur]

Xorg.log
(**) Option "xkb_rules" "evdev"
(**) Option "xkb_model" "evdev"
(**) Option "xkb_layout" "hu,us"
(**) Option "xkb_variant" "standard"
(**) Option "xkb_options" "grp:shift_toggle,compose:ralt"
(EE) XKB: Could not invoke xkbcomp
(EE) XKB: Couldn't compile keymap
(WW) Couldn't load XKB keymap, falling back to pre-XKB keymap

grsec.log
grsec: (root:U:/usr/bin/Xorg) failed fork with errno -12 by /usr/bin/Xorg[X:pid] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/gdm-binary[gdm:pid] uid/euid:0/0 gid/egid:0/0

Hardened Gentoo System running:
xorg-server-1.5.3-r6
xkbcomp-1.0.5
hardened-sources-2.6.29

Are there any suggestions?

[PaX Team]

hardened-sources-2.6.29

which version of grsec is that? and can you try spender's latest test patch?

[Dwokfur]

hardened-sources-2.6.29

which version of grsec is that? and can you try spender's latest test patch?

1.) hardened-sources-2.6.28-r7 operates well: it includes grsec-2.1.13-2.6.28.8-200903191958
2.) hardened-sources-2.6.28-r9 failed at a very early stage of loading the kernel (grsec-2.1.13-2.6.28.10-200905171500)
- I was about to take a snapshot of the remote console about this boot failure, when I noticed, that hardened-sources-2.6.29 was out
3.) hardened-sources-2.9.29 worked well remotely making me happy (grsec-2.1.14-2.6.29.3-200905192213), except when I approached the server I noticed I cannot switch to a text VT...

The issue causing Xorg to fail forking appeared somewhere between 1.) and 3.).

It may be also interesting, that all the above mentioned kernel work well on my Pentium M laptop. The issue (and the early boot hang with 2.)) hit only the Athlon MP server.

I'll give a try to the latest grsec patch and get back with the results soon.

Kind Regards:
Dw.

[Dwokfur]

hardened-sources-2.6.29

which version of grsec is that? and can you try spender's latest test patch?

I've first bumped hardened sources using grsecurity-2.1.14-2.6.29.4-200906031821.patch. The problem was there. I used a vanilla kernel after that. It also has the symptom.

What's next? How I can help you to track down the problem?
It appears only on my Athlon MP system and not on my Pentium M laptop. Only Xorg triggers it. I don't have exec logging enabled. Last time I could se

I cannot find "-12" in errno.h...

Refs:
viewtopic.php?p=6030

Regards:
Dw.

[specs]

There are more problems with vanilla kernels and X.org.

For example some Intel IGP solutions have known problems. They do not work correct with 2.6.28 and 2.6.29.
Send in an error report to the LKML or search internet if the problem is known and solved. If you need a stable and working system you could think about downgrading to 2.6.27 with the latest patches.

(Unfortunately grsecurity-2.1.12-2.6.27.10-200812271347 does not work with 2.6.27.24 and I did not have success with the 2.6.30-rc kernels either.)

FYI the bug in intel igp's was due to the in-kernel drivers leading to an "Failed to set tiling in front buffer: rejected by kernel".
And only certain chipsets are susceptible (like the 910ML). If you're in luck like me the patch will be shipped with 2.6.30.

[Dwokfur]

The affected Athlon MP system uses a Radeon 8500LE AGP card for display and my laptops have either use Radeon 9000 Mobility Fire GL or a Radeon 7500. All systems has the regular radeon driver compiled into the kernel for framebuffer and drm support and I also use the opensource Radeon driver shipped with Xorg.
The problem pops up only on the Athlon MP and not with Pentium M. Kernel based on 2.6.28.8 runs perfectly on the Athlon MP server. The issue started to arise after that.
Could you please confirm, that I may have been hit by an Intel IGP related bug despite it's not compiled in? How I can check it out if some harmful parts get compiled in anyways? Which options I should be aware of? (I've heard of those problems around introducing new graphically related improvements into the kernel regarding Intel controllers. It can be - at least partially - switched off to fall back to the good old implementetion).

Don't hesitate to communicate any useful tips about this topic.

Regards:
Dw.

[specs]

I think it's highly unlikely you suffer from the "intel igp bug".

It happens to only a few older intel chipsets (not even all chipsets, the GM950 seems not to be affected).
If I'm correct 2.6.28 was supposed to introduce GEM (memory management for onboard videocards without dedicated RAM).
The second part KMS was supposed to be introduced in 2.6.29.
As far as I know only Intel had it's drivers ready. There was a Radeon KMS-driver but it was released after 2.6.28.

Kernel mode setting in the kernel:
http://www.phoronix.com/scan.php?page=n ... &px=NzE2NA
AMD releases new driver (mai 2009)
http://www.phoronix.com/scan.php?page=a ... uide&num=1
And it needs testing (april 2009)
http://www.phoronix.com/scan.php?page=n ... &px=NzIzOA

Normally with new features one can expect new bugs. Especially with a revision of this scale.

[PaX Team]

I've first bumped hardened sources using grsecurity-2.1.14-2.6.29.4-200906031821.patch. The problem was there. I used a vanilla kernel after that. It also has the symptom.

What's next? How I can help you to track down the problem?

since it's apparently a vanilla problem, you should post to lkml and/or the kernel bugzilla and let the kernel devs figure it out. my guess is that fork fails because some vma cannot be copied, probably some device mapping and in turn that may be due to PAT/caching/whatever, hard to tell without instrumenting that code and seeing at which point it fails exactly.

[Dwokfur]

I've first bumped hardened sources using grsecurity-2.1.14-2.6.29.4-200906031821.patch. The problem was there. I used a vanilla kernel after that. It also has the symptom.

What's next? How I can help you to track down the problem?

since it's apparently a vanilla problem, you should post to lkml and/or the kernel bugzilla and let the kernel devs figure it out. my guess is that fork fails because some vma cannot be copied, probably some device mapping and in turn that may be due to PAT/caching/whatever, hard to tell without instrumenting that code and seeing at which point it fails exactly.

I put it wrong - I was probably too tired. I had first tried a hardened kernel bumped to the latest grsec. When it showed the symptoms, I wanted to rule out the possible role of other hardened patches. I continued with a vanilla patched using the latest grsec. I'm sorry for the inconvenience.
What instrumentation should I employ to further facilitate tracking down the problem?

Regards,
Dw.

[spender]

Could you try a vanilla kernel with the same configuration, except without grsecurity/PaX patched in?

None of the code I add to fork.c would cause fork to fail with -EAGAIN (the errno you observe), but there's this for instance:

Code: Select all

        retval = -EAGAIN;
        if (atomic_read(&p->real_cred->user->processes) >=
                        p->signal->rlim[RLIMIT_NPROC].rlim_cur) {
                if (!capable(CAP_SYS_ADMIN) && !capable(CAP_SYS_RESOURCE) &&
                    p->real_cred->user != INIT_USER)
                        goto bad_fork_free;
        }


or these cases:

Code: Select all

        retval = -EAGAIN;
        if (nr_threads >= max_threads)
                goto bad_fork_cleanup_count;

        if (!try_module_get(task_thread_info(p)->exec_domain->module))
                goto bad_fork_cleanup_count;

        if (p->binfmt && !try_module_get(p->binfmt->module))
                goto bad_fork_cleanup_put_domain;


or this:

Code: Select all

                if (fs->in_exec) {
                        write_unlock(&fs->lock);
                        return -EAGAIN;
                }


that can cause fork to fail with that errno. Some of these changes are new to 2.6.29.

If you want to debug the problem further, you can add printks to those places to determine which case is triggering the problem.

-Brad

[Dwokfur]

Could you try a vanilla kernel with the same configuration, except without grsecurity/PaX patched in?

<SNIP>

Code: Select all

                if (fs->in_exec) {
                        write_unlock(&fs->lock);
                        return -EAGAIN;
                }


that can cause fork to fail with that errno. Some of these changes are new to 2.6.29.

If you want to debug the problem further, you can add printks to those places to determine which case is triggering the problem.

-Brad

Sorry that I blamed grsecurity and thank you for pointing these out.

I'll investigate this using a vanilla kernel. Unfortunately it's a production server and I cannot use my laptop system for figuring this out. But it worth to take it down for debugging this issue.

Regards:
Dw.

[fly_b747@gmx.de]

Hello,

the solution is to deactivate PAX Features of /usr/bin/Xorg via paxctl. Then try again. It should work now.


thorsten

[Dwokfur]

Hello,

the solution is to deactivate PAX Features of /usr/bin/Xorg via paxctl. Then try again. It should work now.

thorsten

Nope.
paxctl -psEm do not solve the problem.

Thx:
Dw.

[Dwokfur]

Could you try a vanilla kernel with the same configuration, except without grsecurity/PaX patched in?

None of the code I add to fork.c would cause fork to fail with -EAGAIN (the errno you observe), but there's this for instance:

< SNIP >

that can cause fork to fail with that errno. Some of these changes are new to 2.6.29.

If you want to debug the problem further, you can add printks to those places to determine which case is triggering the problem.

-Brad

Dear Brad,

I've booted linux-2.6.30 and Xorg starts flawlessly. Xkbcomp is OK.

I've also tried linux-2.6.29.5 with the following grsec patch applied: grsecurity-2.1.14-2.6.29.5-200906152045.patch.
Xkbcomp problem is there. Xorg fails to fork. I've placed printk statements to all places you pointed out, but I could observe no messages appear in the logs.
However I have this:

Code: Select all

------------[ cut here ]------------
WARNING: at arch/x86/mm/pat.c:738 track_pfn_vma_copy+0xb9/0xd0()
Hardware name: System Name
Modules linked in: radeon i2c_amd756
Pid: 4627, comm: X Not tainted 2.6.29-hardened #3
Call Trace:
 [<0002e637>] warn_slowpath+0x87/0xe0
 [<00003046>] exit_thread+0x56/0xf0
 [<00020497>] kmap_atomic_prot+0x17/0xf0
 [<00020404>] kunmap_atomic+0x44/0xc0
 [<00065abb>] get_page_from_freelist+0x30b/0x510
 [<004862ec>] _spin_unlock+0xc/0x20
 [<000284d0>] thread_group_sched_runtime+0x20/0x90
 [<00003246>] __switch_to+0x26/0x1d0
 [<000084d0>] __constant_memcpy3d+0x30/0x140
 [<00065d9d>] __alloc_pages_internal+0xbd/0x470
 [<000284d0>] thread_group_sched_runtime+0x20/0x90
 [<00003046>] exit_thread+0x56/0xf0
 [<00003282>] __switch_to+0x62/0x1d0
 [<00071f1b>] __inc_zone_state+0x1b/0x90
 [<00003282>] __switch_to+0x62/0x1d0
 [<0001f3b9>] track_pfn_vma_copy+0xb9/0xd0
 [<004862ec>] _spin_unlock+0xc/0x20
 [<00074c80>] __pte_alloc+0xb0/0xc0
 [<0007633b>] copy_page_range+0x37b/0x390
 [<00003046>] exit_thread+0x56/0xf0
 [<001e09be>] rb_insert_color+0x6e/0x100
 [<0002c728>] dup_mm+0x228/0x400
 [<0002d31b>] copy_process+0x9eb/0xf60
 [<00039e42>] do_sigaction+0x102/0x190
 [<0002d90b>] do_fork+0x7b/0x2e0
 [<0000474a>] restore_all+0x0/0x18
 [<00002792>] sys_clone+0x32/0x40
 [<00004732>] syscall_call+0x7/0xb
 [<00003206>] __switch_to_xtra+0x126/0x140
---[ end trace 893378f1e2cf99f2 ]---
grsec: failed fork with errno -12 by /usr/bin/Xorg[X:4627] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/gdm-binary[gdm:4619] uid/euid:0/0 gid/egid:0/0
grsec: failed fork with errno -12 by /usr/bin/Xorg[X:4627] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/gdm-binary[gdm:4619] uid/euid:0/0 gid/egid:0/0


And some of these after restaring X service:

Code: Select all

Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc282000-fc292000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc292000-fc2a2000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc2a2000-fc2b2000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc2b2000-fc2c2000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc2c2000-fc2d2000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc2d2000-fc2e2000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc2e2000-fc2f2000
Jun 21 15:40:18 hostname X:4627 freeing invalid memtype fc2f2000-fc302000


I'll try to disable PAT support in kernel, but it's still not normal if that will solve it. I'll get back.

Regards:
Dw.

[Dwokfur]

Could you try a vanilla kernel with the same configuration, except without grsecurity/PaX patched in?

None of the code I add to fork.c would cause fork to fail with -EAGAIN (the errno you observe), but there's this for instance:

< SNIP >

that can cause fork to fail with that errno. Some of these changes are new to 2.6.29.

If you want to debug the problem further, you can add printks to those places to determine which case is triggering the problem.

-Brad

I'll try to disable PAT support in kernel, but it's still not normal if that will solve it. I'll get back.

Regards:
Dw.

Dear Brad,

Disabling PAT support workarounds the issue.
However it still strange, that it pops up only on the Athlon MP server and not on the Pentium M laptop with PAT enabled.

Please let me know, if there is anything else I can help with.

Regards:
Dw.