Performance improvements to chroot restrictions

[spender]

I've just uploaded a patch to:
http://grsecurity.net/~spender/new_chroot.patch

Based on some feedback I received from a user on IRC about reduced interactivity during heavy activity within chroots, I've written the chroot detection/root comparison code to eliminate any such performance hit. I've tested it locally here and ran it through all the chroot regression tests successfully, but I'd like it to receive more widespread testing before I put it in the stable tree. Especially if you've noticed the aforementioned performance hit, I'd like to hear your feedback on the patch. The patch at the address above applies on top of the latest 2.6.32.15 grsecurity patch.

-Brad

[Grach]

The patch doesn't change anything here. An interactivity is being killed with any more or less serious activity inside chroot, like emerge -uNDp world. Sorry for such late responce.

[spender]

What about a normal kernel compile inside the chroot? And the interactivity hit goes away when disabling CHROOT_FINDTASK for you?

-Brad

[spender]

Could you also try some profiling of the interactivity using latencytop and show me the results?

-Brad

[Grach]

Kernel compile causes the same interactivity kill. And yes, disabling chroot_findtask removes any lags completely. Btw, even when I just start a shell inside chroot from any X terminal, it works like enter key is jammed for a while, like this:

Code: Select all

# chroot . bash -l
#
#
#
#
#

I also observe this key jamming behavior during kernel compilation inside chroot when working with ssh sessions to another hosts started from an X terminal, so it's clearly the interactivity related issue. Keypresses and mouse pointer movement are lagging. The same behavior I was observing since my start with grsecurty, so nothing changed. And I double-checked I have new_chroot.patch applied and testing on the right kernel, so no mistake:

Code: Select all

# uname -a
Linux worm3 2.6.32.15-grsec-2.2.0-201006271253 #1 Fri Jul 9 22:16:48 KRAST 2010 i686 AMD Athlon(TM) XP 2500+ AuthenticAMD GNU/Linux
# pwd
/usr/src/linux-2.6.32.15-grsec-2.2.0-201006271253
# cmp arch/x86/boot/bzImage /boot/vmlinuz-2.6.32.15-grsec-2.2.0-201006271253
# patch -R -p1 < /path/to/new_chroot.patch
patching file fs/fs_struct.c
patching file grsecurity/gracl.c
patching file grsecurity/grsec_chroot.c
patching file grsecurity/grsec_disabled.c
patching file include/linux/grinternal.h
patching file include/linux/grsecurity.h
patching file include/linux/init_task.h
patching file include/linux/sched.h
patching file kernel/fork.c


Now I'm gonna recompile the kernel with CONFIG_LATENCYTOP (with new_chroot.patch reapplied ;) and then post latencytop results.

[Grach]

OK, I've measured... something. I was starting a kernel compilation inside chroot and moving the mouse all around.
With findtask:

Code: Select all

Cause                                                Maximum     Percentage
Reading EXT3 block bitmaps                         25.9 msec          0.5 %
Fork() system call                                  7.5 msec          2.5 %
Scheduler: waiting for cpu                          7.3 msec         63.7 %
Waiting for event (select)                          5.0 msec         10.2 %
Waiting for event (poll)                            5.0 msec         10.0 %
Userspace lock contention                           4.9 msec          1.5 %
Waiting for a process to die                        4.9 msec          6.6 %
Waiting for event (epoll)                           4.7 msec          1.6 %
Reading from a pipe                                 4.5 msec          3.3 %


Process Xorg (2035)                        Total: 1850.1 msec                   
Scheduler: waiting for cpu                          6.0 msec         99.2 %
Waiting for event (select)                          3.6 msec          0.8 %


Without it:

Code: Select all

Cause                                                Maximum     Percentage
Scheduler: waiting for cpu                          7.7 msec         68.0 %
Fork() system call                                  7.1 msec          1.3 %
Waiting for event (select)                          5.0 msec         19.6 %
Waiting for event (poll)                            4.9 msec          7.4 %
Waiting for a process to die                        4.8 msec          1.8 %
Waiting for event (epoll)                           4.1 msec          0.3 %
Reading from a pipe                                 4.1 msec          0.5 %
Userspace lock contention                           3.0 msec          1.0 %

Process Xorg (2035)                        Total: 3469.8 msec                   
Scheduler: waiting for cpu                          6.9 msec         98.3 %
Waiting for event (select)                          4.8 msec          1.7 %

The data differs from time to time, so I'm not sure what I've measured and if it's useful at all. Should I measure somehow/something particular? And should I build a kernel with debugfs support (seems like latencytop wants it, but works without it)?

[Grach]

Oh, I should press r repeatedly to make it refresh the stats... How convenient. So I kept r pressed for a while and then released it. That made latencytop to refresh its stats over and over, and while it was refreshing, there was no lags. Magically. And its stats was very similar with and without findtask.

[spender]

Could you mail me your kernel config?

-Brad

[Grach]

Mailed it.

[spender]

In grsecurity/grsec_chroot.c:gr_pid_is_chrooted():

Before the "return 1;" line, can you add:
dump_stack();

perform the build again inside the chroot (it'll be considerably slower this time) and mail me those compressed logs. In the case where the task isn't chrooted, CHROOT_FINDTASK should be equivalent to having it disabled (it's only a few more instructions). I imagine this interactivity hit is coming from some user of find_task_by_*() when CHROOT_FINDTASK tells the caller the process doesn't exist. By dumping the call stack, it will tell me which areas of the kernel are doing this, and I can compare it against my own logs here.

-Brad

[spender]

Thanks for your help in debugging it. I've uploaded new patches that, based on the logs you sent me, I am pretty confident will resolve the issue.

-Brad

[spender]

Just an update for anyone following this thread:

We confirmed yesterday that the latest patches remove any interactivity hit from the CHROOT_FINDTASK option. We've also resolved the bug that caused /proc entries to disappear within a chroot under memory pressure with CHROOT_FINDTASK.

-Brad