grsecurity forums

Hi,

I am running grsecurity for personal convenience on my workstation.

While trying to run scylladb the product made by my current customer I got the following trace.

The setup is an fedora 23 lxc container running on an ubuntu 4.3.5 grsec host.

mprotect(0x348c50da000, 12288, PROT_READ) = 0
mprotect(0x348c52de000, 4096, PROT_READ) = 0
mprotect(0x1c2d000, 4096, PROT_READ) = 0
mprotect(0x348c5500000, 4096, PROT_READ) = 0
munmap(0x348c54f1000, 37083) = 0
set_tid_address(0x348c54db750) = 9874
set_robust_list(0x348c54db760, 24) = 0
rt_sigaction(SIGRTMIN, {0x348c13bdb20, [], SA_RESTORER|SA_SIGINFO, 0x348c13c8430}, NULL, = 0
rt_sigaction(SIGRT_1, {0x348c13bdbb0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x348c13c8430}, NULL, = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
statfs("/sys/fs/selinux", 0x38d24f612e0) = -1 ENOENT (No such file or directory)
statfs("/selinux", {f_type="EXT2_SUPER_MAGIC", f_bsize=4096, f_blocks=49077591, f_bfree=41930274, f_bavail=39431510, f_files=12476416, f_ffree=11989493, f_fsid={861819776, -1156077281}, f_namelen=255, f_frsize=4096, f_flags=4128}) = 0
mmap(NULL, 35184372088832, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = -1 ENOMEM (Cannot allocate memory)
rt_sigprocmask(SIG_UNBLOCK, [ABRT], NULL, = 0
tgkill(9874, 9874, SIGABRT) = 0
--- SIGABRT {si_signo=SIGABRT, si_code=SI_TKILL, si_pid=9874, si_uid=0} ---
+++ killed by SIGABRT (core dumped) +++
Abandon (core dumped)

So the Scylla database is failing to mmap anonymous memory and core dumping.

Any idea why mmap is failling this way ?

Best regards

Benoît Canet

does this fail on a vanilla kernel? other than that, we don't restrict mmap sizes or rlimits or overcommit ourselves, so maybe check those first.

edit: actually there's one restriction for such big sizes, if you have UDEREF enabled then the userland address space size will be reduced to 42 bits (from 47) thus a 45 bit allocation won't fit.

Ok I will test vanilla and lxc on friday.
(I am currently on travel an jailed in the container).

and also removing uderef.

Thanks

Best regards

Benoît

if you know you had UDEREF in the kernel then i'm pretty sure that's the limit you ran into.

I previously just disabled my grsecurity kernel and now a few month later I tried again with 4.6.4
while removing UDEREF.

Same behavior.

on amd64 KERNEXEC also reduces the userland address space size (as it's a side effect of the underlying per-cpu pgd feature).

Thanks a lot it work when removing KERNEXEC and UDEREF from the kernel config.