CentOS7: system fails to power off (3.14.12)

[Ashmodai]

Hi,

When issuing a 'halt' or 'poweroff' command on a CentOS 7 virtual machine with kernel 3.14.12+grsec, I get the following output:

Code: Select all

[    29.836293] grsec: denied exec of usermode helper binary  located outside of /sbin and system library paths
[    29.836494] grsec: denied exec of usermode helper binary  located outside of /sbin and system library paths
[    29.967172] grsec: denied exec of usermode helper binary  located outside of /sbin and system library paths
[    30.012212] reboot: System halted


The virtual machine is, however, not actually powered off at this point.

[PaX Team]

is the full path to the binary actually empty or did you remove it? in any case, we need to know where else outside the regular paths centos7 managed to store an important binary.

[Ashmodai]

It's actually empty - that's verbatim, there's two spaces between "binary" and "located", obviously the string is null, somehow

[spender]

Can you add a dump_stack(); just below that printk on line 282 and above the retval = -EPERM line of kernel/kmod.c and give me the output?

Thanks,
-Brad

[Ashmodai]

Hi Brad,

See below:

Code: Select all

[   14.997214] sr 2:0:0:0: Attached scsi generic sg1 type 5
[   17.101075] ip_tables: (C) 2000-2006 Netfilter Core Team
[   17.336910] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[   17.395244] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   17.516562] grsec: denied write to CPU MSR by /usr/bin/x86_energy_perf_policy[x86_energy_perf:673] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/tuned[tuned:624] uid/euid:0/0 gid/egid:0/0
[   17.537180] grsec: denied write to CPU MSR by /usr/bin/x86_energy_perf_policy[x86_energy_perf:675] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/tuned[tuned:624] uid/euid:0/0 gid/egid:0/0
[   17.555428] grsec: denied write to CPU MSR by /usr/bin/x86_energy_perf_policy[x86_energy_perf:678] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/tuned[tuned:624] uid/euid:0/0 gid/egid:0/0
[   17.567585] Ebtables v2.0 registered
[   17.573410] grsec: denied write to CPU MSR by /usr/bin/x86_energy_perf_policy[x86_energy_perf:680] uid/euid:0/0 gid/egid:0/0, parent /usr/sbin/tuned[tuned:624] uid/euid:0/0 gid/egid:0/0
[   17.645574] Bridge firewalling registered
[   18.361368] vmxnet3 0000:0b:00.0 ens192: intr type 3, mode 0, 5 vectors allocated
[   18.367693] vmxnet3 0000:0b:00.0 ens192: NIC Link is Up 10000 Mbps

CentOS Linux 7 (Core)
Kernel 3.14.12-S3-ESXiTEST on an x86_64

localhost login: [   26.710377] audit: type=1305 audit(1405079059.673:384): audit_pid=0 old=529 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1
[   26.711460] audit: type=1131 audit(1405079059.674:385): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="auditd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   26.712562] audit: type=1131 audit(1405079059.675:386): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="rhel-readonly" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   26.776026] audit: type=1131 audit(1405079059.738:387): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="lvm2-monitor" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   26.782956] audit: type=1131 audit(1405079059.745:388): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="lvm2-lvmetad" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   26.812073] audit: type=1131 audit(1405079059.774:389): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="systemd-remount-fs" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[   26.864622] systemd-journald[413]: Received SIGTERM
[   26.866251] grsec: denied exec of usermode helper binary  located outside of /sbin and system library paths
[   26.868570] CPU: 1 PID: 2005 Comm: kworker/u8:1 Not tainted 3.14.12-S3-ESXiTEST #6
[   26.870244] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
[   26.872509]  0000000000000000 925d678b2d867f0d ffffffff81720423 ffff88007a1c8d50
[   26.874596]  ffffffff810723f9 ffff880037388880 ffffffff81072420 ffff880072de3b80
[   26.876617]  0000000000000000 ffffffff81729194 0000000000000000 0000000000000000
[   26.878607] Call Trace:
[   26.879282]  [<ffffffff81720423>] ? dump_stack+0x41/0x57
[   26.880511]  [<ffffffff810723f9>] ? ____call_usermodehelper+0x229/0x250
[   26.882066]  [<ffffffff81072420>] ? ____call_usermodehelper+0x250/0x250
[   26.883603]  [<ffffffff81729194>] ? ret_from_fork+0x74/0xa0
[   26.884903]  [<ffffffff81072420>] ? ____call_usermodehelper+0x250/0x250
[   26.886524] grsec: denied exec of usermode helper binary  located outside of /sbin and system library paths
[   26.888676] CPU: 1 PID: 2006 Comm: kworker/u8:1 Not tainted 3.14.12-S3-ESXiTEST #6
[   26.890343] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
[   26.892913]  0000000000000000 a026fccc18c59f1c ffffffff81720423 ffff88007291b690
[   26.894921]  ffffffff810723f9 ffff880037388880 ffffffff81072420 ffff880072de3b80
[   26.896902]  0000000000000000 ffffffff81729194 0000000000000000 0000000000000000
[   26.898824] Call Trace:
[   26.899445]  [<ffffffff81720423>] ? dump_stack+0x41/0x57
[   26.900663]  [<ffffffff810723f9>] ? ____call_usermodehelper+0x229/0x250
[   26.902247]  [<ffffffff81072420>] ? ____call_usermodehelper+0x250/0x250
[   26.903738]  [<ffffffff81729194>] ? ret_from_fork+0x74/0xa0
[   26.905014]  [<ffffffff81072420>] ? ____call_usermodehelper+0x250/0x250
[   27.005569] dracut Warning: Killing all remaining processes
[   27.028266] grsec: denied exec of usermode helper binary  located outside of /sbin and system library paths
[   27.030501] CPU: 0 PID: 2037 Comm: kworker/u8:1 Not tainted 3.14.12-S3-ESXiTEST #6
[   27.032197] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/30/2013
[   27.034461]  0000000000000000 578e26304d7cabda ffffffff81720423 ffff880072ced650
[   27.036317]  ffffffff810723f9 ffff88007ad8d540 ffffffff81072420 ffff88007a08ea00
[   27.038261]  0000000000000000 ffffffff81729194 0000000000000000 0000000000000000
[   27.040325] Call Trace:
[   27.041179]  [<ffffffff81720423>] ? dump_stack+0x41/0x57
[   27.042407]  [<ffffffff810723f9>] ? ____call_usermodehelper+0x229/0x250
[   27.043891]  [<ffffffff81072420>] ? ____call_usermodehelper+0x250/0x250
[   27.045297]  [<ffffffff81729194>] ? ret_from_fork+0x74/0xa0
[   27.046505]  [<ffffffff81072420>] ? ____call_usermodehelper+0x250/0x250
[   27.062634] dracut Warning: Unmounted /oldroot.
[   27.078771] dracut: Disassembling device-mapper devices
[   27.092185] reboot: System halted


Also, the system not powering off after issuing 'halt' seems to be a CentOS bug/(feature?) -- issuing poweroff does the Right Thing(tm). In addition, note the denied access to CPU MSR by x86_energy_perf_policy. Not sure if this is something you care about,and I'm not sure these denials are actually having any real impact on anything - it seems it might just be noise.

[strcat]

Also, the system not powering off after issuing 'halt' seems to be a CentOS bug/(feature?) -- issuing poweroff does the Right Thing(tm). In addition, note the denied access to CPU MSR by x86_energy_perf_policy. Not sure if this is something you care about,and I'm not sure these denials are actually having any real impact on anything - it seems it might just be noise.

It's intended as a feature. I think the motivation is probably to make it easier to debug a broken shutdown process.