Kernel 2.4.31 + grsecurity 2.1.6 SMP server freezes.

Discuss usability issues, general maintenance, and general support issues for a grsecurity-enabled system.

Postby bplant » Tue Nov 08, 2005 3:12 am

My server Freezes itself with an average uptime of 4 weeks and my auditing operations reveals that mostly happened in the early morning... :O)

Mumble.. mumble.. In early morning my server performs a mirror of himself for delivery reasons and this operation makes a strong use of "wget".
My mirror script performs its work in an aggressive mode over several instances of jetty and tomcat (both are java processes).


I had a similar issue. Every morning cron would run rsync to update the portage tree (using Gentoo Linux). The machine froze after approximately 1 week. After freezing twice, I booted a standard Gentoo kernel and the machine hasn't missed a beat since (~3 months).

I do have exec logging turned on.... just in case that indicates an issue with the locks in the logging code.

What is interesting is that I have a mirror of the problematic machine and it has been running grsec 2.1.6 for ~4 months without any issues. The only difference between them is their hardware and as such I compiled the different kernels accordingly.

Cheers,

Brad
bplant
 
Posts: 73
Joined: Sat May 28, 2005 10:36 pm

Freeze differences

Postby Kp » Thu Nov 10, 2005 1:38 am

bplant wrote:I had a similar issue. Every morning cron would run rsync to update the portage tree (using Gentoo Linux). The machine froze after approximately 1 week. After freezing twice, I booted a standard Gentoo kernel and the machine hasn't missed a beat since (~3 months).

I do have exec logging turned on.... just in case that indicates an issue with the locks in the logging code.

What is interesting is that I have a mirror of the problematic machine and it has been running grsec 2.1.6 for ~4 months without any issues. The only difference between them is their hardware and as such I compiled the different kernels accordingly.


Could you post a unified diff of the two kernel configurations? It might be easier to track the SMP freeze problems (if any exist in the latest version of GRsec) if the developers could see what features you turned off on the system that's stable. Also, if the stable kernel is capable of running on the troubled machine (from your wording, it may lack support for some of the hardware?), you should try it there. If it works, then it's definitely a combination of GRsec options that's getting you.
Kp
 
Posts: 46
Joined: Tue Sep 20, 2005 12:56 am

Postby bplant » Thu Nov 10, 2005 2:17 am

Could you post a unified diff of the two kernel configurations? It might be easier to track the SMP freeze problems (if any exist in the latest version of GRsec) if the developers could see what features you turned off on the system that's stable. Also, if the stable kernel is capable of running on the troubled machine (from your wording, it may lack support for some of the hardware?), you should try it there. If it works, then it's definitely a combination of GRsec options that's getting you.


I actually have SMP support disabled in both kernel configurations. Also, on boomer, (the pentium 4), hyperthreading has been disabled in the BIOS. Boomer is the troublesome machine, while Radon (Pentium 3) works flawlessly. Unfortunately the stable kernel (Radon) will not work on Boomer because it lacks the appropriate drivers.

Here is the unified diff between configurations as requested:

Code: Select all
--- radon.kernel_config   2005-11-10 17:22:16.000000000 +1100
+++ boomer.kernel_config   2005-11-10 17:22:21.000000000 +1100
@@ -1,5 +1,5 @@
 #
-# Automatically generated make config: don't edit
+# Automatically generated by make menuconfig: don't edit
 #
 CONFIG_X86=y
 # CONFIG_SBUS is not set
@@ -24,8 +24,8 @@
 # CONFIG_M586TSC is not set
 # CONFIG_M586MMX is not set
 # CONFIG_M686 is not set
-CONFIG_MPENTIUMIII=y
-# CONFIG_MPENTIUM4 is not set
+# CONFIG_MPENTIUMIII is not set
+CONFIG_MPENTIUM4=y
 # CONFIG_MK6 is not set
 # CONFIG_MK7 is not set
 # CONFIG_MK8 is not set
@@ -45,7 +45,7 @@
 CONFIG_X86_POPAD_OK=y
 # CONFIG_RWSEM_GENERIC_SPINLOCK is not set
 CONFIG_RWSEM_XCHGADD_ALGORITHM=y
-CONFIG_X86_L1_CACHE_SHIFT=5
+CONFIG_X86_L1_CACHE_SHIFT=7
 CONFIG_X86_ALIGNMENT_16=y
 CONFIG_X86_HAS_TSC=y
 CONFIG_X86_GOOD_APIC=y
@@ -248,10 +248,6 @@
 # CONFIG_IP_SCTP is not set
 # CONFIG_ATM is not set
 # CONFIG_VLAN_8021Q is not set
-
-#
-# 
-#
 # CONFIG_IPX is not set
 # CONFIG_ATALK is not set
 # CONFIG_DECNET is not set
@@ -291,10 +287,6 @@
 # IDE, ATA and ATAPI Block devices
 #
 CONFIG_BLK_DEV_IDE=y
-
-#
-# Please see Documentation/ide.txt for help/info on IDE drives
-#
 # CONFIG_BLK_DEV_HD_IDE is not set
 # CONFIG_BLK_DEV_HD is not set
 # CONFIG_BLK_DEV_IDE_SATA is not set
@@ -308,10 +300,6 @@
 # CONFIG_BLK_DEV_IDEFLOPPY is not set
 # CONFIG_BLK_DEV_IDESCSI is not set
 # CONFIG_IDE_TASK_IOCTL is not set
-
-#
-# IDE chipset support/bugfixes
-#
 # CONFIG_BLK_DEV_CMD640 is not set
 # CONFIG_BLK_DEV_CMD640_ENHANCED is not set
 # CONFIG_BLK_DEV_ISAPNP is not set
@@ -349,7 +337,7 @@
 # CONFIG_BLK_DEV_SC1200 is not set
 # CONFIG_BLK_DEV_SVWKS is not set
 # CONFIG_BLK_DEV_SIIMAGE is not set
-CONFIG_BLK_DEV_SIS5513=y
+# CONFIG_BLK_DEV_SIS5513 is not set
 # CONFIG_BLK_DEV_SLC90E66 is not set
 # CONFIG_BLK_DEV_TRM290 is not set
 # CONFIG_BLK_DEV_VIA82CXXX is not set
@@ -366,7 +354,85 @@
 #
 # SCSI support
 #
-# CONFIG_SCSI is not set
+CONFIG_SCSI=y
+CONFIG_BLK_DEV_SD=y
+CONFIG_SD_EXTRA_DEVS=40
+# CONFIG_CHR_DEV_ST is not set
+# CONFIG_CHR_DEV_OSST is not set
+# CONFIG_BLK_DEV_SR is not set
+CONFIG_CHR_DEV_SG=y
+# CONFIG_SCSI_DEBUG_QUEUES is not set
+# CONFIG_SCSI_MULTI_LUN is not set
+# CONFIG_SCSI_CONSTANTS is not set
+# CONFIG_SCSI_LOGGING is not set
+
+#
+# SCSI low-level drivers
+#
+# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
+# CONFIG_SCSI_7000FASST is not set
+# CONFIG_SCSI_ACARD is not set
+# CONFIG_SCSI_AHA152X is not set
+# CONFIG_SCSI_AHA1542 is not set
+# CONFIG_SCSI_AHA1740 is not set
+# CONFIG_SCSI_AACRAID is not set
+# CONFIG_SCSI_AIC7XXX is not set
+# CONFIG_SCSI_AIC79XX is not set
+# CONFIG_SCSI_AIC7XXX_OLD is not set
+# CONFIG_SCSI_DPT_I2O is not set
+# CONFIG_SCSI_ADVANSYS is not set
+# CONFIG_SCSI_IN2000 is not set
+# CONFIG_SCSI_AM53C974 is not set
+# CONFIG_SCSI_MEGARAID is not set
+# CONFIG_SCSI_MEGARAID2 is not set
+CONFIG_SCSI_SATA=y
+# CONFIG_SCSI_SATA_AHCI is not set
+# CONFIG_SCSI_SATA_SVW is not set
+CONFIG_SCSI_ATA_PIIX=y
+# CONFIG_SCSI_SATA_NV is not set
+# CONFIG_SCSI_SATA_QSTOR is not set
+# CONFIG_SCSI_SATA_PROMISE is not set
+# CONFIG_SCSI_SATA_SX4 is not set
+# CONFIG_SCSI_SATA_SIL is not set
+# CONFIG_SCSI_SATA_SIS is not set
+# CONFIG_SCSI_SATA_ULI is not set
+# CONFIG_SCSI_SATA_VIA is not set
+# CONFIG_SCSI_SATA_VITESSE is not set
+# CONFIG_SCSI_BUSLOGIC is not set
+# CONFIG_SCSI_CPQFCTS is not set
+# CONFIG_SCSI_DMX3191D is not set
+# CONFIG_SCSI_DTC3280 is not set
+# CONFIG_SCSI_EATA is not set
+# CONFIG_SCSI_EATA_DMA is not set
+# CONFIG_SCSI_EATA_PIO is not set
+# CONFIG_SCSI_FUTURE_DOMAIN is not set
+# CONFIG_SCSI_GDTH is not set
+# CONFIG_SCSI_GENERIC_NCR5380 is not set
+# CONFIG_SCSI_IPS is not set
+# CONFIG_SCSI_INITIO is not set
+# CONFIG_SCSI_INIA100 is not set
+# CONFIG_SCSI_NCR53C406A is not set
+# CONFIG_SCSI_NCR53C7xx is not set
+# CONFIG_SCSI_SYM53C8XX_2 is not set
+# CONFIG_SCSI_NCR53C8XX is not set
+# CONFIG_SCSI_SYM53C8XX is not set
+# CONFIG_SCSI_PAS16 is not set
+# CONFIG_SCSI_PCI2000 is not set
+# CONFIG_SCSI_PCI2220I is not set
+# CONFIG_SCSI_PSI240I is not set
+# CONFIG_SCSI_QLOGIC_FAS is not set
+# CONFIG_SCSI_QLOGIC_ISP is not set
+# CONFIG_SCSI_QLOGIC_FC is not set
+# CONFIG_SCSI_QLOGIC_1280 is not set
+# CONFIG_SCSI_SEAGATE is not set
+# CONFIG_SCSI_SIM710 is not set
+# CONFIG_SCSI_SYM53C416 is not set
+# CONFIG_SCSI_DC390T is not set
+# CONFIG_SCSI_T128 is not set
+# CONFIG_SCSI_U14_34F is not set
+# CONFIG_SCSI_ULTRASTOR is not set
+# CONFIG_SCSI_NSP32 is not set
+# CONFIG_SCSI_DEBUG is not set
 
 #
 # Fusion MPT device support
@@ -412,57 +478,7 @@
 #
 # Ethernet (10 or 100Mbit)
 #
-CONFIG_NET_ETHERNET=y
-# CONFIG_SUNLANCE is not set
-# CONFIG_HAPPYMEAL is not set
-# CONFIG_SUNBMAC is not set
-# CONFIG_SUNQE is not set
-# CONFIG_SUNGEM is not set
-# CONFIG_NET_VENDOR_3COM is not set
-# CONFIG_LANCE is not set
-# CONFIG_NET_VENDOR_SMC is not set
-# CONFIG_NET_VENDOR_RACAL is not set
-# CONFIG_AT1700 is not set
-# CONFIG_DEPCA is not set
-# CONFIG_HP100 is not set
-# CONFIG_NET_ISA is not set
-CONFIG_NET_PCI=y
-# CONFIG_PCNET32 is not set
-# CONFIG_AMD8111_ETH is not set
-# CONFIG_ADAPTEC_STARFIRE is not set
-# CONFIG_AC3200 is not set
-# CONFIG_APRICOT is not set
-# CONFIG_B44 is not set
-# CONFIG_CS89x0 is not set
-# CONFIG_TULIP is not set
-# CONFIG_DE4X5 is not set
-# CONFIG_DGRS is not set
-# CONFIG_DM9102 is not set
-# CONFIG_EEPRO100 is not set
-# CONFIG_EEPRO100_PIO is not set
-# CONFIG_E100 is not set
-# CONFIG_LNE390 is not set
-# CONFIG_FEALNX is not set
-# CONFIG_NATSEMI is not set
-# CONFIG_NE2K_PCI is not set
-# CONFIG_FORCEDETH is not set
-# CONFIG_NE3210 is not set
-# CONFIG_ES3210 is not set
-# CONFIG_8139CP is not set
-CONFIG_8139TOO=y
-# CONFIG_8139TOO_PIO is not set
-# CONFIG_8139TOO_TUNE_TWISTER is not set
-# CONFIG_8139TOO_8129 is not set
-# CONFIG_8139_OLD_RX_RESET is not set
-# CONFIG_SIS900 is not set
-# CONFIG_EPIC100 is not set
-# CONFIG_SUNDANCE is not set
-# CONFIG_SUNDANCE_MMIO is not set
-# CONFIG_TLAN is not set
-# CONFIG_VIA_RHINE is not set
-# CONFIG_VIA_RHINE_MMIO is not set
-# CONFIG_WINBOND_840 is not set
-# CONFIG_NET_POCKET is not set
+# CONFIG_NET_ETHERNET is not set
 
 #
 # Ethernet (1000 Mbit)
@@ -474,7 +490,7 @@
 # CONFIG_NS83820 is not set
 # CONFIG_HAMACHI is not set
 # CONFIG_YELLOWFIN is not set
-# CONFIG_R8169 is not set
+CONFIG_R8169=y
 # CONFIG_SK98LIN is not set
 # CONFIG_TIGON3 is not set
 # CONFIG_FDDI is not set
@@ -556,14 +572,6 @@
 # Joysticks
 #
 # CONFIG_INPUT_GAMEPORT is not set
-
-#
-# Input core support is needed for gameports
-#
-
-#
-# Input core support is needed for joysticks
-#
 # CONFIG_QIC02_TAPE is not set
 # CONFIG_IPMI_HANDLER is not set
 # CONFIG_IPMI_PANIC_EVENT is not set


Note the Grsec settings are identical. I have listed them below:

Code: Select all
#
# Grsecurity
#
CONFIG_GRKERNSEC=y
CONFIG_CRYPTO=y
CONFIG_CRYPTO_SHA256=y
# CONFIG_GRKERNSEC_LOW is not set
# CONFIG_GRKERNSEC_MID is not set
# CONFIG_GRKERNSEC_HI is not set
CONFIG_GRKERNSEC_CUSTOM=y

#
# PaX Control
#
# CONFIG_GRKERNSEC_PAX_SOFTMODE is not set
CONFIG_GRKERNSEC_PAX_EI_PAX=y
CONFIG_GRKERNSEC_PAX_PT_PAX_FLAGS=y
# CONFIG_GRKERNSEC_PAX_NO_ACL_FLAGS is not set
CONFIG_GRKERNSEC_PAX_HAVE_ACL_FLAGS=y
# CONFIG_GRKERNSEC_PAX_HOOK_ACL_FLAGS is not set

#
# Address Space Protection
#
CONFIG_GRKERNSEC_PAX_NOEXEC=y
# CONFIG_GRKERNSEC_PAX_PAGEEXEC is not set
CONFIG_GRKERNSEC_PAX_SEGMEXEC=y
# CONFIG_GRKERNSEC_PAX_EMUTRAMP is not set
CONFIG_GRKERNSEC_PAX_MPROTECT=y
# CONFIG_GRKERNSEC_PAX_NOELFRELOCS is not set
CONFIG_GRKERNSEC_PAX_ASLR=y
CONFIG_GRKERNSEC_PAX_RANDKSTACK=y
CONFIG_GRKERNSEC_PAX_RANDUSTACK=y
CONFIG_GRKERNSEC_PAX_RANDMMAP=y
CONFIG_GRKERNSEC_KMEM=y
CONFIG_GRKERNSEC_IO=y
CONFIG_RTC=y
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_HIDESYM=y

#
# Role Based Access Control Options
#
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
CONFIG_GRKERNSEC_ACL_TIMEOUT=30

#
# Filesystem Protections
#
CONFIG_GRKERNSEC_PROC=y
CONFIG_GRKERNSEC_PROC_USER=y
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_MOUNT=y
CONFIG_GRKERNSEC_CHROOT_DOUBLE=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_CHMOD=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_FINDTASK=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_CHROOT_CAPS=y

#
# Kernel Auditing
#
# CONFIG_GRKERNSEC_AUDIT_GROUP is not set
CONFIG_GRKERNSEC_EXECLOG=y
CONFIG_GRKERNSEC_RESLOG=y
CONFIG_GRKERNSEC_CHROOT_EXECLOG=y
CONFIG_GRKERNSEC_AUDIT_CHDIR=y
CONFIG_GRKERNSEC_AUDIT_MOUNT=y
CONFIG_GRKERNSEC_AUDIT_IPC=y
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_TIME=y
CONFIG_GRKERNSEC_PROC_IPADDR=y
CONFIG_GRKERNSEC_AUDIT_TEXTREL=y

#
# Executable Protections
#
CONFIG_GRKERNSEC_EXECVE=y
CONFIG_GRKERNSEC_SHM=y
CONFIG_GRKERNSEC_DMESG=y
CONFIG_GRKERNSEC_RANDPID=y
CONFIG_GRKERNSEC_TPE=y
CONFIG_GRKERNSEC_TPE_ALL=y
CONFIG_GRKERNSEC_TPE_INVERT=y
CONFIG_GRKERNSEC_TPE_GID=1005

#
# Network Protections
#
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_RANDSRC=y
CONFIG_GRKERNSEC_SOCKET=y
# CONFIG_GRKERNSEC_SOCKET_ALL is not set
# CONFIG_GRKERNSEC_SOCKET_CLIENT is not set
# CONFIG_GRKERNSEC_SOCKET_SERVER is not set

#
# Sysctl support
#
CONFIG_GRKERNSEC_SYSCTL=y
CONFIG_GRKERNSEC_SYSCTL_ON=y

#
# Logging options
#
CONFIG_GRKERNSEC_FLOODTIME=5
CONFIG_GRKERNSEC_FLOODBURST=10


Thanks for your reply,

Brad
bplant
 
Posts: 73
Joined: Sat May 28, 2005 10:36 pm

Previous

Return to grsecurity support