problem with icmp, ISN

[nirgil]

Hello all,

kernel 2.6.11-11 with grsecurity-2.1.5-2.6.11.9-200505121617.patch and we have problem with icmp, when I run ping from this machine to another host I get reply for some time but then the ping stop, process run but when a attached on it with strace a get ping but the the destination addresses are different from the first one o//
On this machine is monitor software like nagios and smokeping and it seems that the ping processes mixed together, a try hping2 and fping too and the result are same.

and two questions:

1) where ISN are sets ? ISN are randomized but in sysctl I have not any option in sysctl and in /proc/sys/kernel/grsecurity/ and in kernel config are nothing about ISN? I have more machine with grsec but none randomize ISN in icmp o/

ns:~# sysctl -a |grep grsec
kernel.grsecurity.grsec_lock = 0
kernel.grsecurity.destroy_unused_shm = 1
kernel.grsecurity.chroot_findtask = 1
kernel.grsecurity.dmesg = 1
kernel.grsecurity.audit_mount = 1
kernel.grsecurity.rand_tcp_src_ports = 1
kernel.grsecurity.rand_pids = 1
kernel.grsecurity.chroot_deny_sysctl = 1
kernel.grsecurity.chroot_caps = 1
kernel.grsecurity.chroot_restrict_nice = 1
kernel.grsecurity.chroot_deny_mknod = 1
kernel.grsecurity.chroot_deny_chmod = 1
kernel.grsecurity.chroot_enforce_chdir = 1
kernel.grsecurity.chroot_deny_pivot = 1
kernel.grsecurity.chroot_deny_chroot = 1
kernel.grsecurity.chroot_deny_fchdir = 1
kernel.grsecurity.chroot_deny_mount = 1
kernel.grsecurity.chroot_deny_unix = 1
kernel.grsecurity.chroot_deny_shmat = 1
kernel.grsecurity.forkfail_logging = 1
kernel.grsecurity.execve_limiting = 1
kernel.grsecurity.fifo_restrictions = 1
kernel.grsecurity.linking_restrictions = 1
kernel.osrelease = 2.6.11.11-grsec

ns:~# ls -l /proc/sys/kernel/grsecurity/
audit_mount chroot_deny_mknod chroot_deny_unix dmesg linking_restrictions
chroot_caps chroot_deny_mount chroot_enforce_chdir execve_limiting rand_pids
chroot_deny_chmod chroot_deny_pivot chroot_findtask fifo_restrictions rand_tcp_src_ports
chroot_deny_chroot chroot_deny_shmat chroot_restrict_nice forkfail_logging
chroot_deny_fchdir chroot_deny_sysctl destroy_unused_shm grsec_lock
.config

# Network Protections
#
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_RANDSRC=y
# CONFIG_GRKERNSEC_SOCKET is not set

EXAMPLE: ping from host with grsec..

PING hq (212.71.xxx.xxx) from 212.71.yyy.yyy : 56(84) bytes of data.
64 bytes from hq (212.71.xxx.xxx): icmp_seq=0 ttl=61 time=6.620 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=1 ttl=61 time=5.421 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=2 ttl=61 time=4.588 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=3 ttl=61 time=9.528 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=4 ttl=61 time=6.598 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=5 ttl=61 time=6.129 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=6 ttl=61 time=5.162 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=7 ttl=61 time=9.750 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=8 ttl=61 time=6.796 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=9 ttl=61 time=19.978 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=10 ttl=61 time=4.626 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=11 ttl=61 time=19.373 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=12 ttl=61 time=5.788 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=13 ttl=61 time=9.626 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=14 ttl=61 time=6.497 msec
64 bytes from hq (212.71.xxx.xxx): icmp_seq=15 ttl=61 time=10.583 msec

and nothing else ping process run but ping to hq is staled... o//

in other term

ns:/var# tcpdump -i eth0 -n -f 'icmp and host hq'
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
15:17:04.354908 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 2048
15:17:04.361649 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 2048
15:17:05.355715 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 2304
15:17:05.375658 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 2304
15:17:06.356596 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 2560
15:17:06.361175 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 2560
15:17:07.357425 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 2816
15:17:07.376766 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 2816
15:17:08.358284 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 3072
15:17:08.364051 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 3072
15:17:09.359155 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 3328
15:17:09.368739 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 3328
15:17:10.359984 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 3584
15:17:10.366452 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 3584
15:17:11.360877 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 3840
15:17:11.371409 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 3840
15:18:52.468171 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 0
15:18:52.475626 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 0
15:18:53.469055 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 256
15:18:53.483822 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 256
15:18:54.469916 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 512
15:18:54.475900 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 512
15:18:55.478386 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 768
15:18:55.482447 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 768
15:18:56.479631 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 1024
15:18:56.488638 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 1024
15:18:57.480490 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 1280
15:18:57.489273 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 1280
15:18:58.481343 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 1536
15:18:58.486699 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 1536
15:18:59.482195 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 1792
15:18:59.495446 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 1792
15:19:00.483044 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 2048
15:19:00.490305 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 2048
15:19:01.483906 IP 212.71.yyy.yyy > 212.71.xxx.xxx: icmp 64: echo request seq 2304
15:19:01.493230 IP 212.71.xxx.xxx > 212.71.yyy.yyy: icmp 64: echo reply seq 2304

nothing else .. o/

ns:~# ps aux |grep ping |grep hq
root 1110 0.0 0.0 1804 632 pts/2 S+ 15:16 0:00 ping hq
ns:~# strace -p 1110
Process 1110 attached - interrupt to quit
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.xxx.x")}, msg_iov(1)=[{"E\0\5\334\16)\0\0?\1m8\324G\251\6\324G\251*\0\0\351E\205"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.yyy.y")}, msg_iov(1)=[{"E\0\5\334\244\303\0\0?\1\326\240\324G\251\3\324G\251*\0"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.yyy.y")}, msg_iov(1)=[{"E\0\5\334\244\304\0\0?\1\326\237\324G\251\3\324G\251*\0"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.xxx.x")}, msg_iov(1)=[{"E\0\5\334\16*\0\0?\1m7\324G\251\6\324G\251*\0\0{`\205_"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("172.24.59.135")}, msg_iov(1)=[{"E\0\0T!
P@\0>\1\266G\254\30;\207\324G\251*\0\0\246ob,\0"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.nnn.n")}, msg_iov(1)=[{"E\0\0T\1\253\0\0=\1k\342\324G\276b\324G\251*\0\0_\21\344"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=0}, 0) = 84
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.yyy.y")}, msg_iov(1)=[{"E\0\5\334\244\305\0\0?\1\326\236\324G\251\3\324G\251*\0"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.xxx.x")}, msg_iov(1)=[{"E\0\5\334\16+\0\0?\1m6\324G\251\6\324G\251*\0\0@\353\205"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.yyy.y")}, msg_iov(1)=[{"E\0\5\334\244\306\0\0?\1\326\235\324G\251\3\324G\251*\0"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
recvmsg(3, {msg_name(16)={sa_family=AF_INET, sin_port=htons(9207), sin_addr=inet_addr("212.71.xxx.x")}, msg_iov(1)=[{"E\0\5\334\16,\0\0?\1m5\324G\251\6\324G\251*\0\0\23v\205"..., 192}], msg_controllen=20, {cmsg_len=20, cmsg_level=SOL_SOCKET, cmsg_type=0x1d /* SCM_??? */, ...}, msg_flags=MSG_TRUNC}, 0) = 192
...
</snip>

2) process run and send icmp echo request to host from strace, why when with this one process ping only one host ? Maybe ISN are mixed and process mixed together then too ?


thx for reply and excuse me my horrible english..

[lilo]

Hi

We use kernel 2.4.31 and latest grsec patch (grsecurity-2.1.6-2.4.31-200506141150.patch), and we have similar problem. (I try grsec with sysctl support, and try disable some option, but ping problem never resolved with grsec kernel).

The ping randomly going to hold, and never return to run.

reality:
- MP and SMP kernel affected
- PAX softmode=1 (and aslr=0) not help
- the "ping hold" problem never come to forward with vanilla kernel. (without grsec patch)

[spender]

Can you try the latest patches in http://grsecurity.net/~spender ?
I've fixed a problem in them that may be related to this.

-Brad