php-cgi and nonexisting connections to udp/80 (and udp/0)?

[mnalis]

I'm (sometimes) getting strange "ghost" udp/80 connections in addition to regular tcp/80 ones, has anyone seen this?

Code: Select all

Mar 20 19:28:34 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:20185] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19891] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:34 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:20185] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19891] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:34 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type stream protocol tcp by /usr/bin/php5-cgi[php-cgi:20185] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19891] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:34 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type stream protocol tcp by /usr/bin/php5-cgi[php-cgi:20185] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19891] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:43 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:20468] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19693] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:43 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:20468] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19693] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:43 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type stream protocol tcp by /usr/bin/php5-cgi[php-cgi:20468] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19693] uid/euid:33/33 gid/egid:33/33
Mar 20 19:28:43 data kernel: grsec: From 93.138.118.36: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type stream protocol tcp by /usr/bin/php5-cgi[php-cgi:20468] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:19693] uid/euid:33/33 gid/egid:33/33



kernel is 2.6.32.57-grsec201202200919

i have rule which blocks most connections:

Code: Select all

subject /usr/bin/php5-cgi AKCdT
[...]
bind disabled
connect 192.168.200.254/32:53 dgram udp
connect 192.168.200.254/32:53 stream tcp
#connect 72.233.56.128/25:80 stream tcp  # wordpress.org


if I uncomment the tcp lime (commented out above), I still get udp "grsec denied" messages.

The machine in question (two of them actually, with different sites but similar error) has few wordpress sites (running on apache with suexec+php-cgi), who as far as I can tell from time to time seem to want to do a normal HTTP POST on api.wordpress.org, which looks like it boils down to doing PHP fsockopen() call. So there should be no udp/80 connections, only tcp/80 ones.

I've also noticed I get a few other strange errors, like

Code: Select all

grsec: From 66.249.66.228: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:16382] uid/euid:36334/36334 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:13691] uid/euid:33/33 gid/egid:33/33
grsec: From 88.207.10.227: (virtual:G:/usr/bin/php5-cgi) denied connect() to 173.194.35.144 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:12527] uid/euid:36334/36334 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:10382] uid/euid:33/33 gid/egid:33/33


Now, I don't think udp/0 is normally allowed port anyway... Anyway I've looked over and the sites in question do not seem to be cracked. Any ideas about what may be the problem? I tried allowing them through grsec, but only udp packets that tcpdump sees are standard udp/53 DNS and my udp/514 remote syslog and udp/161 snmp queries, none of this 0.0.0.0 udp/0 or udp/80 "ghost" stuff.

Is this a possible glitch in grsecurity patch?

[spender]

I'll look into this, thanks for the report.

-Brad

[mnalis]

Just some more confirmations. With allowed only TCP/80 connections (uncommented line in example above) when trying to login to worpress /wp-admin URL, it fails (giving me "internal error..." in web browser, probably due to empty page output) and log shows:

Code: Select all

Mar 21 13:41:07 data kernel: grsec: From 161.53.11.142: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:1656] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:1654] uid/euid:33/33 gid/egid:33/33

I could reproduce it at will by visiting that URL. I then commented out all network related stuff for that subject from policy and reloaded grsec, and sure enough, site works and tcpdump show only TCP/80 connection and /wp-admin URL opens normally.

Code: Select all

data# tcpdump -np net 72.233.56.128/25 and port 80                     
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 
13:57:15.948336 IP data.54306 > 72.233.56.139.80: Flags [S], seq 1839531018, win 5840, options [mss 1460,sackOK,TS val 221097280 ecr 0,nop,wscale 7], length 0
13:57:16.091742 IP 72.233.56.139.80 > data.54306: Flags [S.], seq 3323661475, ack 1839531019, win 5792, options [mss 1460,sackOK,TS val 2609410299 ecr 221097280,nop,wsca
13:57:16.091755 IP data.54306 > 72.233.56.139.80: Flags [.], ack 1, win 46, options [nop,nop,TS val 221097316 ecr 2609410299], length 0
13:57:16.091872 IP data.54306 > 72.233.56.139.80: Flags [P.], seq 1:418, ack 1, win 46, options [nop,nop,TS val 221097316 ecr 2609410299], length 417
13:57:16.240307 IP 72.233.56.139.80 > data.54306: Flags [.], ack 418, win 14, options [nop,nop,TS val 2609410336 ecr 221097316], length 0
13:57:16.241617 IP 72.233.56.139.80 > data.54306: Flags [P.], seq 1:487, ack 418, win 14, options [nop,nop,TS val 2609410336 ecr 221097316], length 486
13:57:16.241624 IP data.54306 > 72.233.56.139.80: Flags [.], ack 487, win 54, options [nop,nop,TS val 221097353 ecr 2609410336], length 0
13:57:16.241628 IP 72.233.56.139.80 > data.54306: Flags [F.], seq 487, ack 418, win 14, options [nop,nop,TS val 2609410336 ecr 221097316], length 0
13:57:16.241731 IP data.54306 > 72.233.56.139.80: Flags [F.], seq 418, ack 488, win 54, options [nop,nop,TS val 221097353 ecr 2609410336], length 0
13:57:16.395964 IP 72.233.56.139.80 > data.54306: Flags [.], ack 419, win 14, options [nop,nop,TS val 2609410375 ecr 221097353], length 0


However, I'll have to try to make smaller reproduceable example if needed, as when I re-enabled the same network related policies for that subject and reloaded grsec, the problem disappeared and site worked -- but after a quick check that is due the wordpress caching that it recently contacted the site and deciding not to contact api.wordpress.org again. I found URL /wp-admin/update-core.php when I can force recheck, and that one also generates the "grsec denied udp/80" messages, but strangely enough, does not break the page (as the auto-triggered one does) but updates the last checked timestamp... but that might be due to different codepaths in PHP, as grsec still logs the same error.

Let me know if I can help with some more data/tests.

[spender]

To grsecurity/gracl_ip.c, just under the line:

Code: Select all

exit_fail:

could you add:

Code: Select all

dump_stack();

and give me the call traces of the places triggering the errors?

Thanks,
-Brad

[mnalis]

Hm, I did (also upgrading to 2.6.32.59-grsec201203212033 in the process), but it doesn't seem to log anything more than before:

Code: Select all

Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 173.194.35.144 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 173.194.35.146 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 173.194.35.145 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 173.194.35.147 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13007] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33
Mar 22 18:05:23 data kernel: grsec: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:13304] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11966] uid/euid:33/33 gid/egid:33/33


it was in the end of gr_search_socket() function, just above GR_SOCK_MSG/GR_SOCK_NOINET_MSG logging.

But as my errors are not "denied socket" but instead "denied connect() to" messages, perhaps I should add dump_stack() just under "denied:" line in grsecurity/gracl_ip.c (function gr_search_connectbind()) instead?
That one seems to me to log GR_BIND_ACL_MSG/GR_CONNECT_ACL_MSG errors?

[spender]

Yes, please do that -- sorry about the mixup.

-Brad

[mnalis]

ok, added it there too, now it prints something more. Hope it helps.

Code: Select all

Pid: 9398, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff81002a17>] ? sysret_check+0x22/0x5d
 [<ffffffff810029eb>] system_call_fastpath+0x18/0x1d
grsec: From 192.168.200.1: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:9398] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:5582] uid/euid:33/33 gid/egid:33/33
Pid: 9398, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff810029eb>] system_call_fastpath+0x18/0x1d
grsec: From 192.168.200.1: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:9398] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:5582] uid/euid:33/33 gid/egid:33/33
Pid: 9398, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff810029eb>] system_call_fastpath+0x18/0x1d
grsec: From 192.168.200.1: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:9398] uid/euid:10874/10874 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:5582] uid/euid:33/33 gid/egid:33/33


[spender]

Could you also grab an strace of the process with -e socket,connect ?

-Brad

[spender]

Any luck getting the strace?

-Brad

[mnalis]

Not yet , unfortunately (but see below). Sorry for the delay... I can't touch too much on production server (like running whole quite busy apache through strace) , and this seems to be somewhat elusive bug. Here is what I've managed so far:

I've traced through wordpress and managed to create minimal PHP script which reproduces problem, here it is:

Code: Select all

<?php

  $url = "http://api.wordpress.org/core/version-check/1.6/?version=3.3.1&php=5.3.3-7%2Bsqueeze8&locale=hr&mysql=5.1.61&local_package=hr&blogs=1&users=1&multisite_enabled=0";

  $handle = curl_init();

  curl_setopt( $handle, CURLOPT_URL, $url);
  curl_setopt( $handle, CURLOPT_RETURNTRANSFER, true );
  curl_setopt( $handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0 );

  $theResponse = curl_exec( $handle );
  print_r ("finished " . time() . " - " . $theResponse);



when I call it from web browser, it manages to produce following dmesg messages:

Code: Select all

Pid: 15262, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff810029eb>] system_call_fastpath+0x18/0x1d
 [<ffffffff81002a17>] ? sysret_check+0x22/0x5d
grsec: From 161.53.11.142: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:15262] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:14992] uid/euid:33/33 gid/egid:33/33
Pid: 15262, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff81002a17>] ? sysret_check+0x22/0x5d
 [<ffffffff810029eb>] system_call_fastpath+0x18/0x1d
grsec: From 161.53.11.142: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:15262] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:14992] uid/euid:33/33 gid/egid:33/33
Pid: 15262, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff81002a17>] ? sysret_check+0x22/0x5d
 [<ffffffff810029eb>] system_call_fastpath+0x18/0x1d
grsec: From 161.53.11.142: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:15262] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:14992] uid/euid:33/33 gid/egid:33/33


Howver, when I run the same php script from shell via php-cgi, no dmesg output is produced at all (although I run php-cgi as that same user/group as the apache suexec does, so it should be running in same role/subject as when run through apache).

Here is the strace (but as it does not produce the dmesg errors, it might be different from what it does when run through web server!)

Code: Select all

# strace -ff -e socket,connect setuidgid user36923 php-cgi test_mn.php
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_NETLINK, SOCK_RAW, 0)         = 3
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
Process 31831 attached (waiting for parent)
Process 31831 resumed (parent 31830 ready)
Process 31831 detached
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_FILE, SOCK_STREAM|SOCK_CLOEXEC|SOCK_NONBLOCK, 0) = 3
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)
socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
socket(PF_NETLINK, SOCK_RAW, 0)         = 3
socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.200.254")}, 16) = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.139")}, 16) = 0
connect(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = 0
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.139")}, 16) = -1 EINPROGRESS (Operation now in progress)
X-Powered-By: PHP/5.3.3-7+squeeze8
Content-type: text/html

finished 1333749010 - a:1:{s:6:"offers";a:2:{i:0;a:9:{s:8:"response";s:6:"latest";s:8:"download";s:46:"http://hr.wordpress.org/wordpress-3.3.1-hr.zip";s:6:"locale";s:2:"hr";s:8:"packages";a:4:{s:4:"full";s:46:"http://hr.wordpress.org/wordpress-3.3.1-hr.zip";s:10:"no_content";b:0;s:11:"new_bundled";b:0;s:7:"partial";b:0;}s:7:"current";s:5:"3.3.1";s:11:"php_version";s:5:"5.2.4";s:13:"mysql_version";s:3:"5.0";s:11:"new_bundled";s:3:"3.2";s:15:"partial_version";b:0;}i:1;a:9:{s:8:"response";s:6:"latest";s:8:"download";s:40:"http://wordpress.org/wordpress-3.3.1.zip";s:6:"locale";s:5:"en_US";s:8:"packages";a:4:{s:4:"full";s:40:"http://wordpress.org/wordpress-3.3.1.zip";s:10:"no_content";s:51:"http://wordpress.org/wordpress-3.3.1-no-content.zip";s:11:"new_bundled";s:52:"http://wordpress.org/wordpress-3.3.1-new-bundled.zip";s:7:"partial";b:0;}s:7:"current";s:5:"3.3.1";s:11:"php_version";s:5:"5.2.4";s:13:"mysql_version";s:3:"5.0";s:11:"new_bundled";s:3:"3.2";s:15:"partial_version";b:0;}}}#                         


I'll setup test server in next few days and try to strace httpd there (so I can get you real syscalls which problematic php script executes), and also try to trim down grsec policy file to see if I can reduce policy file to minimal set that reproduces the problem.

[mnalis]

ok, I've managed to get clean strace faster then expected... the same PHP script, just with added sleep at the beginning (so I can strace it before it starts doing networking), produces the following dmesg output:

Code: Select all

Pid: 21677, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff81002c01>] tracesys+0xe0/0xe5
grsec: From 83.139.110.2: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:21677] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11204] uid/euid:33/33 gid/egid:33/33
Pid: 21677, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff81002c01>] tracesys+0xe0/0xe5
grsec: From 83.139.110.2: (virtual:G:/usr/bin/php5-cgi) denied connect() to 0.0.0.0 port 0 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:21677] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11204] uid/euid:33/33 gid/egid:33/33
Pid: 21677, comm: php-cgi Not tainted 2.6.32.59-grsec201203212033-debug2 #9
Call Trace:
 [<ffffffff811d4ba5>] gr_search_connectbind+0x409/0x4d1
 [<ffffffff811d4c8f>] gr_search_connect+0x22/0x2d
 [<ffffffff812fa372>] sys_connect+0x9c/0x115
 [<ffffffff81002c01>] tracesys+0xe0/0xe5
grsec: From 83.139.110.2: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:21677] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:11204] uid/euid:33/33 gid/egid:33/33


and stracing it at the same time produces:

Code: Select all

socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
socket(PF_NETLINK, SOCK_RAW, 0)         = 3
socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.200.254")}, 16) = 0
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = -1 EACCES (Permission denied)
connect(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = -1 EACCES (Permission denied)
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.139")}, 16) = -1 EACCES (Permission denied)
socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = -1 EINPROGRESS (Operation now in progress)


The relevant part of the policy file should be this:

Code: Select all

subject /usr/bin/php5-cgi AdT
        /proc/stat                      r
        /proc/cpuinfo                   r
        /sys
        /dev/log                        rw
        /var/lib/php5                   rwcdl
        /var/lib/phpmyadmin             r
        /etc/phpmyadmin                 r
        /etc/roundcube                  r
        /var/log/roundcube              rwcdl

        bind disabled
        connect 192.168.200.254/32:53 dgram udp
        connect 192.168.200.254/32:53 stream tcp
        connect 192.168.200.14/32:3306 stream tcp
        connect 192.168.200.18/32:25 stream tcp

        connect 72.233.56.128/25:80 stream tcp  # wordpress.org

        # [...] repeat connect "xxxxx:80 stream tcp"  for lots of other, unrelated web sites

# next two lines, if uncommented, make the bug go away
#       connect 0.0.0.0/0:0     dgram udp
#       connect 0.0.0.0/0:80    dgram udp

        sock_allow_family inet6
        sock_allow_family netlink    # on unrelated note, I don't see why pgp-cgi would need netlink, but there were some problems if it was disabled IIRC, so it's here...


Hope some of this helps in tracing it down.

Thanks for all the effort!
Matija

[mnalis]

Can I do anything more to help tracking this down?

Thanks,
Matija

[spender]

Can you re-try the straces with the latest patch? I've allowed the AF_UNSPEC connects on dgram sockets (a disassociation with an existing connection) which I think may have been producing the messages with port 0. According to my reading, however, the combination of dgram socket and IP protocol should be interpreted by the kernel as UDP -- I'm not convinced that RBAC is reporting anything incorrectly here (minus the AF_UNSPEC case that I've fixed).

Edit: could you also add shutdown and close to the syscalls being traced? Use strace -f -e connect,socket,shutdown,close

-Brad

[mnalis]

it seems that both udp/0 and udp/80 grsec errors are gone with grsec-2.9-2.6.32.59-201204192143 !

the strace -f -e connect,socket,shutdown,close output of test php script is now:

Code: Select all

32220 socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
32220 close(3)                          = 0
32220 socket(PF_NETLINK, SOCK_RAW, 0)   = 3
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
32220 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.200.254")}, 16) = 0
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
32220 connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = 0
32220 connect(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
32220 connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.139")}, 16) = 0
32220 close(3)                          = 0
32220 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
32220 connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = -1 EINPROGRESS (Operation now in progress)
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 close(3)                          = 0
32220 close(3)                          = 0


I'll put it on production machine to check if it fixes all cases and let you know how it fares in few days.
Thanks!

[mnalis]

Sorry, I was too quick, and did not exit the admin mode first, so ignore previous post...

with grsec-2.9-2.6.32.59-201204192143, udp/0 is gone, but udp/80 message is still there. If I understand correctly, that remaining problem is now due to "feature" in PHP curl_exec() which tries (for some unknown reason) to retrieve HTTP URLs over UDP/80 first before trying TCP/80?

Is there way in grsec to deny but silence such "connect"s ?

Code: Select all

grsec: From 83.139.110.2: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.138 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:16942] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:4120] uid/euid:33/33 gid/egid:33/33
grsec: From 83.139.110.2: (virtual:G:/usr/bin/php5-cgi) denied connect() to 72.233.56.139 port 80 sock type dgram protocol udp by /usr/bin/php5-cgi[php-cgi:16942] uid/euid:36923/36923 gid/egid:1000/1000, parent /usr/lib/apache2/mpm-prefork/apache2[apache2:4120] uid/euid:33/33 gid/egid:33/33


"strace -f -e connect,socket,shutdown,close" gives:

Code: Select all

16942 socket(PF_INET6, SOCK_DGRAM, IPPROTO_IP) = 3
16942 close(3)                          = 0
16942 socket(PF_NETLINK, SOCK_RAW, 0)   = 3
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
16942 connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.200.254")}, 16) = 0
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
16942 connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = -1 EACCES (Permission denied)
16942 connect(3, {sa_family=AF_UNSPEC, sa_data="\0\0\0\0\0\0\0\0\0\0\0\0\0\0"}, 16) = 0
16942 connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.139")}, 16) = -1 EACCES (Permission denied)
16942 close(3)                          = 0
16942 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
16942 connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("72.233.56.138")}, 16) = -1 EINPROGRESS (Operation now in progress)
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 close(3)                          = 0
16942 close(3)                          = 0