I've tested 3.8.3 + grsecurity-2.9.1-3.8.3-201303190012.patch and the issue is resolved.
With 3.2.40 + grsecurity-2.9.1-3.2.40-201303190045.patch the issue still persists.
tester3:~# dmesg |head -n1
[ 0.000000] Linux version 3.2.40-grsec (root@buildbox) (gcc version 4.4.5 (Debian 4.4.5-8) ) #1 SMP Fri Mar 22 08:39:55 EDT 2013
tester3:~# for i in $(seq 40000 10000 200000); do ulimit -v $i ; echo $i; php -i |grep -m1 "PHP Ver" ;done 2>&1 |grep -m1 -B1 Version
120000
PHP Version => 5.3.17
tester3:~# for i in $(seq 40000 10000 200000); do ulimit -v $i ; echo $i; php -i |grep -m1 "PHP Ver" ;done 2>&1 |grep -m1 -B1 Version
200000
PHP Version => 5.3.17
tester3:~# for i in $(seq 40000 10000 200000); do ulimit -v $i ; echo $i; php -i |grep -m1 "PHP Ver" ;done 2>&1 |grep -m1 -B1 Version
130000
PHP Version => 5.3.17
tester3:~# for i in $(seq 40000 10000 200000); do ulimit -v $i ; echo $i; php -i |grep -m1 "PHP Ver" ;done 2>&1 |grep -m1 -B1 Version
150000
PHP Version => 5.3.17
[Fri Mar 22 22:24:39 2013] grsec: denied resource overstep by requesting 254648320 for RLIMIT_AS against limit 201326592 for /dovecot/libexec/dovecot/imap[imap:14236] uid/euid:0/0 gid/egid:0/0, parent /dovecot/sbin/dovecot[dovecot:5717] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:24:45 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 141385728 for RLIMIT_AS against limit 40960000 for /bin/dmesg[dmesg:14253] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14161] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:24:46 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 240443392 for RLIMIT_AS against limit 40960000 for /bin/dmesg[dmesg:14255] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14161] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:24:46 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 49876992 for RLIMIT_AS against limit 40960000 for /bin/dmesg[dmesg:14257] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14161] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:25:14 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 164429824 for RLIMIT_AS against limit 102400000 for /usr/local/php53/bin/php[php:14604] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14573] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:25:32 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 246222848 for RLIMIT_AS against limit 204800000 for /usr/local/php53/bin/php[php:14633] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14573] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:25:35 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 212287488 for RLIMIT_AS against limit 204800000 for /usr/local/php53/bin/php[php:14656] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14573] uid/euid:0/0 gid/egid:0/0
[Fri Mar 22 22:25:41 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 259092480 for RLIMIT_AS against limit 204800000 for /usr/bin/strace[strace:14661] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:14573] uid/euid:0/0 gid/egid:0/0
tester3:~# ulimit -v 200000
tester3:~# strace /usr/local/php53/bin/php -v
execve("/usr/local/php53/bin/php", ["/usr/local/php53/bin/php", "-v"], [/* 20 vars */] <unfinished ...>
+++ killed by SIGKILL +++
Killed
[Fri Mar 22 22:37:46 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 234651648 for RLIMIT_AS against limit 204800000 for /usr/local/php53/bin/php[php:22793] uid/euid:0/0 gid/egid:0/0, parent /usr/bin/strace[strace:22792] uid/euid:0/0 gid/egid:0/0
tester3:~# strace /usr/local/php53/bin/php -v
Killed
[Fri Mar 22 22:37:48 2013] grsec: From 94.155.37.175: denied resource overstep by requesting 235184128 for RLIMIT_AS against limit 204800000 for /usr/bin/strace[strace:22795] uid/euid:0/0 gid/egid:0/0, parent /bin/bash[bash:22752] uid/euid:0/0 gid/egid:0/0
tester3:~# strace /usr/local/php53/bin/php -v
execve("/usr/local/php53/bin/php", ["/usr/local/php53/bin/php", "-v"], [/* 20 vars */]) = 0
brk(0) = 0x8f98f40
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xe7556000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/usr/local/mysql5/lib/tls/i686/sse2/cmov/libstdc++.so.6", O_RDONLY) = -1 ENOENT (No such file or directory)
stat64("/usr/local/mysql5/lib/tls/i686/sse2/cmov", 0xf56aaec8) = -1 ENOENT (No such file or directory)
...
...
...
...
stat64("/root", {st_mode=S_IFDIR|0700, st_size=12288, ...}) = 0
time(NULL) = 1363984670
socket(PF_FILE, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0
send(3, "<158>Mar 22 22:37:50 php-sureacc"..., 150, MSG_NOSIGNAL) = 150
close(3) = 0
gettimeofday({1363984670, 294778}, NULL) = 0
munmap(0xe53d2000, 163840) = 0
exit_group(0) = ?