java eats 99.9% CPU

[bartosz]

Hi
This time I've searched carefully whole forums and googl but found no answer.
Java process eats 99,9% my cpu and I do't know why.
I don't see nothing strange in tomcat's or system's logs.
Gradm is disabled and PAX not compiled in.
I use java Version 1.4.2 with Tomcat 4.1.
Average load is about 0.03 - 0.5 before starting java.

This is my grsec kernel configuration

Code: Select all

#
# Grsecurity
#
CONFIG_GRKERNSEC=y
CONFIG_CRYPTO=y
CONFIG_CRYPTO_SHA256=y
# CONFIG_GRKERNSEC_LOW is not set
# CONFIG_GRKERNSEC_MID is not set
# CONFIG_GRKERNSEC_HI is not set
CONFIG_GRKERNSEC_CUSTOM=y

#
# PaX Control
#
# CONFIG_GRKERNSEC_PAX_SOFTMODE is not set
# CONFIG_GRKERNSEC_PAX_EI_PAX is not set
# CONFIG_GRKERNSEC_PAX_PT_PAX_FLAGS is not set
CONFIG_GRKERNSEC_PAX_NO_ACL_FLAGS=y
# CONFIG_GRKERNSEC_PAX_HAVE_ACL_FLAGS is not set
# CONFIG_GRKERNSEC_PAX_HOOK_ACL_FLAGS is not set

#
# Address Space Protection
#
# CONFIG_GRKERNSEC_KMEM is not set
# CONFIG_GRKERNSEC_IO is not set
CONFIG_GRKERNSEC_PROC_MEMMAP=y
CONFIG_GRKERNSEC_BRUTE=y
CONFIG_GRKERNSEC_HIDESYM=y

#
# Role Based Access Control Options
#
CONFIG_GRKERNSEC_ACL_HIDEKERN=y
CONFIG_GRKERNSEC_ACL_MAXTRIES=3
CONFIG_GRKERNSEC_ACL_TIMEOUT=30

#
# Filesystem Protections
#
CONFIG_GRKERNSEC_PROC=y
# CONFIG_GRKERNSEC_PROC_USER is not set
CONFIG_GRKERNSEC_PROC_USERGROUP=y
CONFIG_GRKERNSEC_PROC_GID=1001
CONFIG_GRKERNSEC_PROC_ADD=y
CONFIG_GRKERNSEC_LINK=y
CONFIG_GRKERNSEC_FIFO=y
CONFIG_GRKERNSEC_CHROOT=y
CONFIG_GRKERNSEC_CHROOT_MOUNT=y
CONFIG_GRKERNSEC_CHROOT_DOUBLE=y
CONFIG_GRKERNSEC_CHROOT_PIVOT=y
CONFIG_GRKERNSEC_CHROOT_CHDIR=y
CONFIG_GRKERNSEC_CHROOT_CHMOD=y
CONFIG_GRKERNSEC_CHROOT_FCHDIR=y
CONFIG_GRKERNSEC_CHROOT_MKNOD=y
CONFIG_GRKERNSEC_CHROOT_SHMAT=y
CONFIG_GRKERNSEC_CHROOT_UNIX=y
CONFIG_GRKERNSEC_CHROOT_FINDTASK=y
CONFIG_GRKERNSEC_CHROOT_NICE=y
CONFIG_GRKERNSEC_CHROOT_SYSCTL=y
CONFIG_GRKERNSEC_CHROOT_CAPS=y

#
# Kernel Auditing
#
# CONFIG_GRKERNSEC_AUDIT_GROUP is not set
# CONFIG_GRKERNSEC_EXECLOG is not set
# CONFIG_GRKERNSEC_RESLOG is not set
# CONFIG_GRKERNSEC_CHROOT_EXECLOG is not set
# CONFIG_GRKERNSEC_AUDIT_CHDIR is not set
# CONFIG_GRKERNSEC_AUDIT_MOUNT is not set
# CONFIG_GRKERNSEC_AUDIT_IPC is not set
CONFIG_GRKERNSEC_SIGNAL=y
CONFIG_GRKERNSEC_FORKFAIL=y
CONFIG_GRKERNSEC_TIME=y
CONFIG_GRKERNSEC_PROC_IPADDR=y
# CONFIG_GRKERNSEC_AUDIT_TEXTREL is not set

#
# Executable Protections
#
CONFIG_GRKERNSEC_EXECVE=y
# CONFIG_GRKERNSEC_SHM is not set
# CONFIG_GRKERNSEC_DMESG is not set
CONFIG_GRKERNSEC_RANDPID=y
# CONFIG_GRKERNSEC_TPE is not set

#
# Network Protections
#
CONFIG_GRKERNSEC_RANDNET=y
CONFIG_GRKERNSEC_RANDSRC=y
# CONFIG_GRKERNSEC_SOCKET is not set

#
# Sysctl support
#
# CONFIG_GRKERNSEC_SYSCTL is not set

#
# Logging options
#
CONFIG_GRKERNSEC_FLOODTIME=10
CONFIG_GRKERNSEC_FLOODBURST=4

[PaX Team]

Java process eats 99,9% my cpu and I do't know why.
I don't see nothing strange in tomcat's or system's logs.
Gradm is disabled and PAX not compiled in.
I use java Version 1.4.2 with Tomcat 4.1.
Average load is about 0.03 - 0.5 before starting java.

try to strace and ltrace the runaway process, maybe you can see what it is doing. looking at the stack backtrace in gdb a few times might also help (for best info you'll need non-stripped binaries though).

[bartosz]

found this

Code: Select all

An unexpected exception has been detected in native code outside the VM.
Unexpected Signal : 11 occurred at PC=0x0
Function=[Unknown.]
Library=(N/A)

NOTE: We are unable to locate the function name symbol for the error
      just occurred. Please refer to release documentation for possible
      reason and solutions.


Current Java thread:

Dynamic libraries:
08048000-08056000 r-xp 00000000 03:03 635517     /usr/local/java/bin/java
08056000-08059000 rw-p 0000d000 03:03 635517     /usr/local/java/bin/java
40000000-40015000 r-xp 00000000 03:03 537536     /lib/ld-2.3.2.so
40015000-40016000 rw-p 00015000 03:03 537536     /lib/ld-2.3.2.so
40016000-4001e000 r-xp 00000000 03:03 1547966    /usr/local/java/jre/lib/i386/native_threads/libhpi.so
4001e000-4001f000 rw-p 00007000 03:03 1547966    /usr/local/java/jre/lib/i386/native_threads/libhpi.so
40020000-4002d000 r-xp 00000000 03:03 945746     /lib/tls/libpthread-0.60.so
4002d000-4002e000 rw-p 0000c000 03:03 945746     /lib/tls/libpthread-0.60.so
40030000-40032000 r-xp 00000000 03:03 538547     /lib/libdl-2.3.2.so
40032000-40033000 rw-p 00001000 03:03 538547     /lib/libdl-2.3.2.so
40033000-40165000 r-xp 00000000 03:03 944730     /lib/tls/libc-2.3.2.so
40165000-40168000 rw-p 00132000 03:03 944730     /lib/tls/libc-2.3.2.so
4016c000-40568000 r-xp 00000000 03:03 652042     /usr/local/java/jre/lib/i386/client/libjvm.so
40568000-40583000 rw-p 003fb000 03:03 652042     /usr/local/java/jre/lib/i386/client/libjvm.so
40595000-405a7000 r-xp 00000000 03:03 537645     /lib/libnsl-2.3.2.so
405a7000-405a8000 rw-p 00011000 03:03 537645     /lib/libnsl-2.3.2.so
405aa000-405cb000 r-xp 00000000 03:03 945745     /lib/tls/libm-2.3.2.so
405cb000-405cc000 rw-p 00021000 03:03 945745     /lib/tls/libm-2.3.2.so
405cc000-405d0000 rw-s 00000000 03:03 602939     /tmp/hsperfdata_root/2920
405d5000-405e0000 r-xp 00000000 03:03 537648     /lib/libnss_files-2.3.2.so
405e0000-405e1000 rw-p 0000a000 03:03 537648     /lib/libnss_files-2.3.2.so
405e1000-405f1000 r-xp 00000000 03:03 1547973    /usr/local/java/jre/lib/i386/libverify.so
405f1000-405f3000 rw-p 0000f000 03:03 1547973    /usr/local/java/jre/lib/i386/libverify.so
405f3000-40613000 r-xp 00000000 03:03 1547974    /usr/local/java/jre/lib/i386/libjava.so
40613000-40615000 rw-p 0001f000 03:03 1547974    /usr/local/java/jre/lib/i386/libjava.so
40615000-40629000 r-xp 00000000 03:03 1547976    /usr/local/java/jre/lib/i386/libzip.so
40629000-4062c000 rw-p 00013000 03:03 1547976    /usr/local/java/jre/lib/i386/libzip.so
4062c000-407d3000 r--s 00000000 03:03 1841035    /var/tomcat/common/endorsed/xercesImpl-2.0.0.jar
407d3000-407ef000 r--s 00000000 03:03 1841036    /var/tomcat/common/endorsed/xml-apis.jar
407ef000-42193000 r--s 00000000 03:03 1548045    /usr/local/java/jre/lib/rt.jar
421dd000-421f3000 r--s 00000000 03:03 1548000    /usr/local/java/jre/lib/sunrsasign.jar
421f3000-422d0000 r--s 00000000 03:03 1548042    /usr/local/java/jre/lib/jsse.jar
422d0000-422e1000 r--s 00000000 03:03 1548001    /usr/local/java/jre/lib/jce.jar
422e1000-4283a000 r--s 00000000 03:03 1548043    /usr/local/java/jre/lib/charsets.jar

Heap at VM Abort:
Heap
 def new generation   total 4224K, used 0K [0x448f0000, 0x44d80000, 0x453b0000)
  eden space 3776K,   0% used [0x448f0000, 0x448f0048, 0x44ca0000)
  from space 448K,   0% used [0x44ca0000, 0x44ca0000, 0x44d10000)
  to   space 448K,   0% used [0x44d10000, 0x44d10000, 0x44d80000)
 tenured generation   total 56768K, used 0K [0x453b0000, 0x48b20000, 0x4d4f0000)
   the space 56768K,   0% used [0x453b0000, 0x453b0000, 0x453b0200, 0x48b20000)
 compacting perm gen  total 4096K, used 277K [0x4d4f0000, 0x4d8f0000, 0x514f0000)
   the space 4096K,   6% used [0x4d4f0000, 0x4d5356c0, 0x4d535800, 0x4d8f0000)

Local Time = Sun Oct  2 01:45:21 2005
Elapsed Time = 0
#
# The exception above was detected in native code outside the VM
#
# Java VM: Java HotSpot(TM) Client VM (1.4.2_05-b04 mixed mode)
#

[SG]

It can be because bad libs or linux system. Try use recommended linux for you version java/tomcat. I see such errors some times ago when add required libs from other system