grsecurity forums

by **dabetz** » Mon Nov 09, 2009 3:55 pm

Hello,

i have got the exactly same problem with Kernel 2.6.31.4-grsec and MySQL 5.0.51b and 5.1.39. MySQL is compiled from source on an gentoo.
The problem does not appear on 2.6.30.7-grsec ( grsecurity-2.1.14-2.6.30.7-200909151852 )
but on 2.6.31.4-grsec ( grsecurity-2.1.14-2.6.31.4-200910202041 ) and 2.6.31.5-grsec ( grsecurity-2.1.14-2.6.31.5-200910232000 )

Could it be, that the 'bad page state' error fix was removed ?

Greetings,
Daniel

The failure itself:

MySQL 5.1.39-log:

091109 18:41:30 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=1384120320
read_buffer_size=2097152
max_used_connections=7
max_threads=150
threads_connected=4
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 2274233 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x46504488
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x46b213b8 thread_stack 0x30000
/usr/sbin/mysqld-5(my_print_stacktrace+0x21)[0x85def51]
/usr/sbin/mysqld-5(handle_segfault+0x381)[0x825e8f1]
[0xb0710400]
/usr/sbin/mysqld-5(_ZN9Item_func10fix_fieldsEP3THDPP4Item+0x1c5)[0x81d40b5]
/usr/sbin/mysqld-5(_ZN12Item_func_if10fix_fieldsEP3THDPP4Item+0x38)[0x81e59b8]
/usr/sbin/mysqld-5(_Z12setup_fieldsP3THDPP4ItemR4ListIS1_E17enum_mark_columnsPS5_b+0x11f)[0x82a73af]
/usr/sbin/mysqld-5(_ZN4JOIN7prepareEPPP4ItemP10TABLE_LISTjS1_jP8st_orderS7_S1_S7_P13st_select_lexP18st_select_lex_unit+0x202)[0x82ca142]
/usr/sbin/mysqld-5(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x729)[0x82d8949]
/usr/sbin/mysqld-5(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x15e)[0x82ddd7e]
/usr/sbin/mysqld-5[0x826c5e0]
/usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x4157)[0x8273177]
/usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x340)[0x82769f0]
/usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1218)[0x8277c18]
/usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe0)[0x82784a0]
/usr/sbin/mysqld-5(handle_one_connection+0x253)[0x8268db3]
/lib/libpthread.so.0[0xb06f816f]
/lib/libc.so.6(clone+0x5e)[0xb05f7c0e]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x90e2ce8 = SELECT count(*) AS count, if (pa.text<>'',pa.text,pa.number) AS name, pa.name AS title FROM product AS p LEFT JOIN product_attribute AS pa ON ( pa.product_number = p.product_number AND pa.language = p.language ) LEFT JOIN product_attribute AS paTeleskope_Allgemein_Serie ON ( paTeleskope_Allgemein_Serie.product_number = p.product_number AND paTeleskope_Allgemein_Serie.language = p.language ) LEFT JOIN product_attribute AS paTeleskope_Anwendungsgebiete ON ( paTeleskope_Anwendungsgebiete.product_number = p.product_number AND paTeleskope_Anwendungsgebiete.language = p.language ) LEFT JOIN product_attribute AS paTeleskope_Optik_Bauart ON ( paTeleskope_Optik_Bauart.product_number = p.product_number AND paTeleskope_Optik_Bauart.language = p.language ) LEFT JOIN product_attribute AS paTeleskope_Optik_Brennweite ON ( paTeleskope_Optik_Brennweite.product_number = p.product_number AND paTeleskope_Optik_Brennweite.language = p.language ) WHERE p.shop = '100' AND p.language = 'fr' AND p.active_in_filter = 1 AND p.is_deliv
thd->thread_id=34772
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

MySQL 5.0.51b-log:
091109 14:19:11 - mysqld got signal 11;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=1384120320
read_buffer_size=2093056
max_used_connections=8
max_connections=150
threads_connected=3
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections = 2272678 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd=0x50c08b40
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
Cannot determine thread, fp=0x50d6e0b8, backtrace may not be correct.
Stack range sanity check OK, backtrace follows:
0x818c77c
0xb7842e55
0x826aeea
0x81e6517
0x81da37f
0x81dd722
0x81d886b
0x81a1adb
0x81a9440
0x81a0250
0x819fc9e
0x819f0f2
0xb783d13d
0xb76abe7a
New value of fp=(nil) failed sanity check, terminating stack trace!
Please read http://dev.mysql.com/doc/mysql/en/using ... trace.html and follow instructions on how to resolve the stack trace. Resolved
stack trace is much more helpful in diagnosing the problem, so please do
resolve it
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8d51a60 = SELECT p.* FROM product AS p LEFT JOIN product_attribute AS paTeleskope_Optik_Bauart ON ( paTeleskope_Optik_Bauart.product_number = p.product_number AND paTeleskope_Optik_Bauart.language = p.language ) WHERE p.shop = '100' AND p.language = 'de' AND p.active_in_filter = 1 AND p.is_deliverable = 1 AND p.is_master = 0 AND p.class_of_goods_id LIKE '10%' AND ( paTeleskope_Optik_Bauart.refcode = 'Teleskope.Optik.Bauart' AND ( paTeleskope_Optik_Bauart.number IN ( 'Schmidt-Cassegrain' ) OR paTeleskope_Optik_Bauart.text IN ( 'Schmidt-Cassegrain' ) ) ) ORDER BY p.preferred DESC, p.sales DESC LIMIT 9 OFFSET 0
thd->thread_id=46977
The manual page at http://www.mysql.com/doc/en/Crashing.html contains
information that should help you find out what is causing the crash.

by **PaX Team** » Thu Nov 12, 2009 5:47 am

dabetz wrote:i have got the exactly same problem with Kernel 2.6.31.4-grsec and MySQL 5.0.51b and 5.1.39. MySQL is compiled from source on an gentoo.
The problem does not appear on 2.6.30.7-grsec ( grsecurity-2.1.14-2.6.30.7-200909151852 )
but on 2.6.31.4-grsec ( grsecurity-2.1.14-2.6.31.4-200910202041 ) and 2.6.31.5-grsec ( grsecurity-2.1.14-2.6.31.5-200910232000 )

Could it be, that the 'bad page state' error fix was removed ?

the bad page state error was due to SANITIZE and i fixed it back then and haven't got any reports since. so whatever is causing this crash for you is probably due to some newer code (especially if you're not even using SANITIZE). what would help is if you could attach to the mysqld process with gdb and then trigger the problem so that gdb would catch it before mysqld's own handler and then you could get the usual info (x/8i $pc, x/8x $sp, info reg, bt, etc).

by **dabetz** » Tue Dec 08, 2009 4:54 am

Hello Pax Team,

i have done some testing now and i am able now to reproduze the error after some hours of querying the mysql server.
I have tested with 5 different SQL querys. After about 400.000 to 1,2 million querys the mysql crashes with signal 11.

It crashes with 2.6.31.4-grsec ( grsecurity-2.1.14-2.6.31.4-200910202041 ) but not with an vanilla 2.6.31.4 Kernel.

Hope this helps you fixing the problem.

Greetings,
Daniel

Heres the error in the MySQL Log:

091207 13:08:11 [Note] Plugin 'FEDERATED' is disabled.
091207 13:08:11 [Note] Plugin 'ndbcluster' is disabled.
091207 13:08:11 InnoDB: Started; log sequence number 1 3197172279
091207 13:08:11 [Note] Event Scheduler: Loaded 0 events
091207 13:08:11 [Note] /usr/sbin/mysqld-5: ready for connections.
Version: '5.1.39-log' socket: '/var/lib/mysql/mysql5.sock' port: 3306 Source distribution
091207 13:41:01 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=33554432
read_buffer_size=1048576
max_used_connections=94
max_threads=500
threads_connected=14
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1571947 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0xad3232c8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xadd153b8 thread_stack 0x30000
/usr/sbin/mysqld-5(my_print_stacktrace+0x21)[0x85def51]
/usr/sbin/mysqld-5(handle_segfault+0x381)[0x825e8f1]
[0xb7715400]
/usr/sbin/mysqld-5[0x833f861]
/usr/sbin/mysqld-5[0x8342d2a]
/usr/sbin/mysqld-5[0x834307a]
/usr/sbin/mysqld-5[0x8343399]
/usr/sbin/mysqld-5(_ZN10SQL_SELECT17test_quick_selectEP3THD6BitmapILj64EEyyb+0x5ef)[0x834473f]
/usr/sbin/mysqld-5[0x82d37e5]
/usr/sbin/mysqld-5(_ZN4JOIN8optimizeEv+0x684)[0x82d4c64]
/usr/sbin/mysqld-5(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x6c)[0x82d828c]
/usr/sbin/mysqld-5(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x15e)[0x82ddd7e]
/usr/sbin/mysqld-5[0x826c5e0]
/usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x4157)[0x8273177]
/usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x340)[0x82769f0]
/usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1218)[0x8277c18]
/usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe0)[0x82784a0]
/usr/sbin/mysqld-5(handle_one_connection+0x253)[0x8268db3]
/lib/libpthread.so.0[0xb76fd16f]
/lib/libc.so.6(clone+0x5e)[0xb75fcc0e]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x8fa03d0 = SELECT SQL_NO_CACHE * FROM cache_pages WHERE id IN(4878647, 1065806,6910821,1147997,1818073,6618730,2646476,2383120,3009439)
thd->thread_id=474979
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

here ist the full backtrace:

[Thread 0xae0a8b90 (LWP 25798) exited]
[Thread 0xb069db90 (LWP 25800) exited]
[Thread 0xadb6cb90 (LWP 25816) exited]
[New Thread 0xad83bb90 (LWP 25831)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xad66cb90 (LWP 25810)]
0x081e5439 in Item_cond::fix_fields (this=0x9111678, thd=0xb0760008, ref=0x9111b94) at item_cmpfunc.cc:3948
3948 item_cmpfunc.cc: No such file or directory.
in item_cmpfunc.cc
(gdb) backtrace full
#0 0x081e5439 in Item_cond::fix_fields (this=0x9111678, thd=0xb0760008, ref=0x9111b94) at item_cmpfunc.cc:3948
tmp_table_map = <value optimized out>
item = (class Item *) 0x8fdd030
buff = "hÂ°fÂ"
__PRETTY_FUNCTION__ = "virtual bool Item_cond::fix_fields(THD*, Item**)"
#1 0x081e54d0 in Item_cond::fix_fields (this=0x8fdbe40, thd=0xb0760008, ref=0x9113a24) at item_cmpfunc.cc:3960
tmp_table_map = <value optimized out>
item = (class Item *) 0x9111678
buff = "\230Â°fÂ"
__PRETTY_FUNCTION__ = "virtual bool Item_cond::fix_fields(THD*, Item**)"
#2 0x082a2f5e in setup_conds (thd=0xb0760008, tables=0x8fdb878, leaves=0x8fdb878, conds=0x9113a24) at sql_base.cc:8010
select_lex = (SELECT_LEX *) 0xb07613ac
table = (TABLE_LIST *) 0x0
it_is_update = false
save_is_item_list_lookup = true
_db_func_ = 0x8730bb4 "setup_without_group"
_db_file_ = 0x873069d "sql_select.cc"
_db_level_ = 9
_db_framep_ = (char **) 0x819a219
#3 0x082c8582 in JOIN::prepare (this=0x9112890, rref_pointer_array=0xb07614a4, tables_init=0x8fdb878, wild_num=1, conds_init=0x8fdbe40, og_num=0,
order_init=0x0, group_init=0x0, having_init=0x0, proc_param_init=0x0, select_lex_arg=0xb07613ac, unit_arg=0xb076113c) at sql_select.cc:413
_db_func_ = 0x873120c "mysql_select"
_db_file_ = 0x873069d "sql_select.cc"
_db_level_ = 7
_db_framep_ = (char **) 0x86a468c
table_ptr = <value optimized out>
#4 0x082d9c57 in mysql_select (thd=0xb0760008, rref_pointer_array=0xb07614a4, tables=0x8fdb878, wild_num=1, fields=@0xb0761440, conds=0x8fdbe40,
og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=2147764736, result=0x9111bf8, unit=0xb076113c, select_lex=0xb07613ac)
at sql_select.cc:2414
err = <value optimized out>
free_join = <value optimized out>
_db_func_ = 0x87313ca "handle_select"
_db_file_ = 0x873069d "sql_select.cc"
_db_level_ = 6
_db_framep_ = (char **) 0x8634977
join = (JOIN *) 0x9112890
#5 0x082dcd37 in handle_select (thd=0xb0760008, lex=0xb07610e0, result=0x9111bf8, setup_tables_done_option=0) at sql_select.cc:269
unit = (SELECT_LEX_UNIT *) 0xb076113c
res = <value optimized out>
select_lex = (SELECT_LEX *) 0xb07613ac
_db_func_ = 0x86e2b3d "mysql_execute_command"
_db_file_ = 0x86e278b "sql_parse.cc"
_db_level_ = 5
_db_framep_ = (char **) 0x5
#6 0x08262d9a in execute_sqlcom_select (thd=0xb0760008, all_tables=0x8fdb878) at sql_parse.cc:5051
lex = (LEX *) 0xb07610e0
result = (class select_result *) 0x9111bf8
res = 248
#7 0x0826eeef in mysql_execute_command (thd=0xb0760008) at sql_parse.cc:2246
res = 0
need_start_waiting = false
up_result = <value optimized out>
lex = (LEX *) 0xb07610e0
select_lex = (SELECT_LEX *) 0xb07613ac
first_table = (TABLE_LIST *) 0x8fdb878
all_tables = (TABLE_LIST *) 0x8fdb878
unit = (SELECT_LEX_UNIT *) 0xb076113c
have_table_map_for_update = <value optimized out>
_db_func_ = 0x86e2d52 "mysql_parse"
_db_file_ = 0x86e278b "sql_parse.cc"
_db_level_ = 4
_db_framep_ = (char **) 0xad66bcb8
__PRETTY_FUNCTION__ = "int mysql_execute_command(THD*)"
#8 0x082701e3 in mysql_parse (thd=0xb0760008,
inBuf=0x8fdb350 "SELECT SQL_NO_CACHE * FROM pages\nWHERE uid=388 AND pages.deleted=0 AND pages.hidden=0 AND\n(pages.starttime<=1259253720) AND\n(pages.endtime=0 OR pages.endtime>1259253720) AND NOT\n(pages.t3ver_state>0) "..., length=524, found_semicolon=0xad66c2d4) at sql_parse.cc:5970
lex = (LEX *) 0xb07610e0
parser_state = {m_lip = {m_thd = 0xb0760008, yylineno = 12, yytoklen = 2, yylval = 0xad66bc54, m_ptr = 0x8fdb55d "Ã‘\017K",
m_tok_start = 0x8fdb55d "Ã‘\017K", m_tok_end = 0x8fdb55d "Ã‘\017K", m_end_of_query = 0x8fdb55c "", m_tok_start_prev = 0x8fdb55c "",
m_buf = 0x8fdb350 "SELECT SQL_NO_CACHE * FROM pages\nWHERE uid=388 AND pages.deleted=0 AND pages.hidden=0 AND\n(pages.starttime<=1259253720) AND\n(pages.endtime=0 OR pages.endtime>1259253720) AND NOT\n(pages.t3ver_state>0) "..., m_buf_length = 524, m_echo = true, m_echo_saved = false,
m_cpp_buf = 0x8fdb5a8 "SELECT SQL_NO_CACHE * FROM pages\nWHERE uid=388 AND pages.deleted=0 AND pages.hidden=0 AND\n(pages.starttime<=1259253720) AND\n(pages.endtime=0 OR pages.endtime>1259253720) AND NOT\n(pages.t3ver_state>0) "..., m_cpp_ptr = 0x8fdb7b4 "", m_cpp_tok_start = 0x8fdb7b4 "",
m_cpp_tok_start_prev = 0x8fdb7b4 "", m_cpp_tok_end = 0x8fdb7b4 "", m_body_utf8 = 0x0, m_body_utf8_ptr = 0xb0760fe0 "\001",
m_cpp_utf8_processed_ptr = 0x0, next_state = MY_LEX_END, found_semicolon = 0x0, tok_bitmap = 63 '?', ignore_space = false,
stmt_prepare_mode = false, in_comment = NO_COMMENT, in_comment_saved = 2909191552, m_cpp_text_start = 0x8fdb7ae "-1')\n)",
m_cpp_text_end = 0x8fdb7b0 "')\n)", m_underscore_cs = 0x0}, m_yacc = {yacc_yyss = 0x0, yacc_yyvs = 0x0}}
err = <value optimized out>
_db_func_ = 0x86e2dbe "dispatch_command"
_db_file_ = 0x86e278b "sql_parse.cc"
_db_level_ = 3
_db_framep_ = (char **) 0xb7893c1c
__PRETTY_FUNCTION__ = "void mysql_parse(THD*, const char*, uint, const char**)"
#9 0x08271438 in dispatch_command (command=COM_QUERY, thd=0xb0760008,
packet=0xb0774961 "SELECT SQL_NO_CACHE * FROM pages\nWHERE uid=388 AND pages.deleted=0 AND pages.hidden=0 AND\n(pages.starttime<=1259253720) AND\n(pages.endtime=0 OR pages.endtime>1259253720) AND NOT\n(pages.t3ver_state>0) "..., packet_length=524) at sql_parse.cc:1231
packet_end = 0x8fdb55c ""
end_of_stmt = 0x0
net = (NET *) 0xb0760084
error = <value optimized out>
_db_func_ = 0x86e2e56 "do_command"
_db_file_ = 0x86e278b "sql_parse.cc"
_db_level_ = 2
_db_framep_ = (char **) 0xad66c2dc
#10 0x082725cc in do_command (thd=0xb0760008) at sql_parse.cc:872
return_value = <value optimized out>
packet = 0xb0774960 "\003SELECT SQL_NO_CACHE * FROM pages\nWHERE uid=388 AND pages.deleted=0 AND pages.hidden=0 AND\n(pages.starttime<=1259253720) AND\n(pages.endtime=0 OR pages.endtime>1259253720) AND NOT\n(pages.t3ver_state>0)"...
packet_length = 525
net = (NET *) 0xb0760084
command = COM_QUERY
_db_func_ = 0x87bdd74 "?func"
_db_file_ = 0x87bdd7a "?file"
_db_level_ = 1
_db_framep_ = (char **) 0x82429ba
__PRETTY_FUNCTION__ = "bool do_command(THD*)"
#11 0x08260f97 in handle_one_connection (arg=0xb0760008) at sql_connect.cc:1127
thd = (class THD *) 0xb0760008
#12 0xb789316f in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#13 0xb7697c0e in clone () from /lib/libc.so.6
No symbol table info available.
(gdb)

by **PaX Team** » Sat Dec 12, 2009 1:39 pm

dabetz wrote:I have tested with 5 different SQL querys. After about 400.000 to 1,2 million querys the mysql crashes with signal 11.

It crashes with 2.6.31.4-grsec ( grsecurity-2.1.14-2.6.31.4-200910202041 ) but not with an vanilla 2.6.31.4 Kernel.

what i'd need is also the extra info i asked for in the previous post: x/8i $pc, x/8x $sp, info reg, and the corresponding /proc/pid/maps file. these are necessary to see what insn was doing what at the time of the segfault. also could you try to patch in grsec but disable all of it and see if that crashes as well? if it doesn't then it'd help to determine which feature triggers the problem. last but not least, do you think you could create a simple mysql setup (table definitions, queries, etc) that i could play with here that reproduces the problem?

by **dabetz** » Mon Dec 14, 2009 4:21 am

Hello,

thanks for your reply.
Here are all the information you need.
Sorry, but i cant activate bbcode in my profile in this forum.

The setup is an Typo3 Database with Realurl enabled and has an huge cache_pages InnoDB Database. It has around 8 GB of Size. But i have seen crashes of MySQL5 with smaller tables.

I have tested with full disabled grsecurity echo "0" /proc/sys/kernel/grsecurity/* and PAX fully disabled in Kernel config.
Have tried PAX softmode too. MySQL keeps crashing with all settings.
Now i will try to disable alle grsecurity features in kernel, and will post results.

Tested now on 2.6.31.7-grsec ( grsecurity-2.1.14-2.6.31.7-200912111912 )

Informations from gdb:
[New Thread 0xae2ceb90 (LWP 15531)]
[Thread 0xb079db90 (LWP 15513) exited]
[New Thread 0xadad3b90 (LWP 15532)]
[Thread 0xadd81b90 (LWP 15519) exited]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xae29db90 (LWP 15522)]
0x0833872b in check_quick_keys ()
(gdb) backtrace full
#0 0x0833872b in check_quick_keys ()
No symbol table info available.
#1 0x0833879e in check_quick_keys ()
No symbol table info available.
#2 0x0833879e in check_quick_keys ()
No symbol table info available.
#3 0x08338a93 in check_quick_keys ()
No symbol table info available.
#4 0x0833879e in check_quick_keys ()
No symbol table info available.
#5 0x08339094 in check_quick_select ()
No symbol table info available.
#6 0x083393b4 in get_key_scans_params ()
No symbol table info available.
#7 0x083453f6 in SQL_SELECT::test_quick_select ()
No symbol table info available.
#8 0x082d37e5 in make_join_statistics ()
No symbol table info available.
#9 0x082d4c64 in JOIN::optimize ()
No symbol table info available.
#10 0x082d828c in mysql_select ()
No symbol table info available.
#11 0x082ddd7e in handle_select ()
No symbol table info available.
#12 0x0826c5e0 in execute_sqlcom_select ()
No symbol table info available.
#13 0x08273177 in mysql_execute_command ()
No symbol table info available.
#14 0x082769f0 in mysql_parse ()
No symbol table info available.
#15 0x08277c18 in dispatch_command ()
No symbol table info available.
#16 0x082784a0 in do_command ()
No symbol table info available.
#17 0x08268db3 in handle_one_connection ()
No symbol table info available.
#18 0xb77e516f in start_thread () from /lib/libpthread.so.0
No symbol table info available.
#19 0xb76e4c0e in clone () from /lib/libc.so.6
No symbol table info available.
(gdb) x/8i $pc
0x833872b <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+43>: movzbl 0x3(%ecx),%edx
0x833872f <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+47>: mov -0x94(%ebp),%ecx
0x8338735 <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+53>: movl $0x0,-0x78(%ebp)
0x833873c <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+60>: movl $0x0,-0x74(%ebp)
0x8338743 <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+67>: mov 0x248(%ecx),%eax
0x8338749 <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+73>: cmp %eax,%edx
0x833874b <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+75>: cmovae %edx,%eax
0x833874e <_ZL16check_quick_keysP5PARAMjP7SEL_ARGPhjiS3_ji+78>: cmpl $0x877c6e0,0x18(%ebx)
(gdb) x/8x $sp
0xae297500: 0xae297544 0x00000000 0x00000024 0x0000003d
0xae297510: 0x08cf90d6 0x0878bb2c 0x00107400 0x00000000
(gdb) info reg
eax 0x0 0
ecx 0x0 0
edx 0x2 2
ebx 0x0 0
esp 0xae297500 0xae297500
ebp 0xae2975b8 0xae2975b8
esi 0xffffffff -1
edi 0x0 0
eip 0x833872b 0x833872b <check_quick_keys(PARAM*, unsigned int, SEL_ARG*, unsigned char*, unsigned int, int, unsigned char*, unsigned int, int)+43>
eflags 0x210286 [ PF SF IF RF ID ]
cs 0x73 115
ss 0x7b 123
ds 0x7b 123
es 0x7b 123
fs 0x0 0
gs 0x33 51
(gdb)

File /proc/pid/maps:

08048000-08725000 r-xp 00000000 09:01 81395714 /usr/libexec/mysqld-500
08725000-08730000 r--p 006dc000 09:01 81395714 /usr/libexec/mysqld-500
08730000-08775000 rw-p 006e7000 09:01 81395714 /usr/libexec/mysqld-500
08775000-08f28000 rw-p 00000000 00:00 0 [heap]
ad66d000-ad66e000 ---p 00000000 00:00 0
ad66e000-ad69e000 rw-p 00000000 00:00 0
ad69e000-ad69f000 ---p 00000000 00:00 0
ad69f000-ad6cf000 rw-p 00000000 00:00 0
ad6cf000-ad6d0000 ---p 00000000 00:00 0
ad6d0000-ad700000 rw-p 00000000 00:00 0
ad700000-ad723000 rw-p 00000000 00:00 0
ad723000-ad800000 ---p 00000000 00:00 0
ad826000-ad827000 ---p 00000000 00:00 0
ad827000-ad857000 rw-p 00000000 00:00 0
ad857000-ad858000 ---p 00000000 00:00 0
ad858000-ad888000 rw-p 00000000 00:00 0
ad888000-ad889000 ---p 00000000 00:00 0
ad889000-ad8b9000 rw-p 00000000 00:00 0
ad8b9000-ad8ba000 ---p 00000000 00:00 0
ad8ba000-ad8ea000 rw-p 00000000 00:00 0
ad8ea000-ad8eb000 ---p 00000000 00:00 0
ad8eb000-ad91b000 rw-p 00000000 00:00 0
ad91b000-ad91c000 ---p 00000000 00:00 0
ad91c000-ad94c000 rw-p 00000000 00:00 0
ad94c000-ad94d000 ---p 00000000 00:00 0
ad94d000-ad97d000 rw-p 00000000 00:00 0
ad97d000-ad97e000 ---p 00000000 00:00 0
ad97e000-ad9ae000 rw-p 00000000 00:00 0
ad9ae000-ad9af000 ---p 00000000 00:00 0
ad9af000-ad9df000 rw-p 00000000 00:00 0
ad9df000-ad9e0000 ---p 00000000 00:00 0
ad9e0000-ada10000 rw-p 00000000 00:00 0
ada10000-ada11000 ---p 00000000 00:00 0
ada11000-ada41000 rw-p 00000000 00:00 0
ada41000-ada42000 ---p 00000000 00:00 0
ada42000-ada72000 rw-p 00000000 00:00 0
ada72000-ada73000 ---p 00000000 00:00 0
ada73000-adaa3000 rw-p 00000000 00:00 0
adaa3000-adaa4000 ---p 00000000 00:00 0
adaa4000-adad4000 rw-p 00000000 00:00 0
adad4000-adad5000 ---p 00000000 00:00 0
adad5000-adb05000 rw-p 00000000 00:00 0
adb05000-adb06000 ---p 00000000 00:00 0
adb06000-adb36000 rw-p 00000000 00:00 0
adb36000-adb37000 ---p 00000000 00:00 0
adb37000-adb67000 rw-p 00000000 00:00 0
adb67000-adb68000 ---p 00000000 00:00 0
adb68000-adb98000 rw-p 00000000 00:00 0
adb98000-adb99000 ---p 00000000 00:00 0
adb99000-adbc9000 rw-p 00000000 00:00 0
adbc9000-adbca000 ---p 00000000 00:00 0
adbca000-adbfa000 rw-p 00000000 00:00 0
adbfa000-adbfb000 ---p 00000000 00:00 0
adbfb000-adc2b000 rw-p 00000000 00:00 0
adc2b000-adc2c000 ---p 00000000 00:00 0
adc2c000-adc5c000 rw-p 00000000 00:00 0
adc5c000-adc5d000 ---p 00000000 00:00 0
adc5d000-adc8d000 rw-p 00000000 00:00 0
adc8d000-adc8e000 ---p 00000000 00:00 0
adc8e000-adcbe000 rw-p 00000000 00:00 0
adcbe000-adcbf000 ---p 00000000 00:00 0
adcbf000-adcef000 rw-p 00000000 00:00 0
adcef000-adcf0000 ---p 00000000 00:00 0
adcf0000-add20000 rw-p 00000000 00:00 0
add20000-add21000 ---p 00000000 00:00 0
add21000-add51000 rw-p 00000000 00:00 0
add51000-add52000 ---p 00000000 00:00 0
add52000-add82000 rw-p 00000000 00:00 0
add82000-add83000 ---p 00000000 00:00 0
add83000-addb3000 rw-p 00000000 00:00 0
addb3000-addb4000 ---p 00000000 00:00 0
addb4000-adde4000 rw-p 00000000 00:00 0
adde4000-adde5000 ---p 00000000 00:00 0
adde5000-ade15000 rw-p 00000000 00:00 0
ade15000-ade16000 ---p 00000000 00:00 0
ade16000-ade46000 rw-p 00000000 00:00 0
ade46000-ade47000 ---p 00000000 00:00 0
ade47000-ade77000 rw-p 00000000 00:00 0
ade83000-ade84000 ---p 00000000 00:00 0
ade84000-adeb4000 rw-p 00000000 00:00 0
adeb4000-adeb5000 ---p 00000000 00:00 0
adeb5000-adee5000 rw-p 00000000 00:00 0
adee5000-adee6000 ---p 00000000 00:00 0
adee6000-adf16000 rw-p 00000000 00:00 0
adf16000-adf17000 ---p 00000000 00:00 0
adf17000-adf47000 rw-p 00000000 00:00 0
adf47000-adf48000 ---p 00000000 00:00 0
adf48000-adf78000 rw-p 00000000 00:00 0
adf78000-adf79000 ---p 00000000 00:00 0
adf79000-adfa9000 rw-p 00000000 00:00 0
adfa9000-adfaa000 ---p 00000000 00:00 0
adfaa000-adfda000 rw-p 00000000 00:00 0
adfda000-adfdb000 ---p 00000000 00:00 0
adfdb000-ae00b000 rw-p 00000000 00:00 0
ae00b000-ae00c000 ---p 00000000 00:00 0
ae00c000-ae03c000 rw-p 00000000 00:00 0
ae03c000-ae03d000 ---p 00000000 00:00 0
ae03d000-ae06d000 rw-p 00000000 00:00 0
ae06d000-ae06e000 ---p 00000000 00:00 0
ae06e000-ae09e000 rw-p 00000000 00:00 0
ae09e000-ae09f000 ---p 00000000 00:00 0
ae09f000-ae0cf000 rw-p 00000000 00:00 0
ae0cf000-ae0d0000 ---p 00000000 00:00 0
ae0d0000-ae100000 rw-p 00000000 00:00 0
ae100000-ae200000 rw-p 00000000 00:00 0
ae20b000-ae20c000 ---p 00000000 00:00 0
ae20c000-ae23c000 rw-p 00000000 00:00 0
ae23c000-ae23d000 ---p 00000000 00:00 0
ae23d000-ae26d000 rw-p 00000000 00:00 0
ae26d000-ae26e000 ---p 00000000 00:00 0
ae26e000-ae29e000 rw-p 00000000 00:00 0
ae29e000-ae29f000 ---p 00000000 00:00 0
ae29f000-ae2cf000 rw-p 00000000 00:00 0
ae2cf000-ae2d0000 ---p 00000000 00:00 0
ae2d0000-ae300000 rw-p 00000000 00:00 0
ae300000-ae3f6000 rw-p 00000000 00:00 0
ae3f6000-ae400000 ---p 00000000 00:00 0
ae40d000-ae40e000 ---p 00000000 00:00 0
ae40e000-aec0e000 rw-p 00000000 00:00 0
aec0e000-aec0f000 ---p 00000000 00:00 0
aec0f000-af40f000 rw-p 00000000 00:00 0
af40f000-af410000 ---p 00000000 00:00 0
af410000-afc10000 rw-p 00000000 00:00 0
afc10000-afc11000 ---p 00000000 00:00 0
afc11000-b0617000 rw-p 00000000 00:00 0
b062f000-b0630000 ---p 00000000 00:00 0
b0630000-b0660000 rw-p 00000000 00:00 0
b0660000-b0661000 ---p 00000000 00:00 0
b0661000-b0691000 rw-p 00000000 00:00 0
b0691000-b0692000 ---p 00000000 00:00 0
b0692000-b06c2000 rw-p 00000000 00:00 0
b06cd000-b06ce000 ---p 00000000 00:00 0
b06ce000-b06fe000 rw-p 00000000 00:00 0
b06fe000-b06ff000 ---p 00000000 00:00 0
b06ff000-b072f000 rw-p 00000000 00:00 0
b073c000-b073d000 ---p 00000000 00:00 0
b073d000-b076d000 rw-p 00000000 00:00 0
b076d000-b076e000 ---p 00000000 00:00 0
b076e000-b079e000 rw-p 00000000 00:00 0
b079e000-b079f000 ---p 00000000 00:00 0
b079f000-b07cf000 rw-p 00000000 00:00 0
b07cf000-b07d0000 ---p 00000000 00:00 0
b07d0000-b0800000 rw-p 00000000 00:00 0
b0800000-b0900000 rw-p 00000000 00:00 0
b090f000-b0910000 ---p 00000000 00:00 0
b0910000-b0940000 rw-p 00000000 00:00 0
b0940000-b0941000 ---p 00000000 00:00 0
b0941000-b0971000 rw-p 00000000 00:00 0
b0971000-b0972000 ---p 00000000 00:00 0
b0972000-b09a2000 rw-p 00000000 00:00 0
b09a2000-b09a3000 ---p 00000000 00:00 0
b09a3000-b09d3000 rw-p 00000000 00:00 0
b09d3000-b09d4000 ---p 00000000 00:00 0
b09d4000-b0a04000 rw-p 00000000 00:00 0
b0a04000-b0a05000 ---p 00000000 00:00 0
b0a05000-b0a35000 rw-p 00000000 00:00 0
b0a35000-b0a36000 ---p 00000000 00:00 0
b0a36000-b1236000 rw-p 00000000 00:00 0
b1236000-b1237000 ---p 00000000 00:00 0
b1237000-b1a37000 rw-p 00000000 00:00 0
b1a37000-b1a38000 ---p 00000000 00:00 0
b1a38000-b6de7000 rw-p 00000000 00:00 0
b6de7000-b6df0000 r-xp 00000000 09:01 20055584 /lib/libnss_nis-2.9.so
b6df0000-b6df1000 r--p 00008000 09:01 20055584 /lib/libnss_nis-2.9.so
b6df1000-b6df2000 rw-p 00009000 09:01 20055584 /lib/libnss_nis-2.9.so
b6df2000-b6df9000 r-xp 00000000 09:01 20055580 /lib/libnss_compat-2.9.so
b6df9000-b6dfa000 r--p 00006000 09:01 20055580 /lib/libnss_compat-2.9.so
b6dfa000-b6dfb000 rw-p 00007000 09:01 20055580 /lib/libnss_compat-2.9.so
b6dfb000-b6e04000 r-xp 00000000 09:01 20055582 /lib/libnss_files-2.9.so
b6e04000-b6e05000 r--p 00008000 09:01 20055582 /lib/libnss_files-2.9.so
b6e05000-b6e06000 rw-p 00009000 09:01 20055582 /lib/libnss_files-2.9.so
b6e06000-b6e12000 r-xp 00000000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1
b6e12000-b6e13000 r--p 0000b000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1
b6e13000-b6e14000 rw-p 0000c000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1
b6e14000-b6e15000 ---p 00000000 00:00 0
b6e15000-b7616000 rw-p 00000000 00:00 0
b7616000-b7753000 r-xp 00000000 09:01 20055572 /lib/libc-2.9.so
b7753000-b7755000 r--p 0013d000 09:01 20055572 /lib/libc-2.9.so
b7755000-b7756000 rw-p 0013f000 09:01 20055572 /lib/libc-2.9.so
b7756000-b775a000 rw-p 00000000 00:00 0
b775a000-b777e000 r-xp 00000000 09:01 20055578 /lib/libm-2.9.so
b777e000-b777f000 r--p 00023000 09:01 20055578 /lib/libm-2.9.so
b777f000-b7780000 rw-p 00024000 09:01 20055578 /lib/libm-2.9.so
b7780000-b7793000 r-xp 00000000 09:01 20055579 /lib/libnsl-2.9.so
b7793000-b7794000 r--p 00012000 09:01 20055579 /lib/libnsl-2.9.so
b7794000-b7795000 rw-p 00013000 09:01 20055579 /lib/libnsl-2.9.so
b7795000-b7797000 rw-p 00000000 00:00 0
b7797000-b77a0000 r-xp 00000000 09:01 20055574 /lib/libcrypt-2.9.so
b77a0000-b77a1000 r--p 00008000 09:01 20055574 /lib/libcrypt-2.9.so
b77a1000-b77a2000 rw-p 00009000 09:01 20055574 /lib/libcrypt-2.9.so
b77a2000-b77c9000 rw-p 00000000 00:00 0
b77c9000-b77cb000 r-xp 00000000 09:01 20055575 /lib/libdl-2.9.so
b77cb000-b77cc000 r--p 00001000 09:01 20055575 /lib/libdl-2.9.so
b77cc000-b77cd000 rw-p 00002000 09:01 20055575 /lib/libdl-2.9.so
b77cd000-b77de000 r-xp 00000000 09:01 20054396 /lib/libz.so.1.2.3
b77de000-b77df000 rw-p 00010000 09:01 20054396 /lib/libz.so.1.2.3
b77df000-b77f3000 r-xp 00000000 09:01 20055591 /lib/libpthread-2.9.so
b77f3000-b77f4000 r--p 00013000 09:01 20055591 /lib/libpthread-2.9.so
b77f4000-b77f5000 rw-p 00014000 09:01 20055591 /lib/libpthread-2.9.so
b77f5000-b77f8000 rw-p 00000000 00:00 0
b77fd000-b77fe000 r-xp 00000000 00:00 0 [vdso]
b77fe000-b781a000 r-xp 00000000 09:01 20055569 /lib/ld-2.9.so
b781a000-b781b000 r--p 0001c000 09:01 20055569 /lib/ld-2.9.so
b781b000-b781c000 rw-p 0001d000 09:01 20055569 /lib/ld-2.9.so
bffeb000-c0000000 rw-p 00000000 00:00 0 [stack]

by **dabetz** » Mon Dec 14, 2009 7:22 am

Hello,

Kernel 2.6.31.7 with grsecurity patch, but fully disabled in Kernel config doesnt crash the mysql server.

Greetings,
Daniel

by **PaX Team** » Mon Dec 14, 2009 12:14 pm

dabetz wrote:Sorry, but i cant activate bbcode in my profile in this forum.

it's a spam prevention measure, it'll work after 4 posts or so

.

The setup is an Typo3 Database with Realurl enabled and has an huge cache_pages InnoDB Database. It has around 8 GB of Size. But i have seen crashes of MySQL5 with smaller tables.

ok, so for now it'll be up to you to do the debugging, i hope you have some time for this

.

I have tested with full disabled grsecurity echo "0" /proc/sys/kernel/grsecurity/* and PAX fully disabled in Kernel config.
Have tried PAX softmode too. MySQL keeps crashing with all settings.
Now i will try to disable alle grsecurity features in kernel, and will post results.

so, this means that it is a grsec and not a PaX feature that triggers the problem (it's a NULL deref in check_quick_keys). now there're two angles for further debugging: one is to find out which grsec option it is (by disabling them in .config, you can do a binary search to make it faster) and the second is to find out what pointer was NULL and how it got its value, then we can figure out what is really going on. unfortunately it's not easy to do all this by ourselves without access to your system, so i hope you can find some time and optionally a person in your company able to debug this in gdb. i'll try to look at the code myself based on the disasm, but chances are the real cause can be found at runtime only.

by **dabetz** » Tue Dec 15, 2009 3:16 am

Hello,

thank you for your help.
I will do some testing now by disabling grsecurity features one after another and give you feedback as soon as possible.

But its not easy to reproduze. Sometimes it crashes after about 400.000 querys / 10 minutes, but sometimes after more than 2 million querys. Im testing on an Core i7 3,07 Ghz with 12GB of RAM. The server makes about 730 querys per second with an small perl script that forks and generates random typo3 querys with SQL_NO_CACHE.
When you like it for debugging too, i can send it to you.

Greetings,
Daniel

by **dabetz** » Wed Dec 16, 2009 10:33 am

Hello,

ah BBCode is working now

have done some more testing, by disabling grsecurity functions groupwise.
After removing all options under "Kernel Auditing":

Code: Select all: --- config-old 2009-12-16 14:28:03.000000000 +0100 +++ .config 2009-12-16 14:28:59.000000000 +0100 @@ -2274,15 +2274,15 @@ # CONFIG_GRKERNSEC_AUDIT_GROUP is not set # CONFIG_GRKERNSEC_EXECLOG is not set -CONFIG_GRKERNSEC_RESLOG=y +# CONFIG_GRKERNSEC_RESLOG is not set # CONFIG_GRKERNSEC_CHROOT_EXECLOG is not set # CONFIG_GRKERNSEC_AUDIT_CHDIR is not set -CONFIG_GRKERNSEC_AUDIT_MOUNT=y -CONFIG_GRKERNSEC_SIGNAL=y -CONFIG_GRKERNSEC_FORKFAIL=y -CONFIG_GRKERNSEC_TIME=y -CONFIG_GRKERNSEC_PROC_IPADDR=y -CONFIG_GRKERNSEC_AUDIT_TEXTREL=y +# CONFIG_GRKERNSEC_AUDIT_MOUNT is not set +# CONFIG_GRKERNSEC_SIGNAL is not set +# CONFIG_GRKERNSEC_FORKFAIL is not set +# CONFIG_GRKERNSEC_TIME is not set +# CONFIG_GRKERNSEC_PROC_IPADDR is not set +# CONFIG_GRKERNSEC_AUDIT_TEXTREL is not set

the MySQL is running for over 6 million querys (normally crashes after 400k to 2million).

Code: Select all: Uptime: 8255 Threads: 45 Questions: 6036936 Slow queries: 0 Opens: 211 Flush tables: 1 Open tables: 204 Queries per second avg: 731.306

The next step is to enable all features (which i have disabled during tests) to the defaults, but keep "Kernel auditing" disabled and let it run over night.

Do you think, that there could be an bug in the Kernel auditing options, that may cause this crash problem, or was it pure luck ?

Greetings,
Daniel

by **dabetz** » Wed Dec 16, 2009 4:00 pm

Hmmm .. damn.

Now i got two different signals. Signal 6 and 11.

Heres the log:

Code: Select all: *** glibc detected *** /usr/sbin/mysqld-5: corrupted double-linked list: 0xac690ff8 *** ======= Backtrace: ========= /lib/libc.so.6[0xb75c0714] /lib/libc.so.6[0xb75c09b4] /lib/libc.so.6[0xb75c2885] /lib/libc.so.6(__libc_malloc+0x96)[0xb75c4886] /usr/sbin/mysqld-5(my_malloc+0x24)[0x85cac84] /usr/sbin/mysqld-5(init_dynamic_array2+0x6a)[0x85d3a2a] /usr/sbin/mysqld-5[0x82d231e] /usr/sbin/mysqld-5(_ZN4JOIN8optimizeEv+0x684)[0x82d4c64] /usr/sbin/mysqld-5(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x6c)[0x82d828c] /usr/sbin/mysqld-5(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x15e)[0x82ddd7e] /usr/sbin/mysqld-5[0x826c5e0] /usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x4157)[0x8273177] /usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x340)[0x82769f0] /usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1218)[0x8277c18] /usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe0)[0x82784a0] /usr/sbin/mysqld-5(handle_one_connection+0x253)[0x8268db3] /lib/libpthread.so.0[0xb772416f] /lib/libc.so.6(clone+0x5e)[0xb7623c0e] ======= Memory map: ======== 08048000-08725000 r-xp 00000000 09:01 81395714 /usr/libexec/mysqld-500 08725000-08730000 r--p 006dc000 09:01 81395714 /usr/libexec/mysqld-500 08730000-08775000 rw-p 006e7000 09:01 81395714 /usr/libexec/mysqld-500 08775000-09c3e000 rw-p 00000000 00:00 0 [heap] aab00000-aabff000 rw-p 00000000 00:00 0 aabff000-aac00000 ---p 00000000 00:00 0 aae00000-aaf00000 rw-p 00000000 00:00 0 aaf00000-aafd1000 rw-p 00000000 00:00 0 aafd1000-ab000000 ---p 00000000 00:00 0 ab100000-ab200000 rw-p 00000000 00:00 0 ab300000-ab3e3000 rw-p 00000000 00:00 0 ab3e3000-ab400000 ---p 00000000 00:00 0 ab500000-ab600000 rw-p 00000000 00:00 0 ab600000-ab700000 rw-p 00000000 00:00 0 ab800000-ab900000 rw-p 00000000 00:00 0 ab99e000-ab99f000 ---p 00000000 00:00 0 ab99f000-ab9cf000 rw-p 00000000 00:00 0 ab9cf000-ab9d0000 ---p 00000000 00:00 0 ab9d0000-aba00000 rw-p 00000000 00:00 0 aba00000-abb00000 rw-p 00000000 00:00 0 abb00000-abcff000 rw-p 00000000 00:00 0 abcff000-abd00000 ---p 00000000 00:00 0 abd00000-abd9b000 rw-p 00000000 00:00 0 abd9b000-abe00000 ---p 00000000 00:00 0 abe00000-abf00000 rw-p 00000000 00:00 0 abf1c000-abf1d000 ---p 00000000 00:00 0 abf1d000-abf4d000 rw-p 00000000 00:00 0 abf4d000-abf4e000 ---p 00000000 00:00 0 abf4e000-abf7e000 rw-p 00000000 00:00 0 abf7e000-abf7f000 ---p 00000000 00:00 0 abf7f000-abfaf000 rw-p 00000000 00:00 0 abfaf000-abfb0000 ---p 00000000 00:00 0 abfb0000-abfe0000 rw-p 00000000 00:00 0 abfe0000-abfe1000 ---p 00000000 00:00 0 abfe1000-ac011000 rw-p 00000000 00:00 0 ac011000-ac012000 ---p 00000000 00:00 0 ac012000-ac042000 rw-p 00000000 00:00 0 ac042000-ac043000 ---p 00000000 00:00 0 ac043000-ac073000 rw-p 00000000 00:00 0 ac073000-ac074000 ---p 00000000 00:00 0 ac074000-ac0a4000 rw-p 00000000 00:00 0 ac0a4000-ac0a5000 ---p 00000000 00:00 0 ac0a5000-ac0d5000 rw-p 00000000 00:00 0 ac0d5000-ac0d6000 ---p 00000000 00:00 0 ac0d6000-ac106000 rw-p 00000000 00:00 0 ac106000-ac107000 ---p 00000000 00:00 0 ac107000-ac137000 rw-p 00000000 00:00 0 ac137000-ac138000 ---p 00000000 00:00 0 ac138000-ac168000 rw-p 00000000 00:00 0 ac168000-ac169000 ---p 00000000 00:00 0 ac169000-ac199000 rw-p 00000000 00:00 0 ac199000-ac19a000 ---p 00000000 00:00 0 ac19a000-ac1ca000 rw-p 00000000 00:00 0 ac1ca000-ac1cb000 ---p 00000000 00:00 0 ac1cb000-ac1fb000 rw-p 00000000 00:00 0 ac1fb000-ac1fc000 ---p 00000000 00:00 0 ac1fc000-ac22c000 rw-p 00000000 00:00 0 ac22c000-ac22d000 ---p 00000000 00:00 0 ac22d000-ac25d000 rw-p 00000000 00:00 0 ac25d000-ac25e000 ---p 00000000 00:00 0 ac25e000-ac28e000 rw-p 00000000 00:00 0 ac28e000-ac28f000 ---p 00000000 00:00 0 ac28f000-ac2bf000 rw-p 00000000 00:00 0 ac2bf000-ac2c0000 ---p 00000000 00:00 0 ac2c0000-ac2f0000 rw-p 00000000 00:00 0 ac2f0000-ac2f1000 ---p 00000000 00:00 0 ac2f1000-ac321000 rw-p 00000000 00:00 0 ac321000-ac322000 ---p 00000000 00:00 0 ac322000-ac352000 rw-p 00000000 00:00 0 ac352000-ac353000 ---p 00000000 00:00 0 ac353000-ac383000 rw-p 00000000 00:00 0 ac383000-ac384000 ---p 00000000 00:00 0 ac384000-ac3b4000 rw-p 00000000 00:00 0 ac3b4000-ac3b5000 ---p 00000000 00:00 0 ac3b5000-ac3e5000 rw-p 00000000 00:00 0 ac3e5000-ac3e6000 ---p 00000000 00:00 0 ac3e6000-ac416000 rw-p 00000000 00:00 0 ac416000-ac417000 ---p 00000000 00:00 0 ac417000-ac447000 rw-p 00000000 00:00 0 ac447000-ac448000 ---p 00000000 00:00 0 ac448000-ac478000 rw-p 00000000 00:00 0 ac478000-ac479000 ---p 00000000 00:00 0 ac479000-ac4a9000 rw-p 00000000 00:00 0 ac4a9000-ac4aa000 ---p 00000000 00:00 0 ac4aa000-ac4da000 rw-p 00000000 00:00 0 ac4da000-ac4db000 ---p 00000000 00:00 0 ac4db000-ac50b000 rw-p 00000000 00:00 0 ac50b000-ac50c000 ---p 00000000 00:00 0 ac50c000-ac53c000 rw-p 00000000 00:00 0 ac53c000-ac53d000 ---p 00000000 00:00 0 ac53d000-ac56d000 rw-p 00000000 00:00 0 ac56d000-ac56e000 ---p 00000000 00:00 0 ac56e000-ac59e000 rw-p 00000000 00:00 0 ac59e000-ac59f000 ---p 00000000 00:00 0 ac59f000-ac5cf000 rw-p 00000000 00:00 0 ac5cf000-ac5d0000 ---p 00000000 00:00 0 ac5d0000-ac600000 rw-p 00000000 00:00 0 ac600000-ac6fa000 rw-p 00000000 00:00 0 ac6fa000-ac700000 ---p 00000000 00:00 0 ac70b000-ac70c000 ---p 00000000 00:00 0 ac70c000-ac73c000 rw-p 00000000 00:00 0 ac73c000-ac73d000 ---p 00000000 00:00 0 ac73d000-ac76d000 rw-p 00000000 00:00 0 ac76d000-ac76e000 ---p 00000000 00:00 0 ac76e000-ac79e000 rw-p 00000000 00:00 0 ac79e000-ac79f000 ---p 00000000 00:00 0 ac79f000-ac7cf000 rw-p 00000000 00:00 0 ac7cf000-ac7d0000 ---p 00000000 00:00 0 ac7d0000-ac800000 rw-p 00000000 00:00 0 ac800000-ac900000 rw-p 00000000 00:00 0 ac900000-aca00000 rw-p 00000000 00:00 0 aca00000-acb00000 rw-p 00000000 00:00 0 acb00000-acbc8000 rw-p 00000000 00:00 0 acbc8000-acc00000 ---p 00000000 00:00 0 acc0b000-acc0c000 ---p 00000000 00:00 0 acc0c000-acc3c000 rw-p 00000000 00:00 0 acc3c000-acc3d000 ---p 00000000 00:00 0 acc3d000-acc6d000 rw-p 00000000 00:00 0 acc6d000-acc6e000 ---p 00000000 00:00 0 acc6e000-acc9e000 rw-p 00000000 00:00 0 acc9e000-acc9f000 ---p 00000000 00:00 0 acc9f000-acccf000 rw-p 00000000 00:00 0 acccf000-accd0000 ---p 00000000 00:00 0 accd0000-acd00000 rw-p 00000000 00:00 0 acd00000-acdfc000 rw-p 00000000 00:00 0 acdfc000-ace00000 ---p 00000000 00:00 0 ace00000-acf00000 rw-p 00000000 00:00 0 acf0b000-acf0c000 ---p 00000000 00:00 0 acf0c000-acf3c000 rw-p 00000000 00:00 0 acf3c000-acf3d000 ---p 00000000 00:00 0 acf3d000-acf6d000 rw-p 00000000 00:00 0 acf6d000-acf6e000 ---p 00000000 00:00 0 acf6e000-acf9e000 rw-p 00000000 00:00 0 acf9e000-acf9f000 ---p 00000000 00:00 0 acf9f000-acfcf000 rw-p 00000000 00:00 0 acfcf000-acfd0000 ---p 00000000 00:00 0 acfd0000-ad000000 rw-p 00000000 00:00 0 ad000000-ad100000 rw-p 00000000 00:00 0 ad10b000-ad10c000 ---p 00000000 00:00 0 ad10c000-ad13c000 rw-p 00000000 00:00 0 ad13c000-ad13d000 ---p 00000000 00:00 0 ad13d000-ad16d000 rw-p 00000000 00:00 0 ad16d000-ad16e000 ---p 00000000 00:00 0 ad16e000-ad19e000 rw-p 00000000 00:00 0 ad19e000-ad19f000 ---p 00000000 00:00 0 ad19f000-ad1cf000 rw-p 00000000 00:00 0 ad1cf000-ad1d0000 ---p 00000000 00:00 0 ad1d0000-ad200000 rw-p 00000000 00:00 0 ad200000-ad300000 rw-p 00000000 00:00 0 ad307000-ad308000 ---p 00000000 00:00 0 ad308000-ad338000 rw-p 00000000 00:00 0 ad338000-ad339000 ---p 00000000 00:00 0 ad339000-ad369000 rw-p 00000000 00:00 0 ad369000-ad36a000 ---p 00000000 00:00 0 ad36a000-ad39a000 rw-p 00000000 00:00 0 ad39a000-ad39b000 ---p 00000000 00:00 0 ad39b000-ad3cb000 rw-p 00000000 00:00 0 ad3cb000-ad3cc000 ---p 00000000 00:00 0 ad3cc000-ad3fc000 rw-p 00000000 00:00 0 ad3fc000-ad3fd000 ---p 00000000 00:00 0 ad3fd000-ad42d000 rw-p 00000000 00:00 0 ad42d000-ad42e000 ---p 00000000 00:00 0 ad42e000-ad45e000 rw-p 00000000 00:00 0 ad45e000-ad45f000 ---p 00000000 00:00 0 ad45f000-ad48f000 rw-p 00000000 00:00 0 ad48f000-ad490000 ---p 00000000 00:00 0 ad490000-ad4c0000 rw-p 00000000 00:00 0 ad4c0000-ad4c1000 ---p 00000000 00:00 0 ad4c1000-ad4f1000 rw-p 00000000 00:00 0 ad4f1000-ad4f2000 ---p 00000000 00:00 0 ad4f2000-ad522000 rw-p 00000000 00:00 0 ad522000-ad523000 ---p 00000000 00:00 0 ad523000-ad553000 rw-p 00000000 00:00 0 ad553000-ad554000 ---p 00000000 00:00 0 ad554000-ad584000 rw-p 00000000 00:00 0 ad584000-ad585000 ---p 00000000 00:00 0 ad585000-ad5b5000 rw-p 00000000 00:00 0 ad5b5000-ad5b6000 ---p 00000000 00:00 0 ad5b6000-ad5e6000 rw-p 00000000 00:00 0 ad5e6000-ad5e7000 ---p 00000000 00:00 0 ad5e7000-ad617000 rw-p 00000000 00:00 0 ad617000-ad618000 ---p 00000000 00:00 0 ad618000-ad648000 rw-p 00000000 00:00 0 ad648000-ad649000 ---p 00000000 00:00 0 ad649000-ad679000 rw-p 00000000 00:00 0 ad679000-ad67a000 ---p 00000000 00:00 0 ad67a000-ad6aa000 rw-p 00000000 00:00 0 ad6aa000-ad6ab000 ---p 00000000 00:00 0 ad6ab000-ad6db000 rw-p 00000000 00:00 0 ad6db000-ad6dc000 ---p 00000000 00:00 0 ad6dc000-ad70c000 rw-p 00000000 00:00 0 ad70c000-ad70d000 ---p 00000000 00:00 0 ad70d000-ad73d000 rw-p 00000000 00:00 0 ad73d000-ad73e000 ---p 00000000 00:00 0 ad73e000-ad76e000 rw-p 00000000 00:00 0 ad76e000-ad76f000 ---p 00000000 00:00 0 ad76f000-ad79f000 rw-p 00000000 00:00 0 ad79f000-ad7a0000 ---p 00000000 00:00 0 ad7a0000-ad7d0000 rw-p 00000000 00:00 0 ad7d0000-ad7d1000 ---p 00000000 00:00 0 ad7d1000-ad801000 rw-p 00000000 00:00 0 ad801000-ad802000 ---p 00000000 00:00 0 ad802000-ad832000 rw-p 00000000 00:00 0 ad832000-ad833000 ---p 00000000 00:00 0 ad833000-ad863000 rw-p 00000000 00:00 0 ad863000-ad864000 ---p 00000000 00:00 0 ad864000-ad894000 rw-p 00000000 00:00 0 ad894000-ad895000 ---p 00000000 00:00 0 ad895000-ad8c5000 rw-p 00000000 00:00 0 ad8c5000-ad8c6000 ---p 00000000 00:00 0 ad8c6000-ad8f6000 rw-p 00000000 00:00 0 ad8f6000-ad8f7000 ---p 00000000 00:00 0 ad8f7000-ad927000 rw-p 00000000 00:00 0 ad927000-ad928000 ---p 00000000 00:00 0 ad928000-ad958000 rw-p 00000000 00:00 0 ad958000-ad959000 ---p 00000000 00:00 0 ad959000-ad989000 rw-p 00000000 00:00 0 ad989000-ad98a000 ---p 00000000 00:00 0 ad98a000-ad9ba000 rw-p 00000000 00:00 0 ad9ba000-ad9bb000 ---p 00000000 00:00 0 ad9bb000-ad9eb000 rw-p 00000000 00:00 0 ad9eb000-ad9ec000 ---p 00000000 00:00 0 ad9ec000-ada1c000 rw-p 00000000 00:00 0 ada1c000-ada1d000 ---p 00000000 00:00 0 ada1d000-ada4d000 rw-p 00000000 00:00 0 ada4d000-ada4e000 ---p 00000000 00:00 0 ada4e000-ada7e000 rw-p 00000000 00:00 0 ada7e000-ada7f000 ---p 00000000 00:00 0 ada7f000-adaaf000 rw-p 00000000 00:00 0 adaaf000-adab0000 ---p 00000000 00:00 0 adab0000-adae0000 rw-p 00000000 00:00 0 adae0000-adae1000 ---p 00000000 00:00 0 adae1000-adb11000 rw-p 00000000 00:00 0 adb11000-adb12000 ---p 00000000 00:00 0 adb12000-adb42000 rw-p 00000000 00:00 0 adb42000-adb43000 ---p 00000000 00:00 0 adb43000-adb73000 rw-p 00000000 00:00 0 adb73000-adb74000 ---p 00000000 00:00 0 adb74000-adba4000 rw-p 00000000 00:00 0 adba4000-adba5000 ---p 00000000 00:00 0 adba5000-adbd5000 rw-p 00000000 00:00 0 adbd5000-adbd6000 ---p 00000000 00:00 0 adbd6000-adc06000 rw-p 00000000 00:00 0 adc06000-adc07000 ---p 00000000 00:00 0 adc07000-adc37000 rw-p 00000000 00:00 0 adc37000-adc38000 ---p 00000000 00:00 0 adc38000-adc68000 rw-p 00000000 00:00 0 adc68000-adc69000 ---p 00000000 00:00 0 adc69000-adc99000 rw-p 00000000 00:00 0 adc99000-adc9a000 ---p 00000000 00:00 0 adc9a000-adcca000 rw-p 00000000 00:00 0 adcca000-adccb000 ---p 00000000 00:00 0 adccb000-adcfb000 rw-p 00000000 00:00 0 adcfb000-adcfc000 ---p 00000000 00:00 0 adcfc000-add2c000 rw-p 00000000 00:00 0 add2c000-add2d000 ---p 00000000 00:00 0 add2d000-add5d000 rw-p 00000000 00:00 0 add5d000-add5e000 ---p 00000000 00:00 0 add5e000-add8e000 rw-p 00000000 00:00 0 add8e000-add8f000 ---p 00000000 00:00 0 add8f000-addbf000 rw-p 00000000 00:00 0 addbf000-addc0000 ---p 00000000 00:00 0 addc0000-addf0000 rw-p 00000000 00:00 0 addf0000-addf1000 ---p 00000000 00:00 0 addf1000-ade21000 rw-p 00000000 00:00 0 ade21000-ade22000 ---p 00000000 00:00 0 ade22000-ade52000 rw-p 00000000 00:00 0 ade52000-ade53000 ---p 00000000 00:00 0 ade53000-ade83000 rw-p 00000000 00:00 0 ade83000-ade84000 ---p 00000000 00:00 0 ade84000-adeb4000 rw-p 00000000 00:00 0 adeb4000-adeb5000 ---p 00000000 00:00 0 adeb5000-adee5000 rw-p 00000000 00:00 0 adee5000-adee6000 ---p 00000000 00:00 0 adee6000-adf16000 rw-p 00000000 00:00 0 adf16000-adf17000 ---p 00000000 00:00 0 adf17000-adf47000 rw-p 00000000 00:00 0 adf47000-adf48000 ---p 00000000 00:00 0 adf48000-adf78000 rw-p 00000000 00:00 0 adf78000-adf79000 ---p 00000000 00:00 0 adf79000-adfa9000 rw-p 00000000 00:00 0 adfa9000-adfaa000 ---p 00000000 00:00 0 adfaa000-adfda000 rw-p 00000000 00:00 0 adfda000-adfdb000 ---p 00000000 00:00 0 adfdb000-ae00b000 rw-p 00000000 00:00 0 ae00b000-ae00c000 ---p 00000000 00:00 0 ae00c000-ae03c000 rw-p 00000000 00:00 0 ae03c000-ae03d000 ---p 00000000 00:00 0 ae03d000-ae06d000 rw-p 00000000 00:00 0 ae06d000-ae06e000 ---p 00000000 00:00 0 ae06e000-ae09e000 rw-p 00000000 00:00 0 ae09e000-ae09f000 ---p 00000000 00:00 0 ae09f000-ae0cf000 rw-p 00000000 00:00 0 ae0cf000-ae0d0000 ---p 00000000 00:00 0 ae0d0000-ae100000 rw-p 00000000 00:00 0 ae100000-ae101000 ---p 00000000 00:00 0 ae101000-ae131000 rw-p 00000000 00:00 0 ae131000-ae132000 ---p 00000000 00:00 0 ae132000-ae162000 rw-p 00000000 00:00 0 ae162000-ae163000 ---p 00000000 00:00 0 ae163000-ae193000 rw-p 00000000 00:00 0 ae193000-ae194000 ---p 00000000 00:00 0 ae194000-ae1c4000 rw-p 00000000 00:00 0 ae1c4000-ae1c5000 ---p 00000000 00:00 0 ae1c5000-ae1f5000 rw-p 00000000 00:00 0 ae1f5000-ae1f6000 ---p 00000000 00:00 0 ae1f6000-ae226000 rw-p 00000000 00:00 0 ae226000-ae227000 ---p 00000000 00:00 0 ae227000-ae257000 rw-p 00000000 00:00 0 ae257000-ae258000 ---p 00000000 00:00 0 ae258000-ae288000 rw-p 00000000 00:00 0 ae288000-ae289000 ---p 00000000 00:00 0 ae289000-ae2b9000 rw-p 00000000 00:00 0 ae2b9000-ae2ba000 ---p 00000000 00:00 0 ae2ba000-ae2ea000 rw-p 00000000 00:00 0 ae2ea000-ae2eb000 ---p 00000000 00:00 0 ae2eb000-ae31b000 rw-p 00000000 00:00 0 ae31b000-ae31c000 ---p 00000000 00:00 0 ae31c000-ae34c000 rw-p 00000000 00:00 0 ae34c000-ae34d000 ---p 00000000 00:00 0 ae34d000-aeb4d000 rw-p 00000000 00:00 0 aeb4d000-aeb4e000 ---p 00000000 00:00 0 aeb4e000-af34e000 rw-p 00000000 00:00 0 af34e000-af34f000 ---p 00000000 00:00 0 af34f000-afb4f000 rw-p 00000000 00:00 0 afb4f000-afb50000 ---p 00000000 00:00 0 afb50000-b0556000 rw-p 00000000 00:00 0 b0583000-b0584000 ---p 00000000 00:00 0 b0584000-b05b4000 rw-p 00000000 00:00 0 b05b4000-b05b5000 ---p 00000000 00:00 0 b05b5000-b05e5000 rw-p 00000000 00:00 0 b05e5000-b05e6000 ---p 00000000 00:00 0 b05e6000-b0616000 rw-p 00000000 00:00 0 b0616000-b0617000 ---p 00000000 00:00 0 b0617000-b0647000 rw-p 00000000 00:00 0 b0647000-b0648000 ---p 00000000 00:00 0 b0648000-b0678000 rw-p 00000000 00:00 0 b0678000-b0679000 ---p 00000000 00:00 0 b0679000-b06a9000 rw-p 00000000 00:00 0 b06a9000-b06aa000 ---p 00000000 00:00 0 b06aa000-b06da000 rw-p 00000000 00:00 0 b06da000-b06db000 ---p 00000000 00:00 0 b06db000-b070b000 rw-p 00000000 00:00 0 b070b000-b070c000 ---p 00000000 00:00 0 b070c000-b073c000 rw-p 00000000 00:00 0 b073c000-b073d000 ---p 00000000 00:00 0 b073d000-b076d000 rw-p 00000000 00:00 0 b076d000-b076e000 ---p 00000000 00:00 0 b076e000-b079e000 rw-p 00000000 00:00 0 b079e000-b079f000 ---p 00000000 00:00 0 b079f000-b07cf000 rw-p 00000000 00:00 0 b07cf000-b07d0000 ---p 00000000 00:00 0 b07d0000-b0800000 rw-p 00000000 00:00 0 b0800000-b0900000 rw-p 00000000 00:00 0 b0912000-b0913000 ---p 00000000 00:00 0 b0913000-b0943000 rw-p 00000000 00:00 0 b0943000-b0944000 ---p 00000000 00:00 0 b0944000-b0974000 rw-p 00000000 00:00 0 b0974000-b0975000 ---p 00000000 00:00 0 b0975000-b1175000 rw-p 00000000 00:00 0 b1175000-b1176000 ---p 00000000 00:00 0 b1176000-b1976000 rw-p 00000000 00:00 0 b1976000-b1977000 ---p 00000000 00:00 0 b1977000-b6d26000 rw-p 00000000 00:00 0 b6d26000-b6d2f000 r-xp 00000000 09:01 20055584 /lib/libnss_nis-2.9.so b6d2f000-b6d30000 r--p 00008000 09:01 20055584 /lib/libnss_nis-2.9.so b6d30000-b6d31000 rw-p 00009000 09:01 20055584 /lib/libnss_nis-2.9.so b6d31000-b6d38000 r-xp 00000000 09:01 20055580 /lib/libnss_compat-2.9.so b6d38000-b6d39000 r--p 00006000 09:01 20055580 /lib/libnss_compat-2.9.so b6d39000-b6d3a000 rw-p 00007000 09:01 20055580 /lib/libnss_compat-2.9.so b6d3a000-b6d43000 r-xp 00000000 09:01 20055582 /lib/libnss_files-2.9.so b6d43000-b6d44000 r--p 00008000 09:01 20055582 /lib/libnss_files-2.9.so b6d44000-b6d45000 rw-p 00009000 09:01 20055582 /lib/libnss_files-2.9.so b6d45000-b6d51000 r-xp 00000000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6d51000-b6d52000 r--p 0000b000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6d52000-b6d53000 rw-p 0000c000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6d53000-b6d54000 ---p 00000000 00:00 0 b6d54000-b7555000 rw-p 00000000 00:00 0 b7555000-b7692000 r-xp 00000000 09:01 20055572 /lib/libc-2.9.so b7692000-b7694000 r--p 0013d000 09:01 20055572 /lib/libc-2.9.so b7694000-b7695000 rw-p 0013f000 09:01 20055572 /lib/libc-2.9.so b7695000-b7699000 rw-p 00000000 00:00 0 b7699000-b76bd000 r-xp 00000000 09:01 20055578 /lib/libm-2.9.so b76bd000-b76be000 r--p 00023000 09:01 20055578 /lib/libm-2.9.so b76be000-b76bf000 rw-p 00024000 09:01 20055578 /lib/libm-2.9.so b76bf000-b76d2000 r-xp 00000000 09:01 20055579 /lib/libnsl-2.9.so b76d2000-b76d3000 r--p 00012000 09:01 20055579 /lib/libnsl-2.9.so b76d3000-b76d4000 rw-p 00013000 09:01 20055579 /lib/libnsl-2.9.so b76d4000-b76d6000 rw-p 00000000 00:00 0 b76d6000-b76df000 r-xp 00000000 09:01 20055574 /lib/libcrypt-2.9.so b76df000-b76e0000 r--p 00008000 09:01 20055574 /lib/libcrypt-2.9.so b76e0000-b76e1000 rw-p 00009000 09:01 20055574 /lib/libcrypt-2.9.so b76e1000-b7708000 rw-p 00000000 00:00 0 b7708000-b770a000 r-xp 00000000 09:01 20055575 /lib/libdl-2.9.so b770a000-b770b000 r--p 00001000 09:01 20055575 /lib/libdl-2.9.so b770b000-b770c000 rw-p 00002000 09:01 20055575 /lib/libdl-2.9.so b770c000-b771d000 r-xp 00000000 09:01 20054396 /lib/libz.so.1.2.3 b771d000-b771e000 rw-p 00010000 09:01 20054396 /lib/libz.so.1.2.3 b771e000-b7732000 r-xp 00000000 09:01 20055591 /lib/libpthread-2.9.so b7732000-b7733000 r--p 00013000 09:01 20055591 /lib/libpthread-2.9.so b7733000-b7734000 rw-p 00014000 09:01 20055591 /lib/libpthread-2.9.so b7734000-b7737000 rw-p 00000000 00:00 0 b773c000-b773d000 r-xp 00000000 00:00 0 [vdso] b773d000-b7759000 r-xp 00000000 09:01 20055569 /lib/ld-2.9.so b7759000-b775a000 r--p 0001c000 09:01 20055569 /lib/ld-2.9.so b775a000-b775b000 rw-p 0001d000 09:01 20055569 /lib/ld-2.9.so bffea000-c0000000 rw-p 00000000 00:00 0 [stack] 091216 18:22:01 - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=1048576 max_used_connections=98 max_threads=500 threads_connected=44 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1571947 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd: 0xacbba378 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0xac4773b8 thread_stack 0x30000 /usr/sbin/mysqld-5(my_print_stacktrace+0x21)[0x85def51] /usr/sbin/mysqld-5(handle_segfault+0x381)[0x825e8f1] [0xb773c400] /lib/libc.so.6(abort+0x188)[0xb7580e98] /lib/libc.so.6[0xb75ba83d] /lib/libc.so.6[0xb75c0714] /lib/libc.so.6[0xb75c09b4] /lib/libc.so.6[0xb75c2885] /lib/libc.so.6(__libc_malloc+0x96)[0xb75c4886] /usr/sbin/mysqld-5(my_malloc+0x24)[0x85cac84] /usr/sbin/mysqld-5(init_dynamic_array2+0x6a)[0x85d3a2a] /usr/sbin/mysqld-5[0x82d231e] /usr/sbin/mysqld-5(_ZN4JOIN8optimizeEv+0x684)[0x82d4c64] /usr/sbin/mysqld-5(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x6c)[0x82d828c] /usr/sbin/mysqld-5(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x15e)[0x82ddd7e] /usr/sbin/mysqld-5[0x826c5e0] /usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x4157)[0x8273177] /usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x340)[0x82769f0] /usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1218)[0x8277c18] /usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe0)[0x82784a0] /usr/sbin/mysqld-5(handle_one_connection+0x253)[0x8268db3] /lib/libpthread.so.0[0xb772416f] /lib/libc.so.6(clone+0x5e)[0xb7623c0e] Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at 0xac67ea18 is an invalid pointer thd->thread_id=2690790 thd->killed=NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 091216 18:32:31 [Note] Plugin 'FEDERATED' is disabled. 091216 18:32:31 [Note] Plugin 'ndbcluster' is disabled. InnoDB: Log scan progressed past the checkpoint lsn 1 3204510106 091216 18:32:31 InnoDB: Database was not shut down normally! InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Doing recovery: scanned up to log sequence number 1 3204510116 091216 18:32:32 InnoDB: Started; log sequence number 1 3204510116 091216 18:32:32 [Note] Event Scheduler: Loaded 0 events 091216 18:32:32 [Note] /usr/sbin/mysqld-5: ready for connections. Version: '5.1.39-log' socket: '/var/lib/mysql/mysql5.sock' port: 3306 Source distribution 091216 19:06:15 - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=1048576 max_used_connections=108 max_threads=500 threads_connected=10 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1571947 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd: 0xac31ff10 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0xad8153b8 thread_stack 0x30000 /usr/sbin/mysqld-5(my_print_stacktrace+0x21)[0x85def51] /usr/sbin/mysqld-5(handle_segfault+0x381)[0x825e8f1] [0xb782d400] /usr/sbin/mysqld-5(_Z17mysql_lock_tablesP3THDPP8st_tablejjPb+0x3c4)[0x8257414] /usr/sbin/mysqld-5(_Z11lock_tablesP3THDP10TABLE_LISTjPb+0x470)[0x82a9700] /usr/sbin/mysqld-5(_Z28open_and_lock_tables_derivedP3THDP10TABLE_LISTb+0x5d)[0x82afddd] /usr/sbin/mysqld-5[0x826c57a] /usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x4157)[0x8273177] /usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x340)[0x82769f0] /usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1218)[0x8277c18] /usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe0)[0x82784a0] /usr/sbin/mysqld-5(handle_one_connection+0x253)[0x8268db3] /lib/libpthread.so.0[0xb781516f] /lib/libc.so.6(clone+0x5e)[0xb7714c0e] Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at 0x8c395c0 = SELECT SQL_NO_CACHE count(*) FROM tt_news WHERE tt_news.pid IN (17,177,156,48,287,288,289,290,291,292,293,294,295) AND (tt_news.user_ikcategorisation_countries='252' OR tt_news.user_ikcategorisation_countries LIKE '%,252' OR tt_news.user_ikcategorisation_countries LIKE '252,%' OR tt_news.user_ikcategorisation_countries LIKE '%,252,%') AND (tt_news.tx_iknewsfields_workgroup = 0 OR tt_news.tx_iknewsfields_workgroup = 1 OR tt_news.tx_iknewsfields_workgroup = '' OR tt_news.tx_iknewsfields_workgroup LIKE '%1%,' OR tt_news.tx_iknewsfields_workgroup LIKE '%,1%' OR tt_news.tx_iknewsfields_workgroup LIKE '%,1,%' OR tt_news.tx_iknewsfields_workgroup LIKE '%0%,' OR tt_news.tx_iknewsfields_workgroup LIKE '%,0%' OR tt_news.tx_iknewsfields_workgroup LIKE '%,0,%') AND 1=1 AND type =10 AND tt_news.deleted=0 AND tt_news.t3ver_state<=0 AND tt_news.hidden=0 AND (tt_news.starttime<=1259327880) AND (tt_news.endtime=0 OR tt_news.endtime>1259327880) AND (tt_news.fe_group='' OR tt_news.fe_group IS NULL OR tt_news.fe_group='0') thd->thread_id=498017 thd->killed=NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.

Greetings,
Daniel

by **PaX Team** » Thu Dec 17, 2009 7:11 pm

dabetz wrote:have done some more testing, by disabling grsecurity functions groupwise.

could you just disable CONFIG_GRKERNSEC_RESLOG and see if you still get the problem? also, do you get any kernel logs around the time the problem triggers?

by **dabetz** » Fri Dec 18, 2009 3:00 am

Hello,

i have disabled all grsecurity options in kernel and the mysql crashes. :cry:

Now ive got an signal 11.

gdb

Code: Select all: [Thread 0xad43bb90 (LWP 24201) exited] [Thread 0xae3d6b90 (LWP 24187) exited] [Thread 0xade51b90 (LWP 24203) exited] [Thread 0xadc0ab90 (LWP 24185) exited] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xad098b90 (LWP 24179)] 0x0833c40c in key_or () (gdb) bt full #0 0x0833c40c in key_or () No symbol table info available. #1 0x0833f861 in tree_or () No symbol table info available. #2 0x08342d2a in get_func_mm_tree () No symbol table info available. #3 0x0834307a in get_full_func_mm_tree () No symbol table info available. #4 0x08343399 in get_mm_tree () No symbol table info available. #5 0x08343260 in get_mm_tree () No symbol table info available. #6 0x0834473f in SQL_SELECT::test_quick_select () No symbol table info available. #7 0x082d37e5 in make_join_statistics () No symbol table info available. #8 0x082d4c64 in JOIN::optimize () No symbol table info available. #9 0x082d828c in mysql_select () No symbol table info available. #10 0x082ddd7e in handle_select () No symbol table info available. #11 0x0826c5e0 in execute_sqlcom_select () No symbol table info available. #12 0x08273177 in mysql_execute_command () No symbol table info available. #13 0x082769f0 in mysql_parse () No symbol table info available. #14 0x08277c18 in dispatch_command () No symbol table info available. #15 0x082784a0 in do_command () No symbol table info available. #16 0x08268db3 in handle_one_connection () No symbol table info available. #17 0xb781216f in start_thread () from /lib/libpthread.so.0 No symbol table info available. #18 0xb7711c0e in clone () from /lib/libc.so.6 No symbol table info available. (gdb) x/8i $pc 0x833c40c <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+204>: mov 0x18(%eax),%eax 0x833c40f <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+207>: cmp $0x877c6e0,%eax 0x833c414 <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+212>: jne 0x833c40a <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+202> 0x833c416 <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+214>: mov %ecx,-0x288(%ebp) 0x833c41c <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+220>: mov -0x284(%ebp),%edi 0x833c422 <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+226>: mov %edi,-0x9c(%ebp) 0x833c428 <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+232>: xor %esi,%esi 0x833c42a <_ZL6key_orP15RANGE_OPT_PARAMP7SEL_ARGS2_+234>: cmpl $0x877c6e0,-0x284(%ebp) (gdb) x/8x $sp 0xad092560: 0xad5061d0 0xad5cfe78 0xad5cfe70 0x0833c16f 0xad092570: 0xad568280 0x08c98930 0x08c987c0 0x08330e51 (gdb) info reg eax 0x0 0 ecx 0x0 0 edx 0x0 0 ebx 0xad5cffe8 -1386414104 esp 0xad092560 0xad092560 ebp 0xad092808 0xad092808 esi 0xad506150 -1387241136 edi 0xad5cffe8 -1386414104 eip 0x833c40c 0x833c40c <key_or(RANGE_OPT_PARAM*, SEL_ARG*, SEL_ARG*)+204> eflags 0x210283 [ CF SF IF RF ID ] cs 0x73 115 ss 0x7b 123 ds 0x7b 123 es 0x7b 123 fs 0x0 0 gs 0x33 51 (gdb)

/proc/pid/maps

Code: Select all: 08048000-08725000 r-xp 00000000 09:01 81395714 /usr/libexec/mysqld-500 08725000-08730000 r--p 006dc000 09:01 81395714 /usr/libexec/mysqld-500 08730000-08775000 rw-p 006e7000 09:01 81395714 /usr/libexec/mysqld-500 08775000-08fb1000 rw-p 00000000 00:00 0 [heap] acc9e000-acc9f000 ---p 00000000 00:00 0 acc9f000-acccf000 rw-p 00000000 00:00 0 acccf000-accd0000 ---p 00000000 00:00 0 accd0000-acd00000 rw-p 00000000 00:00 0 acd00000-acdd2000 rw-p 00000000 00:00 0 acdd2000-ace00000 ---p 00000000 00:00 0 ace1c000-ace1d000 ---p 00000000 00:00 0 ace1d000-ace4d000 rw-p 00000000 00:00 0 ace4d000-ace4e000 ---p 00000000 00:00 0 ace4e000-ace7e000 rw-p 00000000 00:00 0 ace7e000-ace7f000 ---p 00000000 00:00 0 ace7f000-aceaf000 rw-p 00000000 00:00 0 aceaf000-aceb0000 ---p 00000000 00:00 0 aceb0000-acee0000 rw-p 00000000 00:00 0 acee0000-acee1000 ---p 00000000 00:00 0 acee1000-acf11000 rw-p 00000000 00:00 0 acf11000-acf12000 ---p 00000000 00:00 0 acf12000-acf42000 rw-p 00000000 00:00 0 acf42000-acf43000 ---p 00000000 00:00 0 acf43000-acf73000 rw-p 00000000 00:00 0 acf73000-acf74000 ---p 00000000 00:00 0 acf74000-acfa4000 rw-p 00000000 00:00 0 acfa4000-acfa5000 ---p 00000000 00:00 0 acfa5000-acfd5000 rw-p 00000000 00:00 0 acfd5000-acfd6000 ---p 00000000 00:00 0 acfd6000-ad006000 rw-p 00000000 00:00 0 ad006000-ad007000 ---p 00000000 00:00 0 ad007000-ad037000 rw-p 00000000 00:00 0 ad037000-ad038000 ---p 00000000 00:00 0 ad038000-ad068000 rw-p 00000000 00:00 0 ad068000-ad069000 ---p 00000000 00:00 0 ad069000-ad099000 rw-p 00000000 00:00 0 ad099000-ad09a000 ---p 00000000 00:00 0 ad09a000-ad0ca000 rw-p 00000000 00:00 0 ad0ca000-ad0cb000 ---p 00000000 00:00 0 ad0cb000-ad0fb000 rw-p 00000000 00:00 0 ad0fb000-ad0fc000 ---p 00000000 00:00 0 ad0fc000-ad12c000 rw-p 00000000 00:00 0 ad12c000-ad12d000 ---p 00000000 00:00 0 ad12d000-ad15d000 rw-p 00000000 00:00 0 ad15d000-ad15e000 ---p 00000000 00:00 0 ad15e000-ad18e000 rw-p 00000000 00:00 0 ad18e000-ad18f000 ---p 00000000 00:00 0 ad18f000-ad1bf000 rw-p 00000000 00:00 0 ad1bf000-ad1c0000 ---p 00000000 00:00 0 ad1c0000-ad1f0000 rw-p 00000000 00:00 0 ad1f0000-ad1f1000 ---p 00000000 00:00 0 ad1f1000-ad221000 rw-p 00000000 00:00 0 ad221000-ad222000 ---p 00000000 00:00 0 ad222000-ad252000 rw-p 00000000 00:00 0 ad252000-ad253000 ---p 00000000 00:00 0 ad253000-ad283000 rw-p 00000000 00:00 0 ad283000-ad284000 ---p 00000000 00:00 0 ad284000-ad2b4000 rw-p 00000000 00:00 0 ad2b4000-ad2b5000 ---p 00000000 00:00 0 ad2b5000-ad2e5000 rw-p 00000000 00:00 0 ad2e5000-ad2e6000 ---p 00000000 00:00 0 ad2e6000-ad316000 rw-p 00000000 00:00 0 ad316000-ad317000 ---p 00000000 00:00 0 ad317000-ad347000 rw-p 00000000 00:00 0 ad347000-ad348000 ---p 00000000 00:00 0 ad348000-ad378000 rw-p 00000000 00:00 0 ad378000-ad379000 ---p 00000000 00:00 0 ad379000-ad3a9000 rw-p 00000000 00:00 0 ad3a9000-ad3aa000 ---p 00000000 00:00 0 ad3aa000-ad3da000 rw-p 00000000 00:00 0 ad3da000-ad3db000 ---p 00000000 00:00 0 ad3db000-ad40b000 rw-p 00000000 00:00 0 ad40b000-ad40c000 ---p 00000000 00:00 0 ad40c000-ad43c000 rw-p 00000000 00:00 0 ad43c000-ad43d000 ---p 00000000 00:00 0 ad43d000-ad46d000 rw-p 00000000 00:00 0 ad46d000-ad46e000 ---p 00000000 00:00 0 ad46e000-ad49e000 rw-p 00000000 00:00 0 ad49e000-ad49f000 ---p 00000000 00:00 0 ad49f000-ad4cf000 rw-p 00000000 00:00 0 ad4cf000-ad4d0000 ---p 00000000 00:00 0 ad4d0000-ad500000 rw-p 00000000 00:00 0 ad500000-ad600000 rw-p 00000000 00:00 0 ad621000-ad622000 ---p 00000000 00:00 0 ad622000-ad652000 rw-p 00000000 00:00 0 ad652000-ad653000 ---p 00000000 00:00 0 ad653000-ad683000 rw-p 00000000 00:00 0 ad683000-ad684000 ---p 00000000 00:00 0 ad684000-ad6b4000 rw-p 00000000 00:00 0 ad6b4000-ad6b5000 ---p 00000000 00:00 0 ad6b5000-ad6e5000 rw-p 00000000 00:00 0 ad6e5000-ad6e6000 ---p 00000000 00:00 0 ad6e6000-ad716000 rw-p 00000000 00:00 0 ad716000-ad717000 ---p 00000000 00:00 0 ad717000-ad747000 rw-p 00000000 00:00 0 ad747000-ad748000 ---p 00000000 00:00 0 ad748000-ad778000 rw-p 00000000 00:00 0 ad778000-ad779000 ---p 00000000 00:00 0 ad779000-ad7a9000 rw-p 00000000 00:00 0 ad7a9000-ad7aa000 ---p 00000000 00:00 0 ad7aa000-ad7da000 rw-p 00000000 00:00 0 ad7da000-ad7db000 ---p 00000000 00:00 0 ad7db000-ad80b000 rw-p 00000000 00:00 0 ad80b000-ad80c000 ---p 00000000 00:00 0 ad80c000-ad83c000 rw-p 00000000 00:00 0 ad83c000-ad83d000 ---p 00000000 00:00 0 ad83d000-ad86d000 rw-p 00000000 00:00 0 ad86d000-ad86e000 ---p 00000000 00:00 0 ad86e000-ad89e000 rw-p 00000000 00:00 0 ad89e000-ad89f000 ---p 00000000 00:00 0 ad89f000-ad8cf000 rw-p 00000000 00:00 0 ad8cf000-ad8d0000 ---p 00000000 00:00 0 ad8d0000-ad900000 rw-p 00000000 00:00 0 ad900000-ad9e5000 rw-p 00000000 00:00 0 ad9e5000-ada00000 ---p 00000000 00:00 0 ada21000-ada22000 ---p 00000000 00:00 0 ada22000-ada52000 rw-p 00000000 00:00 0 ada52000-ada53000 ---p 00000000 00:00 0 ada53000-ada83000 rw-p 00000000 00:00 0 ada83000-ada84000 ---p 00000000 00:00 0 ada84000-adab4000 rw-p 00000000 00:00 0 adab4000-adab5000 ---p 00000000 00:00 0 adab5000-adae5000 rw-p 00000000 00:00 0 adae5000-adae6000 ---p 00000000 00:00 0 adae6000-adb16000 rw-p 00000000 00:00 0 adb16000-adb17000 ---p 00000000 00:00 0 adb17000-adb47000 rw-p 00000000 00:00 0 adb47000-adb48000 ---p 00000000 00:00 0 adb48000-adb78000 rw-p 00000000 00:00 0 adb78000-adb79000 ---p 00000000 00:00 0 adb79000-adba9000 rw-p 00000000 00:00 0 adba9000-adbaa000 ---p 00000000 00:00 0 adbaa000-adbda000 rw-p 00000000 00:00 0 adbda000-adbdb000 ---p 00000000 00:00 0 adbdb000-adc0b000 rw-p 00000000 00:00 0 adc0b000-adc0c000 ---p 00000000 00:00 0 adc0c000-adc3c000 rw-p 00000000 00:00 0 adc3c000-adc3d000 ---p 00000000 00:00 0 adc3d000-adc6d000 rw-p 00000000 00:00 0 adc6d000-adc6e000 ---p 00000000 00:00 0 adc6e000-adc9e000 rw-p 00000000 00:00 0 adc9e000-adc9f000 ---p 00000000 00:00 0 adc9f000-adccf000 rw-p 00000000 00:00 0 adccf000-adcd0000 ---p 00000000 00:00 0 adcd0000-add00000 rw-p 00000000 00:00 0 add00000-ade00000 rw-p 00000000 00:00 0 ade21000-ade22000 ---p 00000000 00:00 0 ade22000-ade52000 rw-p 00000000 00:00 0 ade52000-ade53000 ---p 00000000 00:00 0 ade53000-ade83000 rw-p 00000000 00:00 0 ade83000-ade84000 ---p 00000000 00:00 0 ade84000-adeb4000 rw-p 00000000 00:00 0 adeb4000-adeb5000 ---p 00000000 00:00 0 adeb5000-adee5000 rw-p 00000000 00:00 0 adee5000-adee6000 ---p 00000000 00:00 0 adee6000-adf16000 rw-p 00000000 00:00 0 adf16000-adf17000 ---p 00000000 00:00 0 adf17000-adf47000 rw-p 00000000 00:00 0 adf47000-adf48000 ---p 00000000 00:00 0 adf48000-adf78000 rw-p 00000000 00:00 0 adf78000-adf79000 ---p 00000000 00:00 0 adf79000-adfa9000 rw-p 00000000 00:00 0 adfa9000-adfaa000 ---p 00000000 00:00 0 adfaa000-adfda000 rw-p 00000000 00:00 0 adfda000-adfdb000 ---p 00000000 00:00 0 adfdb000-ae00b000 rw-p 00000000 00:00 0 ae00b000-ae00c000 ---p 00000000 00:00 0 ae00c000-ae03c000 rw-p 00000000 00:00 0 ae03c000-ae03d000 ---p 00000000 00:00 0 ae03d000-ae06d000 rw-p 00000000 00:00 0 ae06d000-ae06e000 ---p 00000000 00:00 0 ae06e000-ae09e000 rw-p 00000000 00:00 0 ae09e000-ae09f000 ---p 00000000 00:00 0 ae09f000-ae0cf000 rw-p 00000000 00:00 0 ae0cf000-ae0d0000 ---p 00000000 00:00 0 ae0d0000-ae100000 rw-p 00000000 00:00 0 ae100000-ae200000 rw-p 00000000 00:00 0 ae211000-ae212000 ---p 00000000 00:00 0 ae212000-ae242000 rw-p 00000000 00:00 0 ae242000-ae243000 ---p 00000000 00:00 0 ae243000-ae273000 rw-p 00000000 00:00 0 ae273000-ae274000 ---p 00000000 00:00 0 ae274000-ae2a4000 rw-p 00000000 00:00 0 ae2a4000-ae2a5000 ---p 00000000 00:00 0 ae2a5000-ae2d5000 rw-p 00000000 00:00 0 ae2d5000-ae2d6000 ---p 00000000 00:00 0 ae2d6000-ae306000 rw-p 00000000 00:00 0 ae306000-ae307000 ---p 00000000 00:00 0 ae307000-ae337000 rw-p 00000000 00:00 0 ae337000-ae338000 ---p 00000000 00:00 0 ae338000-ae368000 rw-p 00000000 00:00 0 ae375000-ae376000 ---p 00000000 00:00 0 ae376000-ae3a6000 rw-p 00000000 00:00 0 ae3a6000-ae3a7000 ---p 00000000 00:00 0 ae3a7000-ae3d7000 rw-p 00000000 00:00 0 ae3d7000-ae3d8000 ---p 00000000 00:00 0 ae3d8000-ae408000 rw-p 00000000 00:00 0 ae408000-ae409000 ---p 00000000 00:00 0 ae409000-ae439000 rw-p 00000000 00:00 0 ae439000-ae43a000 ---p 00000000 00:00 0 ae43a000-aec3a000 rw-p 00000000 00:00 0 aec3a000-aec3b000 ---p 00000000 00:00 0 aec3b000-af43b000 rw-p 00000000 00:00 0 af43b000-af43c000 ---p 00000000 00:00 0 af43c000-afc3c000 rw-p 00000000 00:00 0 afc3c000-afc3d000 ---p 00000000 00:00 0 afc3d000-b0643000 rw-p 00000000 00:00 0 b0668000-b0669000 ---p 00000000 00:00 0 b0669000-b0699000 rw-p 00000000 00:00 0 b0699000-b069a000 ---p 00000000 00:00 0 b069a000-b06ca000 rw-p 00000000 00:00 0 b06ca000-b06cb000 ---p 00000000 00:00 0 b06cb000-b06fb000 rw-p 00000000 00:00 0 b070b000-b070c000 ---p 00000000 00:00 0 b070c000-b073c000 rw-p 00000000 00:00 0 b073c000-b073d000 ---p 00000000 00:00 0 b073d000-b076d000 rw-p 00000000 00:00 0 b076d000-b076e000 ---p 00000000 00:00 0 b076e000-b079e000 rw-p 00000000 00:00 0 b079e000-b079f000 ---p 00000000 00:00 0 b079f000-b07cf000 rw-p 00000000 00:00 0 b07cf000-b07d0000 ---p 00000000 00:00 0 b07d0000-b0800000 rw-p 00000000 00:00 0 b0800000-b08ff000 rw-p 00000000 00:00 0 b08ff000-b0900000 ---p 00000000 00:00 0 b090a000-b090b000 ---p 00000000 00:00 0 b090b000-b093b000 rw-p 00000000 00:00 0 b093b000-b093c000 ---p 00000000 00:00 0 b093c000-b096c000 rw-p 00000000 00:00 0 b096c000-b096d000 ---p 00000000 00:00 0 b096d000-b099d000 rw-p 00000000 00:00 0 b099d000-b099e000 ---p 00000000 00:00 0 b099e000-b09ce000 rw-p 00000000 00:00 0 b09ce000-b09cf000 ---p 00000000 00:00 0 b09cf000-b09ff000 rw-p 00000000 00:00 0 b09ff000-b0a00000 ---p 00000000 00:00 0 b0a00000-b0a30000 rw-p 00000000 00:00 0 b0a30000-b0a31000 ---p 00000000 00:00 0 b0a31000-b0a61000 rw-p 00000000 00:00 0 b0a61000-b0a62000 ---p 00000000 00:00 0 b0a62000-b1262000 rw-p 00000000 00:00 0 b1262000-b1263000 ---p 00000000 00:00 0 b1263000-b1a63000 rw-p 00000000 00:00 0 b1a63000-b1a64000 ---p 00000000 00:00 0 b1a64000-b6e13000 rw-p 00000000 00:00 0 b6e13000-b6e1c000 r-xp 00000000 09:01 20055584 /lib/libnss_nis-2.9.so b6e1c000-b6e1d000 r--p 00008000 09:01 20055584 /lib/libnss_nis-2.9.so b6e1d000-b6e1e000 rw-p 00009000 09:01 20055584 /lib/libnss_nis-2.9.so b6e1e000-b6e25000 r-xp 00000000 09:01 20055580 /lib/libnss_compat-2.9.so b6e25000-b6e26000 r--p 00006000 09:01 20055580 /lib/libnss_compat-2.9.so b6e26000-b6e27000 rw-p 00007000 09:01 20055580 /lib/libnss_compat-2.9.so b6e27000-b6e30000 r-xp 00000000 09:01 20055582 /lib/libnss_files-2.9.so b6e30000-b6e31000 r--p 00008000 09:01 20055582 /lib/libnss_files-2.9.so b6e31000-b6e32000 rw-p 00009000 09:01 20055582 /lib/libnss_files-2.9.so b6e32000-b6e3e000 r-xp 00000000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6e3e000-b6e3f000 r--p 0000b000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6e3f000-b6e40000 rw-p 0000c000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6e41000-b6e42000 ---p 00000000 00:00 0 b6e42000-b7643000 rw-p 00000000 00:00 0 b7643000-b7780000 r-xp 00000000 09:01 20055572 /lib/libc-2.9.so b7780000-b7782000 r--p 0013d000 09:01 20055572 /lib/libc-2.9.so b7782000-b7783000 rw-p 0013f000 09:01 20055572 /lib/libc-2.9.so b7783000-b7787000 rw-p 00000000 00:00 0 b7787000-b77ab000 r-xp 00000000 09:01 20055578 /lib/libm-2.9.so b77ab000-b77ac000 r--p 00023000 09:01 20055578 /lib/libm-2.9.so b77ac000-b77ad000 rw-p 00024000 09:01 20055578 /lib/libm-2.9.so b77ad000-b77c0000 r-xp 00000000 09:01 20055579 /lib/libnsl-2.9.so b77c0000-b77c1000 r--p 00012000 09:01 20055579 /lib/libnsl-2.9.so b77c1000-b77c2000 rw-p 00013000 09:01 20055579 /lib/libnsl-2.9.so b77c2000-b77c4000 rw-p 00000000 00:00 0 b77c4000-b77cd000 r-xp 00000000 09:01 20055574 /lib/libcrypt-2.9.so b77cd000-b77ce000 r--p 00008000 09:01 20055574 /lib/libcrypt-2.9.so b77ce000-b77cf000 rw-p 00009000 09:01 20055574 /lib/libcrypt-2.9.so b77cf000-b77f6000 rw-p 00000000 00:00 0 b77f6000-b77f8000 r-xp 00000000 09:01 20055575 /lib/libdl-2.9.so b77f8000-b77f9000 r--p 00001000 09:01 20055575 /lib/libdl-2.9.so b77f9000-b77fa000 rw-p 00002000 09:01 20055575 /lib/libdl-2.9.so b77fa000-b780b000 r-xp 00000000 09:01 20054396 /lib/libz.so.1.2.3 b780b000-b780c000 rw-p 00010000 09:01 20054396 /lib/libz.so.1.2.3 b780c000-b7820000 r-xp 00000000 09:01 20055591 /lib/libpthread-2.9.so b7820000-b7821000 r--p 00013000 09:01 20055591 /lib/libpthread-2.9.so b7821000-b7822000 rw-p 00014000 09:01 20055591 /lib/libpthread-2.9.so b7822000-b7825000 rw-p 00000000 00:00 0 b782a000-b782b000 r-xp 00000000 00:00 0 [vdso] b782b000-b7847000 r-xp 00000000 09:01 20055569 /lib/ld-2.9.so b7847000-b7848000 r--p 0001c000 09:01 20055569 /lib/ld-2.9.so b7848000-b7849000 rw-p 0001d000 09:01 20055569 /lib/ld-2.9.so bfc60000-bfc75000 rw-p 00000000 00:00 0 [stack]

by **psy__** » Tue Dec 22, 2009 3:53 am

Hello guys,

I have the same problem on multiple machines running kernel: 2.6.31.8-grsec and 2.6.31.9-grsec and MySQL: mysql Ver 14.12 Distrib 5.0.85, for unknown-linux-gnu (x86_64) using readline 5.1.

Today I got one crash with this error in the log:

kernel: grsec: From xx.xx.xx.xx: Segmentation fault occurred at 0000000000000064 in /usr/sbin/mysqld[mysqld:18750] uid/euid:100/100 gid/egid:101/101, parent /usr/bin/mysqld_safe[mysqld_safe:4265] uid/euid:0/0 gid/egid:0/0

If this could help you track the problem it would be great.

by **PaX Team** » Wed Dec 30, 2009 12:14 pm

psy__ wrote:Today I got one crash with this error in the log:

kernel: grsec: From xx.xx.xx.xx: Segmentation fault occurred at 0000000000000064 in /usr/sbin/mysqld[mysqld:18750] uid/euid:100/100 gid/egid:101/101, parent /usr/bin/mysqld_safe[mysqld_safe:4265] uid/euid:0/0 gid/egid:0/0

If this could help you track the problem it would be great.

the same information i asked from dabetz would be helpful.

by **dabetz** » Thu Jan 21, 2010 2:26 pm

Hi Pax Team,

same with 2.6.32.3-grsec and 5.1.41-MariaDB-rc on gcc version 4.3.2 (Gentoo 4.3.2-r3 p1.6, pie-10.1.5)

Code: Select all: Version: '5.1.41-MariaDB-rc-log' socket: '/var/lib/mysql/mysql5.sock' port: 3306 (MariaDB - http://mariadb.com/) *** glibc detected *** /usr/sbin/mysqld-5: corrupted double-linked list: 0x0a1aaff8 *** ======= Backtrace: ========= /lib/libc.so.6[0xb772c714] /lib/libc.so.6[0xb772ddd3] /lib/libc.so.6(cfree+0x9c)[0xb772dfcc] /usr/sbin/mysqld-5[0x8503ed1] /usr/sbin/mysqld-5[0x85075af] /usr/sbin/mysqld-5[0x84af7ef] /usr/sbin/mysqld-5[0x84af94d] /usr/sbin/mysqld-5(_Z13rr_sequentialP11READ_RECORD+0x25)[0x82f7cd5] /usr/sbin/mysqld-5(_Z10sub_selectP4JOINP13st_join_tableb+0x76)[0x8288f16] /usr/sbin/mysqld-5[0x82892e3] /usr/sbin/mysqld-5(_ZN4JOIN4execEv+0x1311)[0x8297411] /usr/sbin/mysqld-5(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select_resultP18st_select_lex_unitP13st_select_lex+0x1 3d)[0x82988dd] /usr/sbin/mysqld-5(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x15d)[0x829924d] /usr/sbin/mysqld-5[0x8226d47] /usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x458)[0x8229658] /usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x1ff)[0x82313bf] /usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0xb6e)[0x8231f4e] /usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe1)[0x8232ac1] /usr/sbin/mysqld-5(handle_one_connection+0xca)[0x8222a1a] /lib/libpthread.so.0[0xb788d16f] /lib/libc.so.6(clone+0x5e)[0xb778fc0e] ======= Memory map: ======== 08048000-087d0000 r-xp 00000000 09:01 81395714 /usr/libexec/mysqld-500 087d0000-08834000 rw-p 00787000 09:01 81395714 /usr/libexec/mysqld-500 08834000-0aa30000 rw-p 00000000 00:00 0 [heap] 9b100000-9b200000 rw-p 00000000 00:00 0 9b300000-9b3a4000 rw-p 00000000 00:00 0 9b3a4000-9b400000 ---p 00000000 00:00 0 9b4fe000-9b700000 rw-p 00000000 00:00 0 9b700000-9b7da000 rw-p 00000000 00:00 0 9b7da000-9b800000 ---p 00000000 00:00 0 9b800000-9b900000 rw-p 00000000 00:00 0 9b900000-9b997000 rw-p 00000000 00:00 0 9b997000-9ba00000 ---p 00000000 00:00 0 9bb00000-9bbfe000 rw-p 00000000 00:00 0 9bbfe000-9bc00000 ---p 00000000 00:00 0 9bd00000-9bdff000 rw-p 00000000 00:00 0 9bdff000-9be00000 ---p 00000000 00:00 0 9beff000-9c000000 rw-p 00000000 00:00 0 9c000000-9c0ff000 rw-p 00000000 00:00 0 9c0ff000-9c100000 ---p 00000000 00:00 0 9c128000-9c129000 ---p 00000000 00:00 0 9c129000-9c170000 rw-p 00000000 00:00 0 9c200000-9c2fa000 rw-p 00000000 00:00 0 9c2fa000-9c300000 ---p 00000000 00:00 0 9c300000-9c400000 rw-p 00000000 00:00 0 9c400000-9c4eb000 rw-p 00000000 00:00 0 9c4eb000-9c500000 ---p 00000000 00:00 0 9c528000-9c529000 ---p 00000000 00:00 0 9c529000-9c570000 rw-p 00000000 00:00 0 9c5b8000-9c5b9000 ---p 00000000 00:00 0 9c5b9000-9c600000 rw-p 00000000 00:00 0 9c600000-9c6e3000 rw-p 00000000 00:00 0 9c6e3000-9c700000 ---p 00000000 00:00 0 9c700000-9c800000 rw-p 00000000 00:00 0 9c800000-9c8f7000 rw-p 00000000 00:00 0 9c8f7000-9c900000 ---p 00000000 00:00 0 9ca00000-9cb00000 rw-p 00000000 00:00 0 9cb80000-9cb81000 ---p 00000000 00:00 0 9cb81000-9cbc8000 rw-p 00000000 00:00 0 9ccbf000-9cdc0000 rw-p 00000000 00:00 0 9cdc0000-9cdc1000 ---p 00000000 00:00 0 9cdc1000-9ce08000 rw-p 00000000 00:00 0 9ce08000-9ce09000 ---p 00000000 00:00 0 9ce09000-9ce50000 rw-p 00000000 00:00 0 9cf28000-9cf29000 ---p 00000000 00:00 0 9cf29000-9cf70000 rw-p 00000000 00:00 0 9cfb8000-9cfb9000 ---p 00000000 00:00 0 9cfb9000-9d000000 rw-p 00000000 00:00 0 9d000000-9d0ff000 rw-p 00000000 00:00 0 9d0ff000-9d100000 ---p 00000000 00:00 0 9d100000-9d1f6000 rw-p 00000000 00:00 0 9d1f6000-9d200000 ---p 00000000 00:00 0 9d208000-9d209000 ---p 00000000 00:00 0 9d209000-9d250000 rw-p 00000000 00:00 0 9d250000-9d251000 ---p 00000000 00:00 0 9d251000-9d298000 rw-p 00000000 00:00 0 9d298000-9d299000 ---p 00000000 00:00 0 9d299000-9d2e0000 rw-p 00000000 00:00 0 9d400000-9d500000 rw-p 00000000 00:00 0 9d528000-9d529000 ---p 00000000 00:00 0 9d529000-9d570000 rw-p 00000000 00:00 0 9d570000-9d571000 ---p 00000000 00:00 0 9d571000-9d5b8000 rw-p 00000000 00:00 0 9d5b8000-9d5b9000 ---p 00000000 00:00 0 9d5b9000-9d600000 rw-p 00000000 00:00 0 9d600000-9d700000 rw-p 00000000 00:00 0 9d727000-9d728000 ---p 00000000 00:00 0 9d728000-9d76f000 rw-p 00000000 00:00 0 9d76f000-9d770000 ---p 00000000 00:00 0 9d770000-9d7b7000 rw-p 00000000 00:00 0 9d7b7000-9d7b8000 ---p 00000000 00:00 0 9d7b8000-9d7ff000 rw-p 00000000 00:00 0 9d828000-9d829000 ---p 00000000 00:00 0 9d829000-9d870000 rw-p 00000000 00:00 0 9d870000-9d871000 ---p 00000000 00:00 0 9d871000-9d8b8000 rw-p 00000000 00:00 0 9d8b8000-9d8b9000 ---p 00000000 00:00 0 9d8b9000-9d900000 rw-p 00000000 00:00 0 9d900000-9d9f3000 rw-p 00000000 00:00 0 9d9f3000-9da00000 ---p 00000000 00:00 0 9da38000-9da39000 ---p 00000000 00:00 0 9da39000-9da80000 rw-p 00000000 00:00 0 9da80000-9da81000 ---p 00000000 00:00 0 9da81000-9dac8000 rw-p 00000000 00:00 0 9dae7000-9dae8000 ---p 00000000 00:00 0 9dae8000-9db2f000 rw-p 00000000 00:00 0 9dc30000-9dc31000 ---p 00000000 00:00 0 9dc31000-9dc78000 rw-p 00000000 00:00 0 9dc78000-9dc79000 ---p 00000000 00:00 0 9dc79000-9dcc0000 rw-p 00000000 00:00 0 9dcc0000-9dcc1000 ---p 00000000 00:00 0 9dcc1000-9dd08000 rw-p 00000000 00:00 0 9dd08000-9dd09000 ---p 00000000 00:00 0 9dd09000-9dd50000 rw-p 00000000 00:00 0 9dd50000-9dd51000 ---p 00000000 00:00 0 9dd51000-9dd98000 rw-p 00000000 00:00 0 9dd98000-9dd99000 ---p 00000000 00:00 0 9dd99000-9dde0000 rw-p 00000000 00:00 0 9dde0000-9dde1000 ---p 00000000 00:00 0 9dde1000-9de28000 rw-p 00000000 00:00 0 9de28000-9de29000 ---p 00000000 00:00 0 9de29000-9de70000 rw-p 00000000 00:00 0 9de70000-9de71000 ---p 00000000 00:00 0 9de71000-9deb8000 rw-p 00000000 00:00 0 9deb8000-9deb9000 ---p 00000000 00:00 0 9deb9000-9df00000 rw-p 00000000 00:00 0 9df00000-9dfdc000 rw-p 00000000 00:00 0 9dfdc000-9e000000 ---p 00000000 00:00 0 9e000000-9e100000 rw-p 00000000 00:00 0 9e100000-9e1c1000 rw-p 00000000 00:00 0 9e1c1000-9e200000 ---p 00000000 00:00 0 9e200000-9e2c5000 rw-p 00000000 00:00 0 9e2c5000-9e300000 ---p 00000000 00:00 0 9e300000-9e3df000 rw-p 00000000 00:00 0 9e3df000-9e400000 ---p 00000000 00:00 0 9e400000-9e4fd000 rw-p 00000000 00:00 0 9e4fd000-9e500000 ---p 00000000 00:00 0 9e500000-9e5ce000 rw-p 00000000 00:00 0 9e5ce000-9e600000 ---p 00000000 00:00 0 9e610000-9e611000 ---p 00000000 00:00 0 9e611000-9e658000 rw-p 00000000 00:00 0 9e658000-9e659000 ---p 00000000 00:00 0 9e659000-9e6a0000 rw-p 00000000 00:00 0 9e6a0000-9e6a1000 ---p 00000000 00:00 0 9e6a1000-9e6e8000 rw-p 00000000 00:00 0 9e6e8000-9e6e9000 ---p 00000000 00:00 0 9e6e9000-9e730000 rw-p 00000000 00:00 0 9e730000-9e731000 ---p 00000000 00:00 0 9e731000-9e778000 rw-p 00000000 00:00 0 9e797000-9e798000 ---p 00000000 00:00 0 9e798000-9e7df000 rw-p 00000000 00:00 0 9e7df000-9e7e0000 ---p 00000000 00:00 0 9e7e0000-9e827000 rw-p 00000000 00:00 0 9e827000-9e828000 ---p 00000000 00:00 0 9e828000-9e86f000 rw-p 00000000 00:00 0 9e898000-9e899000 ---p 00000000 00:00 0 9e899000-9e8e0000 rw-p 00000000 00:00 0 9e8e0000-9e8e1000 ---p 00000000 00:00 0 9e8e1000-9e928000 rw-p 00000000 00:00 0 9e928000-9e929000 ---p 00000000 00:00 0 9e929000-9e970000 rw-p 00000000 00:00 0 9e970000-9e971000 ---p 00000000 00:00 0 9e971000-9e9b8000 rw-p 00000000 00:00 0 9e9b8000-9e9b9000 ---p 00000000 00:00 0 9e9b9000-9ea00000 rw-p 00000000 00:00 0 9ea00000-9ea01000 ---p 00000000 00:00 0 9ea01000-9ea48000 rw-p 00000000 00:00 0 9ea48000-9ea49000 ---p 00000000 00:00 0 9ea49000-9ea90000 rw-p 00000000 00:00 0 9ea90000-9ea91000 ---p 00000000 00:00 0 9ea91000-9ead8000 rw-p 00000000 00:00 0 9ead8000-9ead9000 ---p 00000000 00:00 0 9ead9000-9eb20000 rw-p 00000000 00:00 0 9eb20000-9eb21000 ---p 00000000 00:00 0 9eb21000-9eb68000 rw-p 00000000 00:00 0 9eb68000-9eb69000 ---p 00000000 00:00 0 9eb69000-9ebb0000 rw-p 00000000 00:00 0 9ebb0000-9ebb1000 ---p 00000000 00:00 0 9ebb1000-9ebf8000 rw-p 00000000 00:00 0 9ebf8000-9ebf9000 ---p 00000000 00:00 0 9ebf9000-9ec40000 rw-p 00000000 00:00 0 9ec40000-9ec41000 ---p 00000000 00:00 0 9ec41000-9ec88000 rw-p 00000000 00:00 0 9ec88000-9ec89000 ---p 00000000 00:00 0 9ec89000-9ecd0000 rw-p 00000000 00:00 0 9ecd0000-9ecd1000 ---p 00000000 00:00 0 9ecd1000-9ed18000 rw-p 00000000 00:00 0 9ed18000-9ed19000 ---p 00000000 00:00 0 9ed19000-9ed60000 rw-p 00000000 00:00 0 9ed60000-9ed61000 ---p 00000000 00:00 0 9ed61000-9eda8000 rw-p 00000000 00:00 0 9eda8000-9eda9000 ---p 00000000 00:00 0 9eda9000-9edf0000 rw-p 00000000 00:00 0 9edf0000-9edf1000 ---p 00000000 00:00 0 9edf1000-9ee38000 rw-p 00000000 00:00 0 9ee38000-9ee39000 ---p 00000000 00:00 0 9ee39000-9ee80000 rw-p 00000000 00:00 0 9ee80000-9ee81000 ---p 00000000 00:00 0 9ee81000-9eec8000 rw-p 00000000 00:00 0 9eec8000-9eec9000 ---p 00000000 00:00 0 9eec9000-9ef10000 rw-p 00000000 00:00 0 9ef10000-9ef11000 ---p 00000000 00:00 0 9ef11000-9ef58000 rw-p 00000000 00:00 0 9ef58000-9ef59000 ---p 00000000 00:00 0 9ef59000-9efa0000 rw-p 00000000 00:00 0 9efa0000-9efa1000 ---p 00000000 00:00 0 9efa1000-9efe8000 rw-p 00000000 00:00 0 9efe8000-9efe9000 ---p 00000000 00:00 0 9efe9000-9f030000 rw-p 00000000 00:00 0 9f030000-9f031000 ---p 00000000 00:00 0 9f031000-9f078000 rw-p 00000000 00:00 0 9f078000-9f079000 ---p 00000000 00:00 0 9f079000-9f0c0000 rw-p 00000000 00:00 0 9f0c0000-9f0c1000 ---p 00000000 00:00 0 9f0c1000-9f108000 rw-p 00000000 00:00 0 9f108000-9f109000 ---p 00000000 00:00 0 9f109000-9f150000 rw-p 00000000 00:00 0 9f150000-9f151000 ---p 00000000 00:00 0 9f151000-9f198000 rw-p 00000000 00:00 0 9f198000-9f199000 ---p 00000000 00:00 0 9f199000-9f1e0000 rw-p 00000000 00:00 0 9f1e0000-9f1e1000 ---p 00000000 00:00 0 9f1e1000-9f228000 rw-p 00000000 00:00 0 9f228000-9f229000 ---p 00000000 00:00 0 9f229000-9f270000 rw-p 00000000 00:00 0 9f270000-9f271000 ---p 00000000 00:00 0 9f271000-9f2b8000 rw-p 00000000 00:00 0 9f2b8000-9f2b9000 ---p 00000000 00:00 0 9f2b9000-9f300000 rw-p 00000000 00:00 0 9f300000-9f400000 rw-p 00000000 00:00 0 9f400000-9f401000 ---p 00000000 00:00 0 9f401000-9f448000 rw-p 00000000 00:00 0 9f448000-9f449000 ---p 00000000 00:00 0 9f449000-9f490000 rw-p 00000000 00:00 0 9f490000-9f491000 ---p 00000000 00:00 0 9f491000-9f4d8000 rw-p 00000000 00:00 0 9f4d8000-9f4d9000 ---p 00000000 00:00 0 9f4d9000-9f520000 rw-p 00000000 00:00 0 9f520000-9f521000 ---p 00000000 00:00 0 9f521000-9f568000 rw-p 00000000 00:00 0 9f568000-9f569000 ---p 00000000 00:00 0 9f569000-9f5b0000 rw-p 00000000 00:00 0 9f5b0000-9f5b1000 ---p 00000000 00:00 0 9f5b1000-9f5f8000 rw-p 00000000 00:00 0 9f5f8000-9f5f9000 ---p 00000000 00:00 0 9f5f9000-9f640000 rw-p 00000000 00:00 0 9f640000-9f641000 ---p 00000000 00:00 0 9f641000-9f688000 rw-p 00000000 00:00 0 9f688000-9f689000 ---p 00000000 00:00 0 9f689000-9f6d0000 rw-p 00000000 00:00 0 9f6d0000-9f6d1000 ---p 00000000 00:00 0 9f6d1000-9f718000 rw-p 00000000 00:00 0 9f718000-9f719000 ---p 00000000 00:00 0 9f719000-9f760000 rw-p 00000000 00:00 0 9f760000-9f761000 ---p 00000000 00:00 0 9f761000-9f7a8000 rw-p 00000000 00:00 0 9f7a8000-9f7a9000 ---p 00000000 00:00 0 9f7a9000-9f7f0000 rw-p 00000000 00:00 0 9f7f0000-9f7f1000 ---p 00000000 00:00 0 9f7f1000-9f838000 rw-p 00000000 00:00 0 9f838000-9f839000 ---p 00000000 00:00 0 9f839000-9f880000 rw-p 00000000 00:00 0 9f880000-9f881000 ---p 00000000 00:00 0 9f881000-9f8c8000 rw-p 00000000 00:00 0 9f8c8000-9f8c9000 ---p 00000000 00:00 0 9f8c9000-9f910000 rw-p 00000000 00:00 0 9f910000-9f911000 ---p 00000000 00:00 0 9f911000-9f958000 rw-p 00000000 00:00 0 9f958000-9f959000 ---p 00000000 00:00 0 9f959000-9f9a0000 rw-p 00000000 00:00 0 9f9a0000-9f9a1000 ---p 00000000 00:00 0 9f9a1000-9f9e8000 rw-p 00000000 00:00 0 9f9e8000-9f9e9000 ---p 00000000 00:00 0 9f9e9000-9fa30000 rw-p 00000000 00:00 0 9fa30000-9fa31000 ---p 00000000 00:00 0 9fa31000-9fa78000 rw-p 00000000 00:00 0 9fa78000-9fa79000 ---p 00000000 00:00 0 9fa79000-9fac0000 rw-p 00000000 00:00 0 9fac0000-9fac1000 ---p 00000000 00:00 0 9fac1000-9fb08000 rw-p 00000000 00:00 0 9fb08000-9fb09000 ---p 00000000 00:00 0 9fb09000-9fb50000 rw-p 00000000 00:00 0 9fb50000-9fb51000 ---p 00000000 00:00 0 9fb51000-9fb98000 rw-p 00000000 00:00 0 9fb98000-9fb99000 ---p 00000000 00:00 0 9fb99000-9fbe0000 rw-p 00000000 00:00 0 9fbe0000-9fbe1000 ---p 00000000 00:00 0 9fbe1000-9fc28000 rw-p 00000000 00:00 0 9fc28000-9fc29000 ---p 00000000 00:00 0 9fc29000-9fc70000 rw-p 00000000 00:00 0 9fc70000-9fc71000 ---p 00000000 00:00 0 9fc71000-9fcb8000 rw-p 00000000 00:00 0 9fcb8000-9fcb9000 ---p 00000000 00:00 0 9fcb9000-9fd00000 rw-p 00000000 00:00 0 9fd00000-9fd01000 ---p 00000000 00:00 0 9fd01000-9fd48000 rw-p 00000000 00:00 0 9fd48000-9fd49000 ---p 00000000 00:00 0 9fd49000-9fd90000 rw-p 00000000 00:00 0 9fd90000-9fd91000 ---p 00000000 00:00 0 9fd91000-9fdd8000 rw-p 00000000 00:00 0 9fdd8000-9fdd9000 ---p 00000000 00:00 0 9fdd9000-9fe20000 rw-p 00000000 00:00 0 9fe20000-9fe21000 ---p 00000000 00:00 0 9fe21000-9fe68000 rw-p 00000000 00:00 0 9fe68000-9fe69000 ---p 00000000 00:00 0 9fe69000-9feb0000 rw-p 00000000 00:00 0 9feb0000-9feb1000 ---p 00000000 00:00 0 9feb1000-9fef8000 rw-p 00000000 00:00 0 9fef8000-9fef9000 ---p 00000000 00:00 0 9fef9000-9ff40000 rw-p 00000000 00:00 0 9ff40000-9ff41000 ---p 00000000 00:00 0 9ff41000-9ff88000 rw-p 00000000 00:00 0 9ff88000-9ff89000 ---p 00000000 00:00 0 9ff89000-9ffd0000 rw-p 00000000 00:00 0 9ffd0000-9ffd1000 ---p 00000000 00:00 0 9ffd1000-a0018000 rw-p 00000000 00:00 0 a0018000-a0019000 ---p 00000000 00:00 0 a0019000-a0060000 rw-p 00000000 00:00 0 a0060000-a0061000 ---p 00000000 00:00 0 a0061000-a00a8000 rw-p 00000000 00:00 0 a00a8000-a00a9000 ---p 00000000 00:00 0 a00a9000-a00f0000 rw-p 00000000 00:00 0 a00f0000-a00f1000 ---p 00000000 00:00 0 a00f1000-a0138000 rw-p 00000000 00:00 0 a0138000-a0139000 ---p 00000000 00:00 0 a0139000-a0180000 rw-p 00000000 00:00 0 a0180000-a0181000 ---p 00000000 00:00 0 a0181000-a01c8000 rw-p 00000000 00:00 0 a01c8000-a01c9000 ---p 00000000 00:00 0 a01c9000-a0210000 rw-p 00000000 00:00 0 a0210000-a0211000 ---p 00000000 00:00 0 a0211000-a0258000 rw-p 00000000 00:00 0 a0258000-a0259000 ---p 00000000 00:00 0 a0259000-a02a0000 rw-p 00000000 00:00 0 a02a0000-a02a1000 ---p 00000000 00:00 0 a02a1000-a02e8000 rw-p 00000000 00:00 0 a02e8000-a02e9000 ---p 00000000 00:00 0 a02e9000-a0330000 rw-p 00000000 00:00 0 a0330000-a0331000 ---p 00000000 00:00 0 a0331000-a0378000 rw-p 00000000 00:00 0 a0378000-a0379000 ---p 00000000 00:00 0 a0379000-a03c0000 rw-p 00000000 00:00 0 a03c0000-a03c1000 ---p 00000000 00:00 0 a03c1000-a0408000 rw-p 00000000 00:00 0 a0408000-a0409000 ---p 00000000 00:00 0 a0409000-a0450000 rw-p 00000000 00:00 0 a0450000-a0451000 ---p 00000000 00:00 0 a0451000-a0498000 rw-p 00000000 00:00 0 a0498000-a0499000 ---p 00000000 00:00 0 a0499000-a04e0000 rw-p 00000000 00:00 0 a04e0000-a04e1000 ---p 00000000 00:00 0 a04e1000-a0528000 rw-p 00000000 00:00 0 a0528000-a0529000 ---p 00000000 00:00 0 a0529000-a0570000 rw-p 00000000 00:00 0 a0570000-a0571000 ---p 00000000 00:00 0 a0571000-a05b8000 rw-p 00000000 00:00 0 a05b8000-a05b9000 ---p 00000000 00:00 0 a05b9000-a0600000 rw-p 00000000 00:00 0 a0600000-a06ff000 rw-p 00000000 00:00 0 a06ff000-a0700000 ---p 00000000 00:00 0 a0738000-a0739000 ---p 00000000 00:00 0 a0739000-a0780000 rw-p 00000000 00:00 0 a0780000-a0781000 ---p 00000000 00:00 0 a0781000-a07c8000 rw-p 00000000 00:00 0 a07c8000-a07c9000 ---p 00000000 00:00 0 a07c9000-a0810000 rw-p 00000000 00:00 0 a0810000-a0811000 ---p 00000000 00:00 0 a0811000-a0858000 rw-p 00000000 00:00 0 a0858000-a0859000 ---p 00000000 00:00 0 a0859000-a08a0000 rw-p 00000000 00:00 0 a08a0000-a08a1000 ---p 00000000 00:00 0 a08a1000-a08e8000 rw-p 00000000 00:00 0 a08e8000-a08e9000 ---p 00000000 00:00 0 a08e9000-a0930000 rw-p 00000000 00:00 0 a0930000-a0931000 ---p 00000000 00:00 0 a0931000-a0978000 rw-p 00000000 00:00 0 a0978000-a0979000 ---p 00000000 00:00 0 a0979000-a09c0000 rw-p 00000000 00:00 0 a09c0000-a09c1000 ---p 00000000 00:00 0 a09c1000-a0a08000 rw-p 00000000 00:00 0 a0a08000-a0a09000 ---p 00000000 00:00 0 a0a09000-a0a50000 rw-p 00000000 00:00 0 a0a50000-a0a51000 ---p 00000000 00:00 0 a0a51000-a0a98000 rw-p 00000000 00:00 0 a0a98000-a0a99000 ---p 00000000 00:00 0 a0a99000-a0ae0000 rw-p 00000000 00:00 0 a0ae0000-a0ae1000 ---p 00000000 00:00 0 a0ae1000-a0b28000 rw-p 00000000 00:00 0 a0b28000-a0b29000 ---p 00000000 00:00 0 a0b29000-a0b70000 rw-p 00000000 00:00 0 a0b70000-a0b71000 ---p 00000000 00:00 0 a0b71000-a0bb8000 rw-p 00000000 00:00 0 a0bb8000-a0bb9000 ---p 00000000 00:00 0 a0bb9000-a0c00000 rw-p 00000000 00:00 0 a0c00000-a0cff000 rw-p 00000000 00:00 0 a0cff000-a0d00000 ---p 00000000 00:00 0 a0d0d000-a0d0e000 ---p 00000000 00:00 0 a0d0e000-a0d55000 rw-p 00000000 00:00 0 a0d55000-a0d56000 ---p 00000000 00:00 0 a0d56000-a1556000 rw-p 00000000 00:00 0 a1556000-a1557000 ---p 00000000 00:00 0 a1557000-a1d57000 rw-p 00000000 00:00 0 a1d57000-a1d58000 ---p 00000000 00:00 0 a1d58000-a2558000 rw-p 00000000 00:00 0 a2558000-a2559000 ---p 00000000 00:00 0 a2559000-a2f5f000 rw-p 00000000 00:00 0 a2f8d000-a2f8e000 ---p 00000000 00:00 0 a2f8e000-a2fd5000 rw-p 00000000 00:00 0 a2fd5000-a2fd6000 ---p 00000000 00:00 0 a2fd6000-a301d000 rw-p 00000000 00:00 0 a301d000-a301e000 ---p 00000000 00:00 0 a301e000-a3065000 rw-p 00000000 00:00 0 a3065000-a3066000 ---p 00000000 00:00 0 a3066000-a30ad000 rw-p 00000000 00:00 0 a30ad000-a30ae000 ---p 00000000 00:00 0 a30ae000-a30f5000 rw-p 00000000 00:00 0 a30f5000-a30f6000 ---p 00000000 00:00 0 a30f6000-a313d000 rw-p 00000000 00:00 0 a313d000-a313e000 ---p 00000000 00:00 0 a313e000-a3185000 rw-p 00000000 00:00 0 a3185000-a3186000 ---p 00000000 00:00 0 a3186000-a31cd000 rw-p 00000000 00:00 0 a31cd000-a31ce000 ---p 00000000 00:00 0 a31ce000-a3215000 rw-p 00000000 00:00 0 a3215000-a3216000 ---p 00000000 00:00 0 a3216000-a325d000 rw-p 00000000 00:00 0 a325d000-a325e000 ---p 00000000 00:00 0 a325e000-a32a5000 rw-p 00000000 00:00 0 a32a5000-a32a6000 ---p 00000000 00:00 0 a32a6000-a32ed000 rw-p 00000000 00:00 0 a32ed000-a32ee000 ---p 00000000 00:00 0 a32ee000-a3335000 rw-p 00000000 00:00 0 a3335000-a3336000 ---p 00000000 00:00 0 a3336000-a337d000 rw-p 00000000 00:00 0 a337d000-a337e000 ---p 00000000 00:00 0 a337e000-a3b7e000 rw-p 00000000 00:00 0 a3b7e000-a3b7f000 ---p 00000000 00:00 0 a3b7f000-a437f000 rw-p 00000000 00:00 0 a437f000-a4380000 ---p 00000000 00:00 0 a4380000-a4b80000 rw-p 00000000 00:00 0 a4b80000-a4b81000 ---p 00000000 00:00 0 a4b81000-a5381000 rw-p 00000000 00:00 0 a5381000-a5382000 ---p 00000000 00:00 0 a5382000-a5b82000 rw-p 00000000 00:00 0 a5b82000-a5b83000 ---p 00000000 00:00 0 a5b83000-a6383000 rw-p 00000000 00:00 0 a6383000-a6384000 ---p 00000000 00:00 0 a6384000-a6b84000 rw-p 00000000 00:00 0 a6b84000-a6b85000 ---p 00000000 00:00 0 a6b85000-a7385000 rw-p 00000000 00:00 0 a7385000-a7386000 ---p 00000000 00:00 0 a7386000-a7b86000 rw-p 00000000 00:00 0 a7b86000-a7b87000 ---p 00000000 00:00 0 a7b87000-aac00000 rw-p 00000000 00:00 0 aac00000-aacff000 rw-p 00000000 00:00 0 aacff000-aad00000 ---p 00000000 00:00 0 aad00000-aade7000 rw-p 00000000 00:00 0 aade7000-aae00000 ---p 00000000 00:00 0 aae00000-aaf00000 rw-p 00000000 00:00 0 aaf2b000-aaf2c000 ---p 00000000 00:00 0 aaf2c000-aaffb000 rw-p 00000000 00:00 0 aaffb000-aaffc000 ---p 00000000 00:00 0 aaffc000-ab7fc000 rw-p 00000000 00:00 0 ab7fc000-ab7fd000 ---p 00000000 00:00 0 ab7fd000-abffd000 rw-p 00000000 00:00 0 abffd000-abffe000 ---p 00000000 00:00 0 abffe000-ac7fe000 rw-p 00000000 00:00 0 ac7fe000-ac7ff000 ---p 00000000 00:00 0 ac7ff000-ad100000 rw-p 00000000 00:00 0 ad100000-ad200000 rw-p 00000000 00:00 0 ad200000-ad201000 ---p 00000000 00:00 0 ad201000-ad248000 rw-p 00000000 00:00 0 ad248000-ad24e000 rw-s 00000000 09:01 83870198 /var/lib/mysql-500/tc.log ad24e000-ad2ba000 rw-p 00000000 00:00 0 ad2ba000-ad2bb000 ---p 00000000 00:00 0 ad2bb000-adabb000 rw-p 00000000 00:00 0 adabb000-adabc000 ---p 00000000 00:00 0 adabc000-b421a000 rw-p 00000000 00:00 0 b421d000-b6e91000 rw-p 00000000 00:00 0 b6e91000-b6e9a000 r-xp 00000000 09:01 20055584 /lib/libnss_nis-2.9.so b6e9a000-b6e9b000 r--p 00008000 09:01 20055584 /lib/libnss_nis-2.9.so b6e9b000-b6e9c000 rw-p 00009000 09:01 20055584 /lib/libnss_nis-2.9.so b6e9c000-b6ea3000 r-xp 00000000 09:01 20055580 /lib/libnss_compat-2.9.so b6ea3000-b6ea4000 r--p 00006000 09:01 20055580 /lib/libnss_compat-2.9.so b6ea4000-b6ea5000 rw-p 00007000 09:01 20055580 /lib/libnss_compat-2.9.so b6ea5000-b6eae000 r-xp 00000000 09:01 20055582 /lib/libnss_files-2.9.so b6eae000-b6eaf000 r--p 00008000 09:01 20055582 /lib/libnss_files-2.9.so b6eaf000-b6eb0000 rw-p 00009000 09:01 20055582 /lib/libnss_files-2.9.so b6eb0000-b6ebc000 r-xp 00000000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6ebc000-b6ebd000 r--p 0000b000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6ebd000-b6ebe000 rw-p 0000c000 09:01 81594922 /usr/lib/gcc/i686-pc-linux-gnu/4.3.2/libgcc_s.so.1 b6ebe000-b6ebf000 ---p 00000000 00:00 0 b6ebf000-b76c1000 rw-p 00000000 00:00 0 b76c1000-b77fe000 r-xp 00000000 09:01 20055572 /lib/libc-2.9.so b77fe000-b7800000 r--p 0013d000 09:01 20055572 /lib/libc-2.9.so b7800000-b7801000 rw-p 0013f000 09:01 20055572 /lib/libc-2.9.so b7801000-b7804000 rw-p 00000000 00:00 0 b7804000-b7828000 r-xp 00000000 09:01 20055578 /lib/libm-2.9.so b7828000-b7829000 r--p 00023000 09:01 20055578 /lib/libm-2.9.so b7829000-b782a000 rw-p 00024000 09:01 20055578 /lib/libm-2.9.so b782a000-b7833000 r-xp 00000000 09:01 20055574 /lib/libcrypt-2.9.so b7833000-b7834000 r--p 00008000 09:01 20055574 /lib/libcrypt-2.9.so b7834000-b7835000 rw-p 00009000 09:01 20055574 /lib/libcrypt-2.9.so b7835000-b785d000 rw-p 00000000 00:00 0 b785d000-b786e000 r-xp 00000000 09:01 20055594 /lib/libresolv-2.9.so b786e000-b786f000 r--p 00010000 09:01 20055594 /lib/libresolv-2.9.so b786f000-b7870000 rw-p 00011000 09:01 20055594 /lib/libresolv-2.9.so b7870000-b7872000 rw-p 00000000 00:00 0 b7872000-b7879000 r-xp 00000000 09:01 20055595 /lib/librt-2.9.so b7879000-b787a000 r--p 00006000 09:01 20055595 /lib/librt-2.9.so b787a000-b787b000 rw-p 00007000 09:01 20055595 /lib/librt-2.9.so b787b000-b787d000 r-xp 00000000 09:01 20055575 /lib/libdl-2.9.so b787d000-b787e000 r--p 00001000 09:01 20055575 /lib/libdl-2.9.so b787e000-b787f000 rw-p 00002000 09:01 20055575 /lib/libdl-2.9.so b787f000-b7885000 r-xp 00000000 09:01 20054132 /lib/libwrap.so.0.7.6 b7885000-b7887000 rw-p 00005000 09:01 20054132 /lib/libwrap.so.0.7.6 b7887000-b789b000 r-xp 00000000 09:01 20055591 /lib/libpthread-2.9.so b789b000-b789c000 r--p 00013000 09:01 20055591 /lib/libpthread-2.9.so b789c000-b789d000 rw-p 00014000 09:01 20055591 /lib/libpthread-2.9.so b789d000-b789f000 rw-p 00000000 00:00 0 b789f000-b78b2000 r-xp 00000000 09:01 20055579 /lib/libnsl-2.9.so b78b2000-b78b3000 r--p 00012000 09:01 20055579 /lib/libnsl-2.9.so b78b3000-b78b4000 rw-p 00013000 09:01 20055579 /lib/libnsl-2.9.so b78b4000-b78b7000 rw-p 00000000 00:00 0 b78b8000-b78bc000 rw-p 00000000 00:00 0 b78bc000-b78d8000 r-xp 00000000 09:01 20055569 /lib/ld-2.9.so b78d8000-b78d9000 r--p 0001c000 09:01 20055569 /lib/ld-2.9.so b78d9000-b78da000 rw-p 0001d000 09:01 20055569 /lib/ld-2.9.so bfd9d000-bfdb2000 rw-p 00000000 00:00 0 [stack] ffffe000-fffff000 r-xp 00000000 00:00 0 [vdso] 100121 16:13:22 - mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=1048576 max_used_connections=175 max_threads=502 threads_connected=103 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 1578146 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. thd: 0x9c380128 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0xa31843b8 thread_stack 0x48000 /usr/sbin/mysqld-5(my_print_stacktrace+0x21)[0x85ad6c1] /usr/sbin/mysqld-5(handle_segfault+0x391)[0x821acc1] [0xffffe400] /lib/libc.so.6(abort+0x188)[0xb76ece98] /lib/libc.so.6[0xb772683d] /lib/libc.so.6[0xb772c714] /lib/libc.so.6[0xb772ddd3] /lib/libc.so.6(cfree+0x9c)[0xb772dfcc] /usr/sbin/mysqld-5[0x8503ed1] /usr/sbin/mysqld-5[0x85075af] /usr/sbin/mysqld-5[0x84af7ef] /usr/sbin/mysqld-5[0x84af94d] /usr/sbin/mysqld-5(_Z13rr_sequentialP11READ_RECORD+0x25)[0x82f7cd5] /usr/sbin/mysqld-5(_Z10sub_selectP4JOINP13st_join_tableb+0x76)[0x8288f16] /usr/sbin/mysqld-5[0x82892e3] /usr/sbin/mysqld-5(_ZN4JOIN4execEv+0x1311)[0x8297411] /usr/sbin/mysqld-5(_Z12mysql_selectP3THDPPP4ItemP10TABLE_LISTjR4ListIS1_ES2_jP8st_orderSB_S2_SB_yP13select 3d)[0x82988dd] /usr/sbin/mysqld-5(_Z13handle_selectP3THDP6st_lexP13select_resultm+0x15d)[0x829924d] /usr/sbin/mysqld-5[0x8226d47] /usr/sbin/mysqld-5(_Z21mysql_execute_commandP3THD+0x458)[0x8229658] /usr/sbin/mysqld-5(_Z11mysql_parseP3THDPKcjPS2_+0x1ff)[0x82313bf] /usr/sbin/mysqld-5(_Z16dispatch_command19enum_server_commandP3THDPcj+0xb6e)[0x8231f4e] /usr/sbin/mysqld-5(_Z10do_commandP3THD+0xe1)[0x8232ac1] /usr/sbin/mysqld-5(handle_one_connection+0xca)[0x8222a1a] /lib/libpthread.so.0[0xb788d16f] /lib/libc.so.6(clone+0x5e)[0xb778fc0e] Trying to get some variables. Some pointers may be invalid and cause the dump to abort... thd->query at 0x952b670 = SELECT SQL_NO_CACHE * FROM cache_hash LIMIT 307, 367 thd->thread_id=2317111 thd->killed=NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.

And now i have the problem that the customer mysql servers are crashing on our nehalem blade servers with mpt2sas controllers, where the drivers are only in > 2.6.31
So i cant boot them to 2.6.27 ..

I hope you find an solution for this problem soon.
If i can help you please let me know what to do.

Greetings,
Daniel

grsecurity forums

Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1

Re: Problem with 2.6.23.1-grsec and MySQL 5.0.32-7etch1