grsec: bruteforce prevention initiated due to crash
Posted: Thu Oct 31, 2013 4:56 amThis is recent development on my Debian machine.
I'll paste here two Call Traces from /var/log/kern.log of one of my Debian system, the one that I used to go online with, that was hacked and is now soon to be zeroed out and (clone-)restored from backup.
This one roughly corresponds to what I described in these, mailing list/forum :
http://marc.info/?l=mutt-users&m=138269801407404&w=2 (roughly the second half of the text)
http://www.fsarchiver.org/forums/viewtopic.php?f=17&t=1674
In the Mull mail-list you can read:
And there is another one Call Trace:
While after the time of the first Call Trace I was able to reboot into the system and login normally (well, not that me the avarage Joe user noticed much is what mean by "normally"...), after the second there was no more logging into the system... So I had to investigate...
I made sure I took images of the system.
The disk dump images (taken with dd=/dev/the_device_in_point ... of=the_filename_to_store_it.dd and the fsarchiver archiver files are these:
One line of particular interest to the initiated public (
) I guess is this line from the same /var/log/kern.log:
Actually I don't know how much of the context before and after these two lines I would need to provide...
Because there is more to be told about this blind-me-having-been-hacked... I really have lost a lot of my physical eyesight and the lamp that I wasn't able to read well the fineprint inscription on, and that was also blinking, was the, wait now [threatening-mocking music in the background]:
the WiFi lamp
Of course I intend to provide the whole truth! I'm not like those developers that put spying code in their closed-source binaries for your GNU/Linuces and declare their programs "free". I'll admit if I made wrong assumptions in my previous posts, which I can see are pretty popular reads.
(But pls. notice that when I talked of agancies, that I never was precise in my suspicion, that my references to the programs/subjects in question always ended with something like "...or some such subjects..." or used similar non-individualized references.)
I don't know if there is interest on the part of the developers or the initiated readers. Pls. some of the addressees let me know!
I won't hurry myself. Along with unrelated work in my life, it took me almost five days to prepare the systems and take the images (and I also wrote a tip for newbies on Debian Forums:
Grsecurity/Pax installation on Debian GNU/Linux
http://forums.debian.net/viewtopic.php? ... 6&p=516981
in that meantime, so...), because I move rather slowly, namely I'm still struggling to grasp all these concepts,
Miroslav Rovis,
Zagreb, Croatia
I'll paste here two Call Traces from /var/log/kern.log of one of my Debian system, the one that I used to go online with, that was hacked and is now soon to be zeroed out and (clone-)restored from backup.
- Code: Select all
Oct 25 04:37:56 naibd9 kernel: [775666.832037] ------------[ cut here ]------------
Oct 25 04:37:56 naibd9 kernel: [775666.832052] WARNING: CPU: 0 PID: 22905 at net/sched/sch_generic.c:260 dev_watchdog+0xe1/0x14e()
Oct 25 04:37:56 naibd9 kernel: [775666.832055] NETDEV WATCHDOG: eth1 (r8169): transmit queue 0 timed out
Oct 25 04:37:56 naibd9 kernel: [775666.832056] Modules linked in: iptable_filter ip_tables x_tables nfnetlink_queue nfnetlink_log nfnetlink nfsv3 rpcsec_gss_krb5 nfsv4 nls_utf8 nls_cp437 vfat fat ext3 jbd bnep rfcomm bluetooth rfkill nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc ext2 loop fuse sg sr_mod cdrom radeon snd_hda_codec_realtek snd_hda_intel usb_storage snd_hda_codec ttm drm_kms_helper drm snd_hwdep snd_pcm snd_page_alloc snd_seq ohci_pci snd_timer snd_seq_device ohci_hcd ehci_pci firewire_ohci firewire_core ehci_hcd mperf usbcore i2c_ali15x3 psmouse evdev powernow_k8 pata_ali crc_itu_t shpchp snd serio_raw usb_common i2c_ali1535 i2c_algo_bit processor pcspkr floppy i2c_core sata_sil24 k8temp sky2 button soundcore edac_mce_amd r8169 mii edac_core ext4 crc16 mbcache jbd2 dm_mod sd_mod crc_t10dif ahci ata_generic libahci libata scsi_mod thermal thermal_sys
Oct 25 04:37:56 naibd9 kernel: [775666.832115] CPU: 0 PID: 22905 Comm: ffmpeg Not tainted 3.11.3-grsec-131009 #1
Oct 25 04:37:56 naibd9 kernel: [775666.832117] Hardware name: /AT8 32X(ATI RD580-ULI M1575), BIOS 6.00 PG 05/19/2006
Oct 25 04:37:56 naibd9 kernel: [775666.832119] 0000000000000000 0000000000000000 ffffffff813b093c ffff88012fc03e40
Oct 25 04:37:56 naibd9 kernel: [775666.832123] ffffffff8103c8cd ffffffff81301a2c ffffffff81b0944d ffff88012fc03e98
Oct 25 04:37:56 naibd9 kernel: [775666.832126] ffffffff8130194b ffff8801281e8348 ffffffff8103c9ce 0000000000000104
Oct 25 04:37:56 naibd9 kernel: [775666.832129] Call Trace:
Oct 25 04:37:56 naibd9 kernel: [775666.832131] <IRQ> [<ffffffff813b093c>] ? dump_stack+0x41/0x51
Oct 25 04:37:56 naibd9 kernel: [775666.832143] [<ffffffff8103c8cd>] ? warn_slowpath_common+0x74/0x8a
Oct 25 04:37:56 naibd9 kernel: [775666.832146] [<ffffffff81301a2c>] ? dev_watchdog+0xe1/0x14e
Oct 25 04:37:56 naibd9 kernel: [775666.832150] [<ffffffff8130194b>] ? netif_tx_lock+0x7b/0x7b
Oct 25 04:37:56 naibd9 kernel: [775666.832152] [<ffffffff8103c9ce>] ? warn_slowpath_fmt+0x5e/0x65
Oct 25 04:37:56 naibd9 kernel: [775666.832156] [<ffffffff81301938>] ? netif_tx_lock+0x68/0x7b
Oct 25 04:37:56 naibd9 kernel: [775666.832170] [<ffffffffa005f58d>] ? rtl8169_pci_driver_exit+0x2093/0x472b [r8169]
Oct 25 04:37:56 naibd9 kernel: [775666.832173] [<ffffffff81301a2c>] ? dev_watchdog+0xe1/0x14e
Oct 25 04:37:56 naibd9 kernel: [775666.832177] [<ffffffff8104813f>] ? call_timer_fn+0x45/0xe4
Oct 25 04:37:56 naibd9 kernel: [775666.832180] [<ffffffff8130194b>] ? netif_tx_lock+0x7b/0x7b
Oct 25 04:37:56 naibd9 kernel: [775666.832184] [<ffffffff8104871d>] ? run_timer_softirq+0x184/0x1c9
Oct 25 04:37:56 naibd9 kernel: [775666.832188] [<ffffffff81040f79>] ? __do_softirq+0xd5/0x1e2
Oct 25 04:37:56 naibd9 kernel: [775666.832191] [<ffffffff81041135>] ? irq_exit+0x34/0x71
Oct 25 04:37:56 naibd9 kernel: [775666.832195] [<ffffffff81026cc2>] ? smp_apic_timer_interrupt+0x30/0x3c
Oct 25 04:37:56 naibd9 kernel: [775666.832199] [<ffffffff813bb20c>] ? apic_timer_interrupt+0x7c/0x90
Oct 25 04:37:56 naibd9 kernel: [775666.832200] <EOI> [<ffffffff813b55b4>] ? retint_swapgs+0xe/0x11
Oct 25 04:37:56 naibd9 kernel: [775666.832206] ---[ end trace f5b291fb654597d6 ]---
This one roughly corresponds to what I described in these, mailing list/forum :
http://marc.info/?l=mutt-users&m=138269801407404&w=2 (roughly the second half of the text)
http://www.fsarchiver.org/forums/viewtopic.php?f=17&t=1674
In the Mull mail-list you can read:
...was able to recognize it only physically at the hardware switch where while nothing was updated or downloaded by me, there was some frenzied update or download on the link (the lamp blinking uninterruptedly) that pertains to the connection to the internet...
And there is another one Call Trace:
- Code: Select all
Oct 25 15:32:00 naibd9 kernel: [ 530.832025] ------------[ cut here ]------------
Oct 25 15:32:00 naibd9 kernel: [ 530.832046] WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:260 dev_watchdog+0xe1/0x14e()
Oct 25 15:32:00 naibd9 kernel: [ 530.832052] NETDEV WATCHDOG: eth2 (r8169): transmit queue 0 timed out
Oct 25 15:32:00 naibd9 kernel: [ 530.832056] Modules linked in: nls_utf8 nls_cp437 vfat fat ext3 jbd bnep rfcomm bluetooth rfkill nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc ext2 loop fuse usb_storage sg sr_mod cdrom radeon snd_hda_codec_realtek snd_hda_intel snd_hda_codec ohci_pci snd_hwdep ohci_hcd snd_pcm ehci_pci snd_page_alloc snd_seq ehci_hcd snd_timer snd_seq_device snd ttm drm_kms_helper evdev psmouse drm firewire_ohci serio_raw firewire_core usbcore powernow_k8 pata_ali mperf r8169 sata_sil24 floppy crc_itu_t shpchp sky2 pcspkr processor edac_mce_amd i2c_ali15x3 usb_common soundcore i2c_algo_bit i2c_ali1535 i2c_core mii k8temp button edac_core ext4 crc16 mbcache jbd2 dm_mod sd_mod crc_t10dif ata_generic ahci libahci libata scsi_mod thermal thermal_sys
Oct 25 15:32:00 naibd9 kernel: [ 530.832160] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.11.3-grsec-131009 #1
Oct 25 15:32:00 naibd9 kernel: [ 530.832165] Hardware name: /AT8 32X(ATI RD580-ULI M1575), BIOS 6.00 PG 05/19/2006
Oct 25 15:32:00 naibd9 kernel: [ 530.832169] 0000000000000000 0000000000000000 ffffffff813b093c ffff88012fc03e40
Oct 25 15:32:00 naibd9 kernel: [ 530.832177] ffffffff8103c8cd ffffffff81301a2c ffffffff81b0944d ffff88012fc03e98
Oct 25 15:32:00 naibd9 kernel: [ 530.832184] ffffffff8130194b ffff88012ae8c348 ffffffff8103c9ce 0000000000000104
Oct 25 15:32:00 naibd9 kernel: [ 530.832191] Call Trace:
Oct 25 15:32:00 naibd9 kernel: [ 530.832195] <IRQ> [<ffffffff813b093c>] ? dump_stack+0x41/0x51
Oct 25 15:32:00 naibd9 kernel: [ 530.832214] [<ffffffff8103c8cd>] ? warn_slowpath_common+0x74/0x8a
Oct 25 15:32:00 naibd9 kernel: [ 530.832222] [<ffffffff81301a2c>] ? dev_watchdog+0xe1/0x14e
Oct 25 15:32:00 naibd9 kernel: [ 530.832230] [<ffffffff8130194b>] ? netif_tx_lock+0x7b/0x7b
Oct 25 15:32:00 naibd9 kernel: [ 530.832236] [<ffffffff8103c9ce>] ? warn_slowpath_fmt+0x5e/0x65
Oct 25 15:32:00 naibd9 kernel: [ 530.832244] [<ffffffff81301938>] ? netif_tx_lock+0x68/0x7b
Oct 25 15:32:00 naibd9 kernel: [ 530.832267] [<ffffffffa025e58d>] ? rtl8169_pci_driver_exit+0x2093/0x472b [r8169]
Oct 25 15:32:00 naibd9 kernel: [ 530.832275] [<ffffffff81301a2c>] ? dev_watchdog+0xe1/0x14e
Oct 25 15:32:00 naibd9 kernel: [ 530.832283] [<ffffffff8104813f>] ? call_timer_fn+0x45/0xe4
Oct 25 15:32:00 naibd9 kernel: [ 530.832290] [<ffffffff8130194b>] ? netif_tx_lock+0x7b/0x7b
Oct 25 15:32:00 naibd9 kernel: [ 530.832298] [<ffffffff8104871d>] ? run_timer_softirq+0x184/0x1c9
Oct 25 15:32:00 naibd9 kernel: [ 530.832307] [<ffffffff81040f79>] ? __do_softirq+0xd5/0x1e2
Oct 25 15:32:00 naibd9 kernel: [ 530.832315] [<ffffffff81041135>] ? irq_exit+0x34/0x71
Oct 25 15:32:00 naibd9 kernel: [ 530.832323] [<ffffffff81026cc2>] ? smp_apic_timer_interrupt+0x30/0x3c
Oct 25 15:32:00 naibd9 kernel: [ 530.832331] [<ffffffff813bb20c>] ? apic_timer_interrupt+0x7c/0x90
Oct 25 15:32:00 naibd9 kernel: [ 530.832335] <EOI> [<ffffffff8100995a>] ? default_idle+0x14/0x3a
Oct 25 15:32:00 naibd9 kernel: [ 530.832348] [<ffffffff81009ebc>] ? arch_cpu_idle+0x5/0x14
Oct 25 15:32:00 naibd9 kernel: [ 530.832357] [<ffffffff810781f1>] ? cpu_startup_entry+0xf9/0x167
Oct 25 15:32:00 naibd9 kernel: [ 530.832368] [<ffffffff82057d70>] ? 0xffffffff82057d6f
Oct 25 15:32:00 naibd9 kernel: [ 530.832377] [<ffffffff820577c6>] ? 0xffffffff820577c5
Oct 25 15:32:00 naibd9 kernel: [ 530.832382] ---[ end trace f5a54bc41391b338 ]---
While after the time of the first Call Trace I was able to reboot into the system and login normally (well, not that me the avarage Joe user noticed much is what mean by "normally"...), after the second there was no more logging into the system... So I had to investigate...
I made sure I took images of the system.
The disk dump images (taken with dd=/dev/the_device_in_point ... of=the_filename_to_store_it.dd and the fsarchiver archiver files are these:
- Code: Select all
d4fed1032c0985e378e6d639ff57351c9f7fdadd391d1dbcbff9d7ac54af7ffc D1026_naibd9_vg_r-root.dd
252680d7eb9e1aa388079ad56ae1366c8cb40bdc0884a518c1074fd822c1360c D1026_naibd9_vg_r-tmp.dd
62b837625ad798f1973603bafc5a8e97809b76c570909e5c4126955c58cf442b D1026_naibd9_vg_r-usr.dd
9c0da15fa855dfc85a34568521e6396852faa0e3349a116841dce831e4bf9b24 D1026_naibd9_vg_r-var.dd
7f9e358f7dcc92d5263ae81823d5e590495db09facbc07ec74661d1c881ca5f1 vg_r-root.fsa
c0b05474f11a503a759e27c09409e8b5f3c1bb7ffa5c68a9669d2af8b74a88de vg_r-tmp-var.fsa
d110f23d51c560684b5bb6d39f9063fd24edaaf8e96dfff86571d46b26957f2a vg_r-usr.fsa
One line of particular interest to the initiated public (
) I guess is this line from the same /var/log/kern.log:- Code: Select all
Oct 25 15:23:54 naibd9 kernel: [ 44.479728] grsec: Invalid alignment/Bus error occurred at 0000030e9d659080 in /usr/sbin/amavisd-new[/usr/sbin/amavi:1952] uid/euid:118/118 gid/egid:128/128, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0
Oct 25 15:23:54 naibd9 kernel: [ 44.483397] grsec: bruteforce prevention initiated due to crash of /usr/sbin/amavisd-new against uid 118, banning suid/sgid execs for 15 minutes. Please investigate the crash report for /usr/sbin/amavisd-new[/usr/sbin/amavi:1952] uid/euid:118/118 gid/egid:128/128, parent /sbin/init[init:1] uid/euid:0/0 gid/egid:0/0
Actually I don't know how much of the context before and after these two lines I would need to provide...
Because there is more to be told about this blind-me-having-been-hacked... I really have lost a lot of my physical eyesight and the lamp that I wasn't able to read well the fineprint inscription on, and that was also blinking, was the, wait now [threatening-mocking music in the background]:
the WiFi lamp
Of course I intend to provide the whole truth! I'm not like those developers that put spying code in their closed-source binaries for your GNU/Linuces and declare their programs "free". I'll admit if I made wrong assumptions in my previous posts, which I can see are pretty popular reads.
(But pls. notice that when I talked of agancies, that I never was precise in my suspicion, that my references to the programs/subjects in question always ended with something like "...or some such subjects..." or used similar non-individualized references.)
I don't know if there is interest on the part of the developers or the initiated readers. Pls. some of the addressees let me know!
I won't hurry myself. Along with unrelated work in my life, it took me almost five days to prepare the systems and take the images (and I also wrote a tip for newbies on Debian Forums:
Grsecurity/Pax installation on Debian GNU/Linux
http://forums.debian.net/viewtopic.php? ... 6&p=516981
in that meantime, so...), because I move rather slowly, namely I'm still struggling to grasp all these concepts,
Miroslav Rovis,
Zagreb, Croatia