As for the second question, the sort answer is that chances are that you really do not need one but for the shake of the experiment lets get our hands dirty!

First of all, I need to clarify that my interest in this topic was also risen by the fact that Verisign has switched to a two-tier hierarchy of Certificate Authorities, and this has some implications specially in the configuration of web server software:

“As of April 2006, all SSL certificates issued by VeriSign require the installation of an Intermediate CA Certificate. The SSL certificates are signed by an Intermediate CA using a two-tier hierarchy (also known as trust chain) which enhances the security of your SSL Certificate. If the proper Intermediate CA is not installed on the server, your customers will see browser errors and may choose not to proceed further and close their browser.” (boldface is mine)

This means that while the users do not need to modify anything (if their browser already has Verisigns Root CA certificate) the server owners need to ensure that the server is able to provide the so called trust chain to the users’ browser when the SSL handshake is performed.

Never mind, lets get back to it. In order to get your Intermediate CA working, first you need a Root CA (if you already have a CA, feel free to skip the next section). Remember that in order to get this working you need to have a copy of the openssl toolkit installed in your system.

Configure the Root CA

mkdir /var/ca
cd /var/ca/
mkdir certs crl newcerts private
echo "01" > serial
cp /dev/null index.txt
# beware that the location of the sample file is dependent on your environment
cp /usr/lib/ssl/openssl.cnf .

You may want to modify some of the settings in the configuration file to save you some time in the future when creating the certificates: default_bits, countryName, stateOrProvinceName, 0.organizationName_default, organizationalUnitName and emailAddress.

Now you are ready to create the CA:

# generate a private key
openssl genrsa -des3 -out private/cakey.key 4096
# create a self-signed certificate valid for 5 years
openssl req -new -x509 -nodes -sha1 -days 1825 -key private/cakey.pem -out cacert.pem
# go for the default values if you adapted the settings in the openssl.cnf file or enter the values you desire

Now you have everything you need to run a successful CA.

Configure an Intermediate CA

The idea is simple, we will create a new CA following the same template that we used in the previous section, but this time instead of generating a self-signed certificate we will generate a certificate sign request that we will sign using the Root CA.

First we create the folder structure:

cd /var/ca/
mkdir ca2008
cd ca2008
cp ../openssl.cnf .
mkdir certs crl newcerts private
echo "01" > serial
cp /dev/null index.txt

Then the Intermediate CA private key:

#generate the key
openssl genrsa -des3 -out private/cakey.pem 4096
#generate a signing request (valid for 1year)
openssl req -new -sha1 -key private/cakey.pem -out ca2008.csr
# go for the default values if you adapted the settings in the openssl.cnf file or enter the values you desire

Move the sign request to the Root CA directory and sign it:

mv ca2008.csr ..
cd ..
openssl ca -extensions v3_ca -days 365 -out ca2008.crt -in ca2008.csr -config openssl.cnf
mv ca2008.* ca2008/
cd ca2008/
mv ca2008.crt cacert.pem

And that was it. The next thing to do is start using your Intermediate CA to sign your new certificates. But just before that, remember that
to verify a certificate signed by an Intermediate CA the web browser has to verify both the certificate against the Intermediate CA and the certificate of the Intermediate CA against a Root CA.

In order to allow the browser to do this, a certificate chain file needs to be installed in the server. A certificate chain is a plaintext file that contains all the certificates from the Authority issuing a given certificate up to the Root of the certificate tree. In this case our chain has only two levels and the chain file is created like this:-

# first the intermediate CA certificate
cat cacert.pem > chain.crt
# then the Root CA cert
cat ../cacert.pem >> chain.crt

This file is the one you need to specify in the SSLCertificateChainFile of your server.

Create a new server certificate

# make sure you are in the Intermediate CA folder and not in the Root CA one
cd /var/ca/ca2008/
# create the private key
openssl genrsa -des3 -out {server_name}.key 4096
# generate a certificate sign request
openssl req -new -key {server_name}.key -out {server_name}.csr  
# sign the request with the Intermediate CA
openssl ca -config openssl.cnf -policy policy_anything -out {server_name}.crt -infiles {server_name}.csr
# and store the server files in the certs/ directory
mkdir certs/{server_name}
mv {server_name}.key {server_name}.csr {server_name}.crt certs/

Then you should securely copy the .key and .crt files to the server and configure it to use them.

Apache server configuration

Just in case you are using Apache server and for the shake of completeness, these are the settings that you need to modify (possibly in your extra/http-ssl.conf):-

SSLCertificateFile /var/ca/ca2008/certs/{server_name}.crt
SSLCertificateKeyFile /var/ca/ca2008/certs/{server_name}.key
SSLCertificateChainFile /var/ca/ca2008/chain.crt

References

• SSL/TLS Strong Encryption: FAQ
• Creating Your Own CA
• Be your own Certificate Authority
• Very brief introduction to create a CA and a CERT

The only way we could think of getting our hands on the communication was to write a small set of scripts to trick the client and encapsulate the communication inside HTTP requests that we could then manipulate using standard proxy tools such as burp.

Although the information and scripts described in this post are focussed on intercepting a XML communication, the same principles apply to man in the middle any ASCII protocol such as smtp, ftp or pop.

update: slides available here

The first step is to trick the client to connect to our local box instead of connecting to the remote server, this is done by adjusting the hosts file.

A ruby script will sit in the middle of the communication and will be able to intercept and modify messages sent and received by the client:-

Once this is done, our attack will need three elements:

• the xmitm script.
• an external web proxy tool.
• a dummy web server.

The script will intercept the connection and send the data to the proxy. We need the dummy server (the body of the response will be the body of the request) to close the loop with the proxy (I will add some nice graphs to clarify this soon).

The original XML message is encapsulated in an HTTP request and passed through the proxy. The user can inspect and modify the message using a standard web proxy tool. The request is then forwared to a dummy *echo* web server that replies with the same payload that was requested. The script can extract the modified payload and forward it to the server.

The same process is applied to incoming messages.

Below is the main body of the script (you can also grab the code):-

# create a server that accepts connections from the client
server = TCPServer.new($local_host, $local_port)

while(local = server.accept ) do
  # everytime we accept a connection for the client, we open a connection
  # with the server to stablish the dialog.
  remote = TCPSocket.new($remote_host, $remote_port)
  
  # if one of the ends of the communication closes the socket, we
  # toggle this flag
  alive = true
    
  while alive do
    # see the explanation below
    result = select([local, remote], nil, nil)
  
    if result != nil then
      for socket in result[0]

        # detect if one end of the connection is closed and
        # close the other end
        if (socket.eof?)
          local.close
          remote.close
          alive = false
          break
        end
        
        # read the information that one peer wants to send to the other
        data = socket.gets($eom)

        # encapsulate the data into an HTTP proxy request
        res = Net::HTTP.new($proxy_host, $proxy_port).start do |http| 
          req = Net::HTTP::Post.new("http://#{$dummyhttp_host}:#{$dummyhttp_port}/")
          req.body= data
          http.request(req)
        end

        modified_data = res.body.chomp

        # send the modified data to the other end of the connection        
        if (socket == local)
          remote.puts(modified_data)
        else
          local.puts(modified_data)
        end
        socket.flush
      end
    end
  end
end

What the script does can be summarized in the following steps:

1. Create a TCP server, listening on the port the client is expecting.
2. For each connection accepted:
   • Open a connection with the remote server.
   • Wait until one end of the communication (first the client, then the server, then the client, etc.) has something to transmit.
   • Grab the XML message.
   • Put that message as a payload of a new Net::HTTP::Post request.
   • Send the request to the external web proxy.
   • Grab the body of the response given by the proxy (already modified by the user using the external proxy).
   • Send the modified request to the other end of the line.

The most interesting piece of the code is the one regarding Kernel#select function that waits for data to become available from input/output devices.

A note regarding the specifics of the protocol we were dealing with, each peer ends its messages using a special character (a NULL byte), that caracter is defined in the $eom variable and the script keeps reading the socket until that end of message character is read.

The last piece of the puzzle is the dummy HTTP server. I coded two flavours: a ruby version and a java version (not yet available for download based on the SingleFileHTTPServer example). You can pick your choice. Here is the ruby one:-

require 'webrick'

include WEBrick

# create the server, no output, disable logging
s = HTTPServer.new(
  :Port => 2000,
  :Logger => Log.new(nil, BasicLog::FATAL),
  :AccessLog => []  )

# the *echo* functionality
s.mount_proc("/") do |req, res|
  res.body = req.body
  res['Content-Type'] = req['Content-Type']
end

# clean tear down
trap('INT') { s.shutdown }

s.start

And this completes the XML protocol man-in-the-middle DIY kit. Hope you find it useful.

dradis is written in ruby and combines various technologies/libraries. For the server side it uses the Ruby on Rails (RoR) MVC framework, for the client plain ruby and also the Qt library. The documentation page contains useful information on the architecture, instalation process, etc.

It was the development of dradis that lead me to write most of my ruby related posts since last summer. It has been really nice to spend time developing it and I have learned lots of interesting stuff.

dradis is also my first serious contribution to the security community and I am really excited to see what kind of feedback I get.

Before you download it, I recommend you to have a look at the “dradis, an overview” set of slides. You may also find useful two flash videos I created to show what dradis is capable of:

• intro: This video shows how the information is shared between the clients: you add new information from the command line interface and the graphical interface is notified. You can have different clients running different interfaces, they will all share the same information. Play video.
• graphical user interface: Learn what the different elements of the graphical interface are, how to perform basic tasks and how to get help on dradis commands. Play video.

Enjoy, and let me know about your toughts on dradis. Does it look interesting? Have you found it useful? Will it fit in your company way of pentesting?

With Net::DHCP you will be able to craft custom DHCP packages and have access to all the fields defined for the protocol.

The Dynamic Host Configuration Protocol (DHCP) provides a framework for passing configuration information to hosts on a TCPIP network.

UDP is used as transport protocol, all packets sent by the client have a source port of 68 and a destination port of 67. Likewise, packets originated in the server will have source port 67 and destination port 68.

You can create messages, attach options and pack them as the payload of a UDP packet. In the same way, you can use a capturing library such as Ruby/pcap to get packages from the network and parse their contents into well formed and comprehensible ruby objects.

This project’s SVN repository can be checked out through anonymous access with the following command(s):-

svn checkout http://netdhcp.rubyforge.org/svn/
or
svn checkout svn://rubyforge.org/var/svn/netdhcp


I hope you find the library useful. Let me know if you are using it for something!

References

• rfc2131: Dynamic Host Configuration Protocol
• rfc2132: DHCP Options and BOOTP Vendor Extensions
• rfc2563: DHCP Option to Disable Stateless Auto-Configuration in IPv4 Clients
• rfc4578: DHCP Options for the Intel Preboot eXecution Environment (PXE)
• rfc4702: The DHCP Client Fully Qualified Domain Name (FQDN) Option

The robots exclusion standard or robots.txt protocol is a convention to prevent cooperating web spiders and other web robots from accessing all or part of a website. The information specifying the parts that should not be accessed is specified in a file called robots.txt in the top-level directory of the website.

Here is a script that checks for the presence of the file in a list of hosts (you can download the source code). Two main parts can be distinguished: command line parsing and file download.

You can call the script in two different ways. Either you do not specify the protocol (and HTTP will be used):-

./robots.sh <host1> <host2> ...

Or you specify the protocol with:

./robots.sh  -p [http|https] <host1> <host2> ...

Let’s see how this is done:

PROTO=( http https )
HTTP=${PROTO[0]}
FILE=/tmp/robots.txt

# command line parsing
if [ "-p" == $1 ]
then
  for bar in ${PROTO[*]}
  do
    if [ $bar == $2 ];
    then
      HTTP=$2
      HOSTS=${*:3}
    fi
  done
else
  HOSTS=$*
fi

We check if the first argument is “-p” in which case, the next argument should be one of the allowed values (those in $PROTO array). If that is the case, we strip the first two parameters and put everything else in the $HOSTS variable. At the end of the code above, $HTTP will contain either http or https and $HOSTS will consist of a list of hosts whose robots.txt file existance we want to verify.

Once we know what protocol are we using and the list of targets, the only thing left is to try to download the robots.txt file of each server:-

for foo in $HOSTS; do
  echo "================"
  echo "Server: $foo ($HTTP)"
  CODE=`wget -O $FILE $HTTP://$foo/robots.txt 2>&1 | grep HTTP | head -1 | awk '{print $6}'`
  echo "Code: $CODE"
  if [ "200" == $CODE ]
  then
    echo "Contents:"
    echo "----------------"
    cat $FILE
    rm $FILE
    echo "----------------"
  fi
done

If the response code is 200 OK we cat the file to standard output. Otherwise we just move on to the next target of the list. The only tricky bit of the previous code is:

wget -O $FILE $HTTP://$foo/robots.txt 2>&1 | grep HTTP | head -1 | awk '{print $6}'

Where we try to download the file saving it to the location specified by $FILE. In order to get the HTTP error code we redirect standard error to standard output using 2>&1.

One last word, it is acknowledged that the script does not follow HTTP redirects, but if the server replies with a redirect this means that effectively, no robots.txt file is present.

#!/bin/bash

###
### IPTables config file
### Based on the rules compiled by Ranjit San aka 'the grasshopper'
### Created 2007-09-14 by Daniel Martin Gomez <etd[-at-]nomejortu.com>
### Revision 1
###

###
### define variables
###

### path to iptables 
IPT=/sbin/iptables

### This contains a list of approved Debian sites to get software updates.
DEBIAN_SITES=('194.109.137.218' '212.219.56.139' '212.219.56.133' '212.219.56.134' '212.219.56.135' '212.219.56.138')

### This contains the authorised DNS servers configured in /etc/resolv.conf. 
DNS_SERVERS=('') 

### This is a list of external IPs that you want to allow ssh access from.
OTHER_GATEWAYS=('') 

### This is a list of hosts authorised to try ICMP probes to check if the
### server is running. This could be your ISP's IPs
CONTROL_GATEWAYS=('')

### Types of ICMP probes to allow from the previous servers
ICMP_TYPES=('echo-reply' 'destination-unreachable' 'echo-request' 'ttl-exceeded')


#### NTP servers for time synch
NTP_SERVERS=('')

### ------------------------------------------------- do not change below this line

###
### INPUT
###

### will flush the chains or all rules one by one. Therefore all new rules will be created. 
$IPT -F 

### allows inbound packets to be processed
$IPT -P INPUT ACCEPT

### drops packets so that they can not come through one interface and flow out of another. 
$IPT -P FORWARD DROP 

### This allows outbound packets to be processed
$IPT -P OUTPUT ACCEPT


### allows ICMP types (defined above) for hosts in the control list 
for IP in ${CONTROL_GATEWAYS[@]}; do
	for ICMP in ${ICMP_TYPES[@]}; do
		$IPT -A INPUT -s $IP -p icmp --icmp-type $ICMP -j ACCEPT 
	done
done

### this accepts connections for http and https access from anywhere
$IPT -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
$IPT -A INPUT -p tcp -m tcp --dport 443 -j ACCEPT

### this allows remote administration using ssh from your other gateways.
for IP in ${OTHER_GATEWAYS[@]}; do 
    $IPT -A INPUT -s $IP -p tcp -m tcp --dport 22 -j ACCEPT
done


### this allows packets to start a new connection or allows packets that are
### already associated with a connection, required for stateful inspection.
$IPT -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

### this allows NTP traffic from NTP server
for NTP in ${NTP_SERVERS[@]}; do 
    $IPT -A INPUT -s $NTP -p udp -m udp --sport 123 -j ACCEPT
done

### we are about to drop everything else, so first log the discarded traffic
### just in case we want to know what *they* are trying.
$IPT -A INPUT -j LOG

### this drops any traffic that does not match to the INPUT rules
$IPT -A INPUT -j DROP 



###
### OUTPUT
###

### Allows traffic to authorised DNS servers
for IP in ${DNS_SERVERS[@]}; do 
    $IPT -A OUTPUT -d $IP -p udp -m udp --dport 53 -j ACCEPT
done

### Allows http traffic to debain sites for software updates. 
### Initial config rule
for IP in ${DEBIAN_SITES[@]}; do 
    $IPT -A OUTPUT -d $IP -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
done

### this allows packets to start a new connection or allows packets that are
### already associated with a connection, required for stateful inspection. 
$IPT -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT 

### this allows NTP traffic to the NTP servers
for NTP in ${NTP_SERVERS[@]}; do 
    $IPT -A OUTPUT -d $NTP -p udp -m udp --dport 123 -j ACCEPT
done

### this logs all OUTPUT traffic that does not match the rules before it beign
### dropped.
$IPT -A OUTPUT -j LOG 

### this drops any traffic that does not match to the OUTPUT rules
$IPT -A OUTPUT -j DROP

Just two things to add: First, do not forget to set your own values for the variables DNS_SERVERS, OTHER_GATEWAYS, CONTROL_GATEWAYS and NTP_SERVERS. And second, if you want your kung-fu up and ready after boot you may need to issue the following:-

cd /etc/init.d/
wget http://weblog.nomejortu.com/data/code/bash/firewall.sh
chmod +x firewall.sh
update-rc.d firewall.sh defaults

If you ever want to remove it from the boot sequence just issue:-

update-rc.d -f firewall.sh remove

Download an uncompress

cd /usr/local/src/
wget http://mirror.public-internet.co.uk/apache/httpd/httpd-2.2.4.tar.gz
tar -xvvzf httpd-2.2.4.tar.gz
wget http://uk2.php.net/get/php-5.2.3.tar.gz/from/this/mirror
tar -xvvzf php-5.2.3.tar.gz

Install software
Required by Apache:

apt-get install gcc make libc6-dev libc-dev \
linux-kernel-headers libssl-dev zlib1g-dev

Required by PHP:

apt-get install g++ g++-4.1 libfreetype6 \
libfreetype6-dev libgd2-noxpm libgd2-noxpm-dev \
libjpeg62 libjpeg62-dev libmysqlclient15-dev \
libpng12-0 libpng12-dev libstdc++6-4.1-dev \
libxml2 libxml2-dev

Tweak Apache
Get rid of the server banner, edit /usr/local/src/httpd-2.2.4/include/ap_release.h:

define AP_SERVER_BASEVENDOR "nomejortu"
define AP_SERVER_BASEPROJECT "nmt server"
define AP_SERVER_BASEPRODUCT "server"

Configure, compile and install

cd /usr/local/src/httpd-2.2.4/
./configure --disable-info --disable-autoindex \
--disable-include  --disable-userdir --disable-status \
--disable-imagemap --disable-cgid --disable-cgi \
--disable-proxy --enable-ssl=static \
--enable-rewrite=static --enable-dir=static \
--enable-unique_id=static --enable-so
make
make install

With the previous configure line we are removing modules that either disclose too much information or we do not need (wach out! you may need some of them). All inluded modules are statically linked to the binary. The only dynamic modules that we will be using are the mod_php and mod_security.

• –disable-info, –disable-status: we don’t need server info or status at all.
• –disable-autoindex, –disable-userdir: no automatic directory listings, no username enumeration through the /~ technique.
• –enable-dir: redirect malformed urls (requests to directories without trailing slash) and the DirectoryIndex directive.
• –disable-include, –disable-imagemap : no server side includes or image maps handled by the server.
• –disable-cgid, –disable-cgi : no cgi interfaces.
• –disable-proxy, –enable-ssl, –enable-rewrite: disable the proxy capanility, enable SSL and the rewrite engine.
• –enable-unique_id: needed for mod_security (see below).
• –enable-so:

Configure apache
In apache2’s configuration file (/usr/local/apache2/conf/httpd.conf) append:

# server banner
ServerSignature  Off
ServerTokens  Prod
# disable TRACE requests
TraceEnable off

If needed, add the index.php as a default file to DirectoryIndex directive on Line 165:

<IfModule dir_module>
   DirectoryIndex index.php index.html
</IfModule>

In the same way, if you need virtual hosts enabled, uncomment the line 386 (or equivalent):

Include conf/extra/httpd-vhosts.conf

Add your options to that file. And if you need SSL support, uncomment the line 398 (or equivalent) of the same file:

Include conf/extra/httpd-ssl.conf

Change ownership of the htdocs and remove unnecessary files and folders:-

chown daemon.daemon /usr/local/apache2/htdocs/ -R
rm -rf /usr/local/apache2/htdocs/*
rm -rf /usr/local/apache2/cgi-bin/*
rm -rf /usr/local/apache2/icons

If you want your server to start at boot time, issue the following commands:-

rm /etc/init.d/apache2
ln -s /usr/local/apache2/bin/apachectl /etc/init.d/apache2
update-rc.d apache2 defaults

Be careful because if you have configured SSL with a certificate whose private key requires a pass phrase, the system will request the pass phrase and wait upon restart.

PHP
Not much on the PHP side. Download and compile:

cd /usr/local/src/php-5.2.3
./configure --with-apxs2=/usr/local/apache2/bin/apxs --with-mysql=/usr/include/mysql --with-config-file-path=/etc --with-gd --with-zlib-dir=/usr/lib/

make
make install

• –with-apxs2: compile a module for apache2 in the specified location.
• –with-mysql: .enable mysql support.
• –with-config-file-path: .specify where you want the php.ini config file.
• –with-gd: .the graphical library if you need it.
• –with-zlib-dir: .use system’s zlib (downloaded from packages).

Although the php installation adds the LoadModule line, but you still need to edit apache configuration file (httpd.conf) and add the following:

AddType application/x-httpd-php .php .phtml

Modify the DirectoryIndex directive if you want the server to default to index.php when a directory is requested.

mod_security
Download:

cd /usr/local/src/
wget http://www.modsecurity.org/download/modsecurity-apache_2.1.2.tar.gz
tar -xvvzf modsecurity-apache_2.1.2.tar.gz
cd modsecurity-apache_2.1.2/apache2/

Edit the Makefile to adjust the following lines (compile mod_security with Apache’s version of the pcre library):

top_dir      = /usr/local/apache2
INCLUDES = -I /usr/include/libxml2 -I /usr/local/src/httpd-2.2.4/srclib/pcre/

Compile and install:

make
make install

Copy the default rule set to apache directory and include them in the main apache configuration file:

cp -r /usr/local/src/modsecurity-apache_2.1.2/rules/ \
/usr/local/apache2/conf/modsecurity

In /usr/local/apache2/conf/httpd.conf add the following lines:

LoadModule security2_module modules/mod_security2.so
Include conf/modsecurity/*.conf

In order to enforce the rules (by default mod_security would simply log requests that matched the rules), go to each and single file and change the SecDefaultAction to:

SecDefaultAction "phase:2,log,deny,status:400"

The End: up and running
Last but not least do not forget to remove software that you no longer need! No compilers or development libraries should remain in the sever.

First software needed to compile Apache:

apt-get remove --purge binutils cpp cpp-4.1 gcc-4.1 \
libssp0 make gcc libc6-dev libc-dev \
linux-kernel-headers libssl-dev zlib1g-dev

And also the one needed for PHP:

apt-get remove --purge libxml2-dev libfreetype6-dev \
libgd2-noxpm-dev libjpeg62-dev libpng12-dev libgd2-dev \
libmysqlclient15-dev g++ g++-4.1 libstdc++6-4.1-dev

Remove all the sources that we have used:

rm -rf /usr/local/src/*

And of course:-

/usr/local/apache2/bin/apachectl start

References

• Apache 2.0 Hardening Guide

We are using a standard package based installation of lighttpd and ruby:

# apt-get install gcc

gcc is needed for compiling fcgi (fast cgi) for Ruby.

# apt-get install ruby ruby1.8-dev rubygems libfcgi-ruby1.8
# gem install rails
# gem install fcgi

And the web server:

apt-get install lighttpd

The Rails applications will be located under /var/www/lighttpd/rails/ directory and we will have two folders in /etc/lighttpd: railsapp-available and railsapp-enabled. The first will contain the list of available Rails applications, the second – surprisingly – a list of those that we want to enable. With this configuration we will be able to have lots of applications in the server but just enabling (or disabling) them as we need it. Lets prepare this structure:-

# mkdir /var/www/lighttpd/rails/
# ln -s /var/www/lighttpd/rails/ /etc/lighttpd/railsapp-available
# mkdir /etc/lighttpd/railsapp-enabled

The next step is to enable those applications we want to deploy. Say for instance you have app #1, app #2 and app #3 under /var/www/lighttpd/rails/ and right now we only want to deploy app #1 and app #3. We will need to issue the following commands:-

# cd /etc/lighttpd/railsapp-enabled/
# ln -s ../railsapp-available/app1
# ln -s ../railsapp-available/app3

Only applications linked inside railsapp-enabled will be deployed when the server is restarted. To add a new application create a link as shown above, to undeploy an application just remove the link.

No we need the server to be able to understand this settings and only deploy the applications we want. For doing it we will include a script in the main config file that creates the necessary bits of lighttpd configuration required for them to work. Add this line at the end of /etc/lighttpd/lighttpd.conf:-

include_shell "/usr/share/lighttpd/include-app-vhost.rb"

Download the source code for the script and place it in the above location. The code looks like this:-

#!/usr/bin/ruby
 $VIRTUAL_HOST=<<EOV
    $HTTP["host"] =~ "%RAILS_APP%($|\.nomejortu\.loc)" {
        server.document-root = "/var/www/lighttpd/rails/%RAILS_APP%/public/"
        server.error-handler-404 = "/dispatch.fcgi"
        server.indexfiles = ("dispatch.fcgi")
        fastcgi.server = (".fcgi" =>
           ("localhost" =>
                ("socket" => "/tmp/railsapp-%RAILS_APP%.socket",
                 "min-procs" => 2,
                 "max-procs" => 2,
                 "bin-path" => "/var/www/lighttpd/rails/%RAILS_APP%/public/dispatch.fcgi",
                 "bin-environment" => ("RAILS_ENV" => "production")
                )
           )
        )
    }
 EOV
 Dir["/etc/lighttpd/railsapp-enabled/*"].each do |railsapp|
   puts $VIRTUAL_HOST.gsub(/%RAILS_APP%/,  File.basename(railsapp))
 end

The main loop of the script goes trough the /etc/lighttpd/railsapp-enabled/ directory and for each file it finds there, creates a new block of configuration directives. It does this by replacing the %RAILS_APP% string in the $VIRTUAL_HOST variable with the current name of the application under that directory.

Let’s have a closer look to the generated configuration for each application:

$HTTP["host"] =~ "%RAILS_APP%($|\.nomejortu\.com)" {
  server.document-root = "/var/www/lighttpd/rails/%RAILS_APP%/public/"
  server.error-handler-404 = "/dispatch.fcgi"
  server.indexfiles = ("dispatch.fcgi")
  fastcgi.server = (".fcgi" =>
    ("localhost" =>
      ("socket" => "/tmp/railsapp-%RAILS_APP%.socket",
      "min-procs" => 2,
      "max-procs" => 2,
      "bin-path" => "/var/www/lighttpd/rails/%RAILS_APP%/public/dispatch.fcgi",
      "bin-environment" => ("RAILS_ENV" => "production")
      )
    )
  )
}

The first line matches the virtual hostname with the application, then we need to customize a few parameters: server.document-root, socket and bin-path that will point to the Rails directory structure of out application.

Don’t forget to make it executable:

# chmod +x /usr/share/lighttpd/include-app-vhost.rb

Remember that adding an application into /var/www/lighttpd/rails will not make it readily available. You still need to add the correct links in /etc/lighttpd/railsapp-enabled and restart lighttpd!

# cd /etc/lighttpd/railsapp-enabled
# ln -s ../railsapp-available/<app_name>
# /etc/init.d/lighttpd restart

After some research I found a set of scripts that actually do what I want (credit goes to Heiner Steven). The bad news is that this is not a full-bash solution. The scripts use the metasend command to send files as MIME atachments.

This is a easy two-step process. First, you need to install the metamail (this is the name of the Debian GNU/Linux package) in your box. Then grab this two scripts (sendfile, getmimetype). The first one does the call to metasend. From it’s usage information:

usage: sendfile [-f] [-s subject] [-m mimetype] recipient file ...
    -f:  force sending of mail even for invalid recipients
    -s:  subject of the mail message
    -m:  mime-type (i.e. "application/octet-stream")

Multiple files may be specified. If no mimetype was given,
it is determined via a call to "getmimetype".

And you are ready to go.

Let’s begin with the script. Then we can talk about the work it does.

Here is the code:

#!/usr/bin/ruby
##################################################################################################################
#
# runningserver.rb
# 12/DEC/2006
# etd [etd__at__nomejortu.com]
#
# Desc:
#   Script to create a connection on the specified port to check if a server is
#   listening on it.
#
# Version:
#   v1.0 [12/Dec/2006]: first released
#
###################################################################################################################
$help =< <EOH
#{$0}:
   Script to create a connection on the specified port to check if a server is
   listening on it.

Usage:
  #{$0} [-h|-help|--help] [-p <portnumber>] <host1> [<host2> <host3> ...]

Options:
  -h/-help/--help:  This help message
  -p <portnumber>:  Specify which TCP port should be tested.
Arguments:
  host(s):  The different hosts to test.
EOH

#------------------------------ Input argument parsing
if ( 
    ARGV.include?('--help')  || 
    ARGV.include?('-help') || 
    ARGV.include?('-h') || 
    (ARGV.size==0) 
  ) then
  puts
  puts $help
  exit
end

# set the port number
if ARGV.include?('-p') then
  p_position = ARGV.index('-p')
  # read value after the -p
  if ARGV[p_position+1] != nil then
    $port = ARGV[p_position+1].to_i
  else
    puts "ERROR: you kind of need to specify the port if you use the -p parameter"
    puts "\t e.g: -p 8080\n\n" 
    exit
  end
  # clear these values in the array
  ARGV[p_position] = nil
  ARGV[p_position+1] = nil
end

# clear the ARGV of already parsed ARGs
hostlist = ARGV.compact

#------------------------------ Interesting stuff starts here
require 'socket'
require 'net/http'
require 'timeout'

if $port == nil then
  $port=18264
end
#for each host in the command line
hostlist.each() do |host|
  puts
  serverstr = "No server was found in port #{$port}"
  begin
    Timeout::timeout(3) do
      client = TCPSocket.open(host, $port)
      client.close
      serverstr = "Server is running on port #{$port}"
    end
  rescue Exception => e
    serverstr = "No server was found in port #{$port}"
  end
  puts "#{host}: #{serverstr}"
end
puts

Because this is the first ruby script in this blog I will explain it step by step. The first thing that comes in the script is the input argument parsing. Input arguments are passed as a string array, in the variable ARGV. To check if the user is requesting help or usage information we can issue:

if ( 
    ARGV.include?('--help')  || 
    ARGV.include?('-help') || 
    ARGV.include?('-h') || 
    (ARGV.size==0) 
  ) then
  puts
  puts $help
  exit
end

In the same way the script checks if the user has supplied a port number. If this is not the case, we use a default port. If the input argument -p is present, we asume the next argument (p_position+1) to be the TCP port number. It is important to get the port number as an interger. We accomplish this by issuing the .to_i call.

# clear these values in the array
  ARGV[p_position] = nil
  ARGV[p_position+1] = nil

With the code above we clear the elements we are not using any more. Doing so we can compact the ARGV array in order to remove all the nil elements of it just by calling the .compact function of the array.

The body of the script is pretty straight forward. For each host given as input to the script we try to open a socket on the specified port. If successful, we close the socket and that’s it. To avoid hanging for ever on closed ports we are using timeout library (require 'timeout'). The timeout mechanism is very simple. You create a block and give the timeout in seconds (can be a float point value). If the block terminates before the timeout, timeout returns the value of the block. Otherwise, the exception (Timeout::Error) is raised.

begin
    Timeout::timeout(3) do
      #code
    end
  rescue Exception => e
    #exception handling
  end

You see? It wasn’t that difficult, was it?