SecondForge

Since v2.0, Rails has shipped with token-based protection against cross-site request forgery (CSRF) attacks. You may have run into this issue while trying to create and test a form that does not rely on Rails’ built in form_for function. If you create a form in pure HTML and submit it to a Rails controller action, by default, it will throw the warning:

WARNING: Can't verify CSRF token authenticity

What does this mean, and how is it fixed?

The quick fix: include a hidden parameter in your form with the name authenticity_token and value that is the output of the form_authenticity_token helper method, like so:

<input type="hidden" name="authenticity_token" value="form_authenticity_token">

This will put a stop to the warnings and protect your application from CSRF attacks. But what is going on here?

A cross-site request forgery attack refers to an attacker using the authentication information of a trusted user in a request. For example, an attacker could steal a cookie containing the authentication information of a user, and then send that cookie to the website with a forged request. In your Rails app, this could take the form of an attacker using a malicious script and a trusted user’s cookies to POST malicious data to some action. The attack might not break any of your application’s functionality - it could be supported but undesired behavior, e.g. making a withdrawal from a bank account.

Rails protects against CSRF attacks by sending a CSRF token with each request. A correct CSRF token must be provided with each request. In fact, if you are using the popular Devise gem, a user who submits a form with an invalid CSRF token, or without a CSRF token, will be logged out.

We can see how this works by looking at the relevant code: the relevant code is in ActionController::RequestForgeryProtection. The definition of the form_authenticity_token method is simple:

  # Sets the token value for the current session.
  def form_authenticity_token
    session[:_csrf_token] ||= SecureRandom.base64(32)
  end

As we can see, Rails is simply generating a random 32-byte number to serve as the CSRF token, once per session. The method that does the actual testing of CSRF tokens is verified_request?:

  # Returns true or false if a request is verified. Checks:
  #
  # * is it a GET or HEAD request?  Gets should be safe and idempotent
  # * Does the form_authenticity_token match the given token value from the params?
  # * Does the X-CSRF-Token header match the form_authenticity_token
  def verified_request?
    !protect_against_forgery? || request.get? || request.head? ||
      form_authenticity_token == form_authenticity_param ||
      form_authenticity_token == request.headers['X-CSRF-Token']
  end

All this method does is check if the CSRF token provided by the request matches the CSRF token stored in memory. This provides a good measure of protection against CSRF attacks, as it requires the attacker to not just intercept a cookie, but also intercept a page from the current session in order to effectively forge a request.

Inspecting verified_request? can teach us about the limitations of token based CSRF protection. Note that verified_request? will always return true for GET and HEAD requests. As the comments note, ‘Gets should be safe an idempotent’. It is essential that Ruby on Rails developers respect the intended meanings of the HTTP verbs. Ignoring the documented uses of the HTTP verbs can introduce security flaws into your application.

Finally, note that Rails uses the same CSRF token for an entire session. Exercise care when sending forms via AJAX - if an attacker is able to intercept a form from the current session, they will have a CSRF token that will work on any form for the user across your application.

Hopefully you now have a better understanding of how CSRF token verification works and just what Rails is doing under the hood.

Yesterday I launched JCGaaS - James C Gibson as a Service. The website provides a simple JSON API that is backed by me. Tasks or requests that are submitted are performed and fulfilled by yours truly, and billed by the minute. It is human capital packaged as a service.

The past hundred years have been about commoditizing increasingly complex work and packaging it in more useful ways. Purely rote tasks were automated long ago, and packaged appropriately as a result; no human has been paid to add numbers by hand in a century. And even more advanced tasks that are easy for humans but hard for computers have been packaged as services and forced to work in a way that is convenient for computers.

Amazon Mechanical Turk is a prime example. Mechanical Turk does an amazing job of commoditizing the most basic human skills, like distinguishing images or providing basic transcription. These activities are easy for any human to do but hard for a computer to do. It effectively commoditizes and packages labor as a service.

But it is hard to get a mechanical turk worker with a college degree in a given field (for whatever that is worth) or that you would trust to make ethical judgements, and impossible to do so consistently.

Thus JCGaaS is an experiment in commoditizing and packaging human capital and expertise - not just labor - as a service. The world is moving to shorter-term employment and freelancing; billing by the minute is the next logical step. And the only effective way to bill by the minute is to replace contract negotiation with a three-column pricing spread and mediate interactions through a web service’s API.

I do not yet know what JCGaaS will be used for, or if it will be used at all. But I am looking forward to seeing what some clever folks can do with it, and actively looking for feedback and ideas on how to develop the service.

(Creating JCGaaS was also an exercise in writing applications and challenging my own assumptions about web development, but I’ll save that for a future post.)

I deploy websites frequently enough that I really should understand DNS thoroughly, but infrequently enough that it is hard to remember all the details between deployments. This blog post is as much for my future reference as anything else.

For my most recent project, I decided to use Amazon Route 53 for DNS, mostly for the experience of it. Route 53 offers a lot of great features, and is particularly easy to use when integrating with other Amazon Web Services products (EC2, S3), but is not quite as user friendly as other DNS products.

The setup I want is simple: the core domain (jcgaas.com), both with and without the common but unnecessarywww. subdomain, to map to the same static webpage hosted by Amazon S3. Additionally, I need an api. subdomain to point at a server I have hosted with an IPv4 address.

There are four types of records required to make this setup work. A fifth is needed to get IPv6 support. Two of them, the SOA “Start of Authority” and NS “Name Server” record, are set by Route 53 automatically.

The other two are ‘A’ records and CNAMEs.

The A record maps a domain or a subdomain to an IPv4 address. So, for this setup, I need 2 ‘A’ records; one mapping the domain to the Amazon S3 endpoint (which Route 53 conveniently does for you as an ‘Alias’ record), and one mapping the api. subdomain to the API server’s IP address.

An ‘AAAA’ record is for an IPv6 address, an additional AAAA record could be defined to map the api. subdomain to the appropriate IPv6 address.

The CNAME (“canonical name”) record aliases a domain to another domain. In this case, I use it to make the www. subdomain an alias for the core, subdomain-less domain.

Many DNS systems offer a built-in way to redirect the user to another domain, for example making www. send a 301 Moved Permanently HTTP status code referring to the unadorned domain. Apparently, this is not a part of the DNS spec, and Amazon Route 53 does not implement such functionality directly. For Route 53, the way to do this is to make the CNAME for www. point to an Amazon S3 bucket for www.<domain>, then set the S3 bucket to redirect all requests to the unadorned domain. The net result is the same as other DNS system’s redirect functionality, just requires an extra step to set up.

Email - or really, the Simple Mail Transfer Protocol has shortcomings. SMTP is great because it is simple. RFC 821, the original specification, is only 68 pages long, short for a technical RFC. It is relatively easy to implement a minimal SMTP client or server.

One of the features that one might want out of their email library is the ability to verify that an address exists. Perhaps when a user signs up, we could detect that they made a mistake entering their email address with AJAX, alerting them to a problem before they submit the form. Or we could prune old email addresses from our mailing list automatically by periodically checking that the email addresses exist.

It turns out that RFC 821 actually has a VRFY command which allows the client to check if an address exists. However, because it could be used by spammers to prune their lists - and hence, make their activities harder to detect - RFC 2505 specifically suggests limiting VRFY to authenticated / trusted users. As such, no public mail server is going to allow an arbitrary client to use VRFY today, if they turn on VRFY support at all.

Some people have come up with clever ways of attempting to verify addresses in other ways. The most common proposed way of doing this is to start an SMTP transaction, and treat an error after sending the proposed recipient address as proof that the address does not exist. Al Iverson has laid out why this is a bad idea, though, which amounts to: it does not work (many servers have a ‘catchall’ address) and it will get your server on spam blacklists.

So, there is no way to verify an email address without sending a message, much less a good way to do so. The best you can do is ask your user to click a link in an email that you have sent them - but even this has a significant cost in that it inconveniences your user. The best solution is to design your application in such a way that no email address verification is required.

One of the under-appreciated features of the ruby programming language is the excellent built-in array class. Arrays are one of the most common data structures, but arrays in ruby come with some neat benefits.

Since ruby has dynamic, weak types, ruby arrays are most easily thought of as a simple list of objects. Each element of the array is addressed by an index, which starts counting from 0. The default way of accessing the element at an index is with the [] method:

array = [1, 2, 3]
array[0] #=> 1
array[1] #=> 2
array[2] #=> 3

This is all standard programming fare. By default, ruby arrays return nil for nonexistent indices, instead of throwing an exception:

array[5] #=> nil

If you do not want to have nil returned, make use of the often-overlooked fetch method. Fetch lets you pass a value to be returned if the index is out of bounds, or pass a block that will be executed in that case:

array.fetch(5, 100) #=> 100
array.fetch(5) {|index| index\*100} #=> 500

For more fun, if you need to work with an array in reference to its last element, instead of its first element, you can just pass a negative index:

array[-1] #=> 3
array[-2] #=> 2
array[-3] #=> 1

This really useful: if you want the last character of a string in ruby, you can use a negative index like so:

hello = "Hello!"
hello[-1] #=> "!"

This is much more concise than the easiest way of doing the same thing in many other languages, for example Java:

String[] hello = {"H", "e", "l", "l", "o", "!"};
hello[hello.length() - 1]; //=> "!"

You can even use negative array indices with the useful ruby array range selection. The selection range still has to read ‘left to right’, so the lesser negative number comes first. You cannot wrap your selection around the zero.

array[1..2] #=> [2, 3]
array[-1..-2] #=> [] #incorrect
array[-2..-1] #=> [2,3] #correct
array[-1..1] #=> [] #probably not what you wanted

Of these methods, using the ruby array’s [-1] and .fetch are probably the most useful.