#365 Thread-Safety pro

A few weeks ago Aaron Patterson, a.k.a. Tenderlove, wrote a blog post about the threadsafe option in Rails and how it is likely to be enabled by default in production mode in Rails 4.0. In this episode we’ll explain what this option does and how it might affect deployment and development when it’s enabled. To demonstrate this we’ll create a new Rails application called thready.

$ rails new thready
$ cd thready

If we look at this application’s production config file we’ll see that this option is commented-out; in a new Rails 4 application this probably won’t be the case.

# Enable threaded mode
# config.threadsafe!

It’s important to understand that enabling this option doesn’t mean that our application magically becomes multi-threaded, so what does this setting do? If we look at the source code for the threadsafe! method we’ll see that it just sets four other options.

def threadsafe!
  @preload_frameworks = true
  @cache_classes      = true
  @dependency_loading = false
  @allow_concurrency  = true
  self
end

The first three of these have to do with how the application loads, basically telling it to do eager loading so that it preloads the entire Rails application when it starts up instead of relying on autoloading behaviour which is not thread-safe. The fourth option, allow_concurrency, tells Rails to not use the Rack::Lock middleware. If we run rake middleware in development we’ll see that Rack::Lock is one of the first pieces of middleware that a request encounters.

$ rake middleware
use ActionDispatch::Static
use Rack::Lock
# Rest of stack omitted.

If we enable the threadsafe option in production mode then run rake middleware RAILS_ENV=production we won’t see this middleware listed. So what does Rack::Lock do? Let’s take a look at the source code for it.

require 'thread'
require 'rack/body_proxy'

module Rack
  class Lock
    FLAG = 'rack.multithread'.freeze

    def initialize(app, mutex = Mutex.new)
      @app, @mutex = app, mutex
    end

    def call(env)
      old, env[FLAG] = env[FLAG], false
      @mutex.lock
      response = @app.call(env)
      response[2] = BodyProxy.new(response[2]) { @mutex.unlock }
      response
    rescue Exception
      @mutex.unlock
      raise
    ensure
      env[FLAG] = old
    end
  end
end

When a new request comes in the call method is run and this calls lock on a mutex object before handling the request then unlocking that mutex. This ensures that only one request is processed at a time. We’ll test this behaviour to better understand what’s going on and we’ll start by generating a new controller with a single action so that we can experiment with it.

$ rails g controller foo bar

In our new action we’ll sleep the current thread for a second then render some text.

class FooController < ApplicationController
  def bar
    sleep 1
    render text: "foobar\n"
  end
end

We’ll start up the server for this application in development mode (note that this is the WEBrick server). In a separate tab we’ll make a curl request to this action. As expected there’ll be a pause of a second or so before we see the response.

$ curl http://localhost:3000/foo/bar
foobar

Next we’ll run the same command five times and use an ampersand so that it forks off the process each time so that all the requests are made asynchronously. (To do this OS X you’ll need to run this under ZSH rather than bash.)

% repeat 5 (curl http://localhost:3000/foo/bar &)
% foobar
foobar
foobar
foobar
foobar

You can’t see this here but each response is processed separately so this command will take just over five seconds to run, even through the requests were all made at the same time. Next we’ll close the Rails server down then restart it in production mode and then run the curl command again.

% rails s -e production
% repeat 5 (curl http://localhost:3000/foo/bar &)

As we have the threadsafe! configuration option enabled the response come back without the one second pause between them. The Rack::Lock middleware isn’t included in the middleware stack in production mode so now the requests are processed asynchronously.

If this option allows requests to come in concurrently do we now have to start writing thread-safe code? Not necessarily, this depends entirely on our production setup. Most of the popular Rails servers today, including Unicorn or Phusion Passenger, will only pass one request at a time to each worker process. This means that even with this option enabled requests will still be handled separately. We can try this out by enabling Unicorn which we do by uncommenting it in our application’s gemfile then running bundle to install it.

# Use unicorn as the app server
gem 'unicorn'

We can now use the unicorn command to start up our Rails application in production mode.

$ unicorn -E production -p 3000

If we run our curl command again now each request is processed individually as this is how Unicorn works when it only has one worker. In a way the Web server takes on the role of the mutex lock here and if we did have Rack::Lock in place this would be an unnecessary overhead when used with Unicorn. This is why the threadsafe! option will be enabled by default in production: it moves the decision to process requests concurrently to our production environment. Don’t forget, though, that this config option does more than just remove the Mutex lock, it also adds eager loading behaviour which could cause some unexpected side-effects depending on our app. This means that if you plan to use this option in your apps you should test it thoroughly in a staging environment first. Note also that there is talk about renaming this option in Rails 4 to better clarify what it does.

We know that Unicorn and Passenger don’t support multiple threads, but what if we do want to run our application in a multi-threaded environment? One option is to use the Puma server. This is a Web server based on Mongrel that can run any Rack application concurrently. It supports JRuby, Rubinius and even MRI so we’ll try running our application on it. We’ll need to uncomment Unicorn in the gemfile, add Puma and then run bundle again.

# Use unicorn as the app server
# gem 'unicorn'
gem 'puma'

We can now start up the server in the production environment.

$ rails s puma -e production

When we run our curl command now the results come back almost instantly as the requests are processed concurrently through threads thanks to Puma.

Puma is a little constrained under MRI due to MRI’s Global Interpreter Lock. We might still need to start up multiple processes running Puma to take full advantage of the CPU, especially if have a CPU with multiple cores. JRuby and Rubinius have better thread support so Puma should perform better under these environments.

If we are using a server that supports multiple threads we have to be careful to ensure that all code in our application is thread-safe. Here’s a slightly contrived example of unsafe code.

class FooController < ApplicationController
  @@counter = 0
  def bar
    counter = @@counter
    sleep 1
    counter += 1
    @@counter = counter
    render text: "#{@@counter}\n"
  end
end

In the controller we now have a class variable called @@counter and we increment this every time the bar action is called. In the action we read the variable’s value, then sleep for a second before incrementing and writing it back. To see how this works we’ll first start the server in development mode with the default single-threaded Rails server.

$ rails s

When we run the curl command to call the action four times we’ll see 1 2 3 4 printed out with a second’s delay between each one and each request is processed synchronously.

% repeat 4 (curl http://localhost:3000/foo/bar &)
% 1
2
3
4

We’ll stop the server now and start up Puma in production mode.

$ rails s puma -e production

When we run the curl command now we’ll see different output.

% repeat 4 (curl http://localhost:3000/foo/bar &)
% 1
1
1
1

The requests are now processed concurrently and so the last request is made before the first one has had time to increment the counter. This is why we need to be careful when we work with data in memory that’s shared between threads such as our @@counter variable. To get around this problem we can set up a mutex lock similar to how the Rack::Lock middleware works,

class FooController < ApplicationController
  @@counter = 0
  @@mutex = Mutex.new

  def bar
    @@mutex.synchronize do
      counter = @@counter
      sleep 1
      counter += 1
      @@counter = counter
    end
    render text: "#{@@counter}\n"
  end
end

To do this we create a new Mutex and whenever we read from or write to shared data we use the synchronize method to create a lock, wrapping that code in its block. When the block ends the mutex will be unlocked. If we restart the Puma server now and run the curl command again we’ll see that the counter is incremented correctly this time as the mutex lock waits for any other requests to complete before running the code in its block.

The things we should look out for in our Rails applications and wrap in a mutex lock are class variables like our counter, instance variables at the class level, global variables and constants. Constants can be changed in Ruby, although a warning is issued when we do so and don’t forget that a value of a constant can be mutable as well. Strings are mutable in Ruby so we should call freeze on make them immutable. We also need to remember that the code inside the class itself is also shared memory so we should avoid inserting methods into a class dynamically after the application has loaded.

Thankfully making a Rails application thread-safe may not be as big a job as it might seem. Most of the time we don’t need to share mutable data like this and if we are we doing this we should look for an alternative approach as there’s probably a better way to do what we’re trying to achieve. The most difficult part of this process is to ensure that the gems that our application uses, and their dependencies, are thread-safe. One way to do this is to check each gem’s README and see if it mentions and thread-safety issues. If not then it’s worth posting something on that gem’s issue tracker.

Something else to be aware of when working with a multi-threaded Rails application is the database connection pool. This determines how many database connections are available at any one time in a Rails process and by default this is set to five. To see this in effect we’ll comment-out our mutex block so that we can make many requests at once.

def bar
  #@@mutex.synchronize do
    counter = @@counter
    sleep 1
    counter += 1
    @@counter = counter
  #end
  render text: "#{@@counter}\n"
end

As before making four requests at once works without a problem but if we try to make, say, twelve requests at once we’ll see that four requests are made at a time as that’s the limit to the number of database connections we can make simultaneously. Even though this action doesn’t use the database Rails will still reserve a connection for the duration of each request and if one isn’t available it will wait until one is or time out if a connection doesn’t become available. This means that if a request takes a while to run we will start to get timeout errors as other requests fall foul of the timeout before a database connection is available. If we want to support more than five simultaneous requests at one time on a single process, this connection pool setting may become our application’s bottleneck. We can increase this value as long as we’re happy to have that many simultaneous database connections. If we increase this value to 15 then make twelve simultaneous requests they’ll all come back at once (and all with the counter variable having a value of 1 as we removed the mutex lock).

That’s it for our episode on the threadsafe! option. Keep in mind that the effect of enabling this option depends greatly on the web server setup we have in production. If we go with a multi-threaded setup we’ll need to be sure that our app is thread-safe and also the gems that it depends on.