#301 Extracting a Ruby Gem pro

We’ve shown how to create a Ruby gem before, most recently in episode 245, but this time we’re going to show you the whole process involved in making a gem across two episodes. In this episode we’ll show you how to extract functionality from an existing application into a gem and test it while later we’ll show you how to publish a gem and share it with the world.

The format_url Method

The code we want to extract is in a Comment model in a blogging applicaition.

def self.format_url(url)
    if url.to_s !~ url_regexp && "http://#{url}" =~ url_regexp
      "http://#{url}"
    else
      url
    end
  end

  def self.url_regexp
    /^https?:\/\/([^\s:@]+:[^\s:@]*@)?[-[[:alnum:]]]+(\.[-[[:alnum:]]]+)+\.?(:\d{1,5})?([\/?]\S*)?$/iux
  end

  before_validation do
    self.website = self.class.format_url(website)
  end

  validates_format_of :website, with: url_regexp, message: "is not a valid URL"

This code handles the formatting and validation of URLs entered by the user when they add a comment to a post. If the validation fails an error message is shown.

When the user enters a valid URL their comment is added. A valid URL doesn’t need to include the http:// at the start, as long as it is otherwise valid this will be added before its saved to the database.

We want to extract this code out into a gem so that we can use it in other applications. It’s always a good idea to extract a gem from an existing application as this way we have a good idea of the requirements and of the level of abstraction to use. Otherwise this can be a bit of a guessing game. Remember that Rails itself was initially extracted from an existing application.

Creating a New Gem

We’ll be using Bundler to create our gem, as we did in episode 245. To create a new gem we use the bundle gem command.

$ bundle gem url_formatter
      create  url_formatter/Gemfile
      create  url_formatter/Rakefile
      create  url_formatter/.gitignore
      create  url_formatter/url_formatter.gemspec
      create  url_formatter/lib/url_formatter.rb
      create  url_formatter/lib/url_formatter/version.rb
Initializating git repo in /Users/eifion/url_formatter

The first thing we should do is open the generated gemspec file and fill in the homepage, summary and description properties.

# -*- encoding: utf-8 -*-
$:.push File.expand_path("../lib", __FILE__)
require "url_formatter/version"

Gem::Specification.new do |s|
  s.name        = "url_formatter"
  s.version     = UrlFormatter::VERSION
  s.authors     = ["Eifion Bedford"]
  s.email       = ["eifion@asciicasts.com"]
  s.homepage    = "http://github.com/ryanb/url_formatter"
  s.summary     = %q{Format and validate a URL in Active Record}
  s.description = %q{Example of creating a Ruby gem for ASCIIcast #301}

  s.rubyforge_project = "url_formatter"

  s.files         = `git ls-files`.split("\n")
  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
  s.require_paths = ["lib"]

  # specify any dependencies here; for example:
  # s.add_development_dependency "rspec"
  # s.add_runtime_dependency "rest-client"
end

Bundler keeps the version number in a separate file and it’s a good idea to append alpha to the version number while working on a specific version of a gem so that everyone knows that this version is still under development. When we’re ready to release that version we can remove this suffix or mark it as a beta or release candidate version.

module UrlFormatter
  VERSION = "0.0.1.alpha"
end

We shouldn’t be afraid to release version 1.0 once we feel that our application is production-ready and has a stable API. There’s a great article that’s well worth reading on this subject called Semantic Versioning. You might expect us to start coding the gem now but instead we’re going to write the README. This approach is known as Readme-Driven Development.

# URL Formatter

Format and validate a URL attribute in Active Record. This is an example gem created for [RailsCasts episode #301](http://railscasts.com/episodes/301-extracting-a-ruby-gem).


## Installation

Add to your Gemfile and run the `bundle` command to install it.

 ```ruby
 gem "url_formatter"
 ```

**Requires Ruby 1.9.2 or later.**


## Usage

Call `format_url` in an ActiveRecord class and pass the name of the attribute you wish to format into a URL and validate.

 ```ruby
 class Comment < ActiveRecord::Base
   format_url :website
 end
 ```

This will automatically add "http://" to the beginning of the `website` attribute upon saving if no protocol is present. It will also do validation to ensure it looks like a URL.


## Development

Questions or problems? Please post them on the [issue tracker](https://github.com/ryanb/url_formatter/issues). You can contribute changes by forking the project and submitting a pull request. You can ensure the tests passing by running `bundle` and `rake`.

This gem is created by Ryan Bates and is under the MIT License.

We now have a clear idea as to what the interface will look like to the user. You can see from the README that we want the gem to be a simple install through Bundler and that we want a simple format_url method that the user can add to an ActiveRecord model to format and validate a URL. We’ll also need a licence file so that users know what they can do with our gem. We’ll use the MIT licence.

Next we’ll set up RSpec so that we can use it for testing. We’ll need to define RSpec as a development dependency in the gemspec file. There’s already a line of code in this file that we can just uncomment.

s.add_development_dependency "rspec"

To install RSpec we’ll need to run bundle again.

We want RSpec to default to displaying its output in colour. To do this we can create a .rspec file containing the default options we want.

$ echo "--color" > .rspec

While we’re here we’ll also make a spec/url_formatter directory and a spec/spec_helper.rb file.

$ mkdir -p spec/url_formatter
$ touch spec/spec_helper.rb

Inside our new spec_helper file we’ll just load our url_formatter class for now.

require 'url_formatter'

We’ll also add a few lines to the Rakefile so that we can run RSpec through Rake. This will load up the RSpec Rake task and make it the default task.

require "bundler/gem_tasks"
require 'rspec/core/rake_task'

RSpec::Core::RakeTask.new(:spec)
task default: :spec

Running rake now will run RSpec but as we don’t have any spec files we’ll just get an error.

$ rake
No examples matching ./spec{,/*/**}/*_spec.rb could be found

Now we can write our first spec. If we look at the Comment model back in our Rails app we’ll see that most of the logic is in two class methods called format_url and url_regexp. These are utility methods that we could put into any module in our gem. We already have some specs to test these methods so we’ll move these into a new url_formatter_spec file in our gem before we move the code itself. Two of the specs in the application are specific to ActiveRecord and how it handles saving and validation so we’ll leave these out for now.

# encoding: utf-8
require 'spec_helper'
describe UrlFormatter do
  describe ".format_url" do
    it "adds http:// to a URL if not provided" do
      UrlFormatter.format_url("example.com").should eq("http://example.com")
    end

    it "does not add http:// to a URL if already provided" do
      UrlFormatter.format_url("http://example.com").should eq("http://example.com")
    end

    it "returns an invalid URL unchanged" do
      UrlFormatter.format_url("foo bar").should eq("foo bar")
      UrlFormatter.format_url(nil).should eq(nil)
    end
  end

  describe ".url_regexp" do
    it "matches valid URLs" do
      [
        'http://example.com/',
        'HTTP://E-XAMLE.COM',
        'https://example.co.uk./foo',
        'http://example.com:8080',
        'http://www.example.com/anything/after?slash',
        'http://www.example.com?anything_after=question',
        'http://user123:sEcr3t@example.com',
        'http://user123:@example.com',
        'http://example.com/~user',
        'http://1.2.3.4:8080',
        'http://ütf8.com',
      ].each do |url|
        url.should match(UrlFormatter.url_regexp)
      end
    end

    it "does not match invalid URLs" do
      [
        "www.example.com",
        "http://",
        "http://example..com",
        "http://e xample.com",
        "http://example.com/foo bar",
        "http://example", # technically valid but not what we want from user
        "other://example.com", # we also don't want other protocols
      ].each do |url|
        url.should_not match(UrlFormatter.url_regexp)
      end
    end
  end
end

When we run rake we’ll see five failing specs as format_url and url_regexp aren’t defined in our module. To fix this we just need to copy the code over from our application into our gem.

require "url_formatter/version"

module UrlFormatter
  def self.format_url(url)
    if url.to_s !~ url_regexp && "http://#{url}" =~ url_regexp
      "http://#{url}"
    else
      url
    end
  end

  def self.url_regexp
    /^https?:\/\/([^\s:@]+:[^\s:@]*@)?[-[[:alnum:]]]+(\.[-[[:alnum:]]]+)+\.?(:\d{1,5})?([\/?]\S*)?$/iux
  end
end

When we run our specs now, they all pass.

$ rake
/Users/eifion/.rvm/rubies/ruby-1.9.2-p290/bin/ruby -S rspec ./spec/url_formatter_spec.rb
.....

Finished in 0.00153 seconds
5 examples, 0 failures

Adding Validators To The Gem

Now that we’ve moved the URL-related code into our new gem we can remove it from our Rails application’s Comment model.

class Comment < ActiveRecord::Base
  belongs_to :article

  before_validation do
    self.website = self.class.format_url(website)
  end

  validates_format_of :website, with: url_regexp, message: "is not a valid URL"
end

We want our gem to work so that adding the format_url method to a model will automatically generate the callback and formatting validators for the attribute we specify so we still need to move the before_validation callback and the format validator. This code is ActiveRecord-specific so we’ll need to put in in a different location in our gem.

As we’re extending ActiveRecord we’ll put these methods in a module called ModelAdditions so we start by creating a model_additions_spec file. We’ve already have two specs to test the validators in our application so we’ll copy them here and change the class in the describe method from Comment to UrlFormatter::ModelAdditions.

require 'spec_helper'

describe UrlFormatter::ModelAdditions do
  it "adds http:// to URL upon saving" do
    Comment.create!(website: "example.com").website.should eq("http://example.com")
    Comment.create!(website: "http://example.com").website.should eq("http://example.com")
  end

  it "validates URL format" do
    comment = Comment.new(website: "foo bar")
    comment.should_not be_valid
    comment.errors[:website].should eq(["is not a valid URL"])
    comment.website = "example.com"
    comment.should be_valid
  end
end

There’s a problem here, however. These specs rely on a Comment model that we don’t have access to in our gem as we don’t load ActiveRecord there. There are a couple of nice solutions that will handle this situation for us, one of which is a gem called Supermodel. This gem is great if you need to test the interaction between a gem and ActiveRecord. It uses ActiveModel internally with an memory-based store and simulates ActiveRecord without us having to set up a database. If we do need to use ActiveRecord with a database in our gem’s specs we can use a different gem called WithModel, but Supermodel will work perfectly well for us here as our gem only needs to interact with validations and callbacks.

To add Supermodel to a gem we need to add it as a development dependency in the gemspec file and then run bundle to install it.

  # specify any dependencies here; for example:
  s.add_development_dependency "rspec"
  s.add_runtime_dependency "supermodel"

Next we’ll modify the spec_helper file and require Supermodel so that it’s loaded.

require 'url_formatter'
require 'supermodel'

We can now use Supermodel to simulate the ActiveRecord Comment model in the specs. To use it we need to create a Comment class that inherits from SuperModel::Base.

require 'spec_helper'

class Comment < SuperModel::Base
  extend UrlFormatter::ModelAdditions
  format_url :website
end

describe UrlFormatter::ModelAdditions do
  # specs omitted
end

To truly simulate what we’re doing in the Rails application we’ve called format_url on the website attribute in the Comment class here. This method is something we’re still yet to write, but we’ll do so in the ModelAdditions module. This model isn’t available in our Comment class and so we’ve had to extend the class with this module.

When we run rake now we get an error message that says that RSpec doesn’t know about our ModelAdditions module. This isn’t surprising as we haven’t written it yet, so lets do that now.

module UrlFormatter
  module ModelAdditions
    def format_url(attribute)
      
    end
  end
end

The validations and formatting will be handled inside the format_url method but let’s try what we’ve got for now.

Files aren’t autoloaded inside a Ruby gem so it’s up to us to include them inside the url_formatter base file.

require "url_formatter/version"
require "url_formatter/model_additions"

When we run our specs now we get two failures, one related to validation and one related to formatting. This is to be expected given that we haven’t written the format_url method yet. We can copy the code from the same method in our Rails application and modify it a little to suit.

module UrlFormatter
  module ModelAdditions
    def format_url(attribute)
      before_validation do
        send("#{attribute}=", UrlFormatter.format_url(send(attribute)))
      end
      validates_format_of attribute, with: UrlFormatter.url_regexp, message: "is not a valid URL"
    end
  end
end

When this code was in our Comment class we set the website attribute directly but here we need to make this dynamic based on the attribute that’s passed in. To do that we call send wherever we called website. To get the formatted value we need to call format_url on the UrlFormatter module. Similarly in the format validator we’ve replaced :website with attribute and have called url_regexp through UrlFormatter.

When we run our specs now they still fail with an error about an undefined before_validation method. This is a Supermodel error as it doesn’t completely simulate ActiveRecord or ActiveModel and seems not to support this callback. Fortunately this is easy to fix. We just have to modify the Comment class in the model_additions_spec file to include ActiveRecord’s Callbacks module.

class Comment < SuperModel::Base
  include ActiveModel::Validations::Callbacks
  extend UrlFormatter::ModelAdditions
  format_url :website
end

This time when we run our specs they all pass. Our gem is now functionally complete and behaves like the equivalent code in our Rails application, but there’s still one more thing we need to add to our gem to get it working.

Extending ActiveRecord

In our Rails application we want to be able to call format_url in any model without also having to manually include UrlFormatter::ModelAdditions. To be able to do this we’re going to have to include the ModelAdditions module in ActiveRecord::Base so that we don’t have to do it every time we want to use format_url.

Whenever we want to load something into Rails like this we should use a Railtie. There’s an excellent article on this topic on EngineYard’s blog. A Railtie allows us to add an initializer call that adds behaviour to a Rails application and we can use this to add our ModelAdditions to ActiveRecord.

Our Railtie goes in the /lib/url_formatter directory.

module UrlFormatter
  class Railtie < Rails::Railtie
    initializer 'url_formatter.model_additions' do
      ActiveSupport.on_load :active_record do
        extend ModelAdditions
      end
    end
  end
end

Our class has to inherit from Rails::Railtie and needs to call initializer. The initializer needs a unique name and its block is called when the Rails application starts up. We add functionality to ActiveRecord through ActiveSupport by calling ActiveSupport.on_load and the code inside the on_load block is executed in the scope of ActiveRecord::Base. By calling extend ModelAdditions in here we extend the module directly into ActiveRecord.

This is all we need to do in our Railtie but the file won’t be included automatically so we need to require it in our url_formatter file but only if Rails is defined. This way it isn’t always included which is important when we run our specs.

require "url_formatter/version"
require "url_formatter/model_additions"
require "url_formatter/railtie" if defined? Rails

This brings up a point: we haven’t tested the code in the railtie file. It’s difficult to do a high-level integration test through an entire Rails app in a gem. It’s better to do these tests outside the gem in a separate Rails application.

Testing Our Gem in a Rails Application

Before we release our gem it’s a good idea to test it in a Rails application and we’ll test it in the app we extracted the gem’s code from. As with any gem we start by adding it to the application’s Gemfile but as our gem hasn’t been published yet we need to use the path option to specify the path on our local hard drive where the gem is located. As ever we’ll need to run bundle to make sure that the gem is included in our application.

gem 'url_formatter', path: '~/url_formatter'

Our application’s code is almost set up to use our new gem already. The Comment model already uses format_url, but we’ll need to change Comment’s specs a little as most of the specs there are now duplicated in our gem.

# encoding: utf-8
require 'spec_helper'

describe Comment do
  it "adds http:// to URL upon saving" do
    Comment.create!(website: "example.com").website.should eq("http://example.com")
    Comment.create!(website: "http://example.com").website.should eq("http://example.com")
  end

  it "validates URL format" do
    comment = Comment.new(website: "foo bar")
    comment.should_not be_valid
    comment.errors[:website].should eq(["is not a valid URL"])
    comment.website = "example.com"
    comment.should be_valid
  end
end

We’ll leave the other two specs in place so that we can check that the gem integrates correctly into our Rails application.

Our application’s specs pass when we run them so we’ll start up our server and test the application in a browser. When we enter a comment with a bad URL now we see the same error message we saw before we moved the code into a gem so it seems that everything is working correctly.

That’s it for this episode We’ve successfully moved some of our application’s logic out into a Ruby gem but we haven’t published our gem and share it with the world. This will be covered in the next pro episode.