#207 Syntax Highlighting

Sometimes you need to display code snippets in a Rails application and to make the code more readable you can use syntax highlighting. This is something I do for almost every episode on this site, but how is it done?

Zuweilen möchte man Code Schnipsel in einer Rails Applikation anzeigen und um es besser lesbar zu machen kann man Syntax Highlighting nutzen. Das ist etwas, dass ich für fast jede Episode auf dieser Seite mache, aber wie wird das gemacht?

Es exisitieren unterschiedliche Möglichkeiten, um Syntax Highlighting zu einer Rails Applikation hinzu zufügen. Die erste Wahl die Sie treffen müssen ist, ob Sie das syntaxs Highlighting auf dem Server erledigen wollen, oder ob Sie eine JavaScript Bibliothek verwenden möchte, um die Seite im Browser zu transformieren. Es gibt eine Anzahl von JavaScript Bibliotheken für das Client-seitige syntax Highlighting aber wir werden diese nicht betrachten, da die nicht Rails spezifisch sind. Anstatt dessen werden wir Ihnen drei Utilities für Syntax-Highlighting zeigen, die in der Rails Welt populär sind.

There are a number of different options available for adding syntax highlighting to Rails applications. The first choice you need to make is to decide whether you’re going to use Rails to perform the syntax highlighting on the server or use a JavaScript library to transform the page in the browser. There are a number of JavaScript libraries available for client-side syntax highlighting but we won’t be covering them here as they’re not Rails-specific. Instead we’ll show you three utilities for syntax-highlighting that are popular in the Rails world.

CodeRay

Als erstes, CodeRay. Das kann einfach als gem installiert werden und hat keine externen Abhängigkeiten und ist auch sehr schnell. Es ist, wie auch immer, etwas limitiert in den Möglichkeiten und den Sprachen die es unterstützt. Wenn es dass tut was Ihre Applikation benötigt, dann ist es eine großartige Lösung und wir werden uns das später in dieser Episode noch genauer anschauen.

First, CodeRay. This can be easily installed as a gem, has no external dependencies and is very fast. It is, however, somewhat limited in its features and the languages it supports. If it does what your application needs, thought, it’s a great solution and we’ll be looking at it more closely later in this episode.

Ultraviolet

Die nächste Möglichkeit ist Ultraviolet. Das ist ebenfalls ein gem welches aber einige externe Abhängigkeiten hat, so kann es ein wenig schwierig werden es zu installieren. Ultraviolet ist langsamer als CodeRay hat aber einen größeren Funktionsumfang. Es kann Textmate syntax Dateien und themes verwenden und ..

The next option is Ultraviolet. This is also a gem but it has a couple of external dependencies so it can be a little difficult to install. Ultraviolet is slower than CodeRay but has a much bigger feature set. It can use Textmate syntax files and themes and so will transform far more languages than CodeRay and is flexible in how the highlighted snippets are displayed.

If you choose Ultraviolet for syntax highlighting then it’s worth looking at Harsh. This is a Rails plugin that provides a nice helper method for displaying the highlighted code inside your view and some rake tasks for installing themes and managing them. The README that comes with it contains useful instructions for installing Ultraviolet.

Pygments

A third popular option is Pygments. This is different from CodeRay and Ultraviolent in that it’s a Python library, but as it provides a command-line utility we can call it from our Rails applications. Pygments is a full-featured library, is well supported and is used by a lot of sites. The downside is that it can be rather slow when you’re using in to parse large documents in your Rails applications so you might have to make use of caching in your app if you’re thinking of using Pygments.

To make using Pygments easier with Rails there’s a great plugin called highlight available. This provides a wrapper around Pygments and allows it to be called from a simple helper method inside your views. The notes for highlight recommend that any output generated by it is cached due to the speed issues mentioned earlier. To do that all we need to do is wrap any output in a cache block like this:

<% cache do %>
  <%= highlight(:ruby, 'class Test; end') -%>
<% end %>

Benchmarking

So now that we’ve taken a look at the three options we have available let see how they compare in terms of speed. An application could be rendering highlighted code quite a bit and so we need to consider whether to cache the output or not. To compare each highlighter we’re going to use the code below:

require 'rubygems'
require 'benchmark'
require 'coderay'
require 'uv'

path = __FILE__
content = File.read(__FILE__)

# run it once to initialize
CodeRay.scan("print 'hello'",  'ruby').div(:css => :class)
Uv.parse("print 'test'", 'xhtml', 'ruby', true, 'amy')

Benchmark.bm(11) do |b|
  b.report 'coderay' do
    50.times { CodeRay.scan(content, 'ruby').div(:css => :class) }
  end
  
  b.report('ultraviolet') do
    50.times { Uv.parse(content, 'xhtml', 'ruby', true, 'amy') }
  end
  
  b.report('pygments') do
    50.times { `pygmentize -f html "#{path}"` }
  end
end

The benchmark program takes the content of its own source code and parses it fifty times through each library. For Pygments it uses the pygmentize command. The results are shown below:

$ ruby benchmark.rb 
                 user     system      total        real
coderay      0.310000   0.010000   0.320000 (  0.312954)
ultraviolet  4.860000   0.020000   4.880000 (  4.886621)
pygments     0.010000   0.120000  12.430000 ( 12.643173)

The results show quite a difference in the performance of each library. CodeRay is obviously the fastest, taking around 0.3 seconds for fifty iterations. Ultraviolet took more than ten times as long to do the same while Pygments was slower still, taking over twelve seconds to complete the test. Given such differences in speed it is important to balance features and performance when choosing which library to use. If you make use of caching then speed will not be as important an issue.

Using CodeRay

For the rest of this episode we’ll show you how to use CodeRay in a Rails application. CodeRay is the easiest of the three libraries to set up and also the fastest so there’s no need to worry about caching.

To demonstrate CodeRay we’re going to use a simple blogging application that you might have seen in previous episodes. We’ll use it to add the ability to have syntax highlighting for code snippets in the articles.

When an author edits an article we want them to be able to add code snippets between code tags, like this:

<code lang="ruby">
puts "Hello, world!"
</code>

When a snippet like this appears in the article’s body it should be displayed with the appropriate syntax highlighting.

To do this we’ll first need to add a reference to the CodeRay gem in the /config/environment.rb file. We’re working with a Rails 2 application here. If it was written with Rails 3 then we would modify the Gemfile instead.

Rails::Initializer.run do |config|
  config.gem "coderay"
  config.gem "RedCloth"
end

We’ll be making use of Textile later so we’ve also added the RedCloth gem in here. There is a gem that combines both available but here we’ve chosen to add them separately.

Next we’ll update the ArticleController’s show view to use CodeRay to parse the text in the code tags and highlight it. To do this we can wrap the line of code that displays the article’s content in a new helper method called coderay like this:

<% title @article.title %>
<p class="author">from <%= @article.author %></p>
<%= coderay(@article.content) %>

The helper method will go into our application_helper file:

# Methods added to this helper will be available to all templates in the application.
module ApplicationHelper  
  def coderay(text)
    text.gsub(/\<code( lang="(.+?)")?\>(.+?)\<\/code\>/m) do
      CodeRay.scan($3, $2).div(:css => :class)
    end
  end
end

The method is fairly simple. It takes in some text that may contain sections that we want to be highlighted and calls gsub on it to find matching sections and replace them with whatever is returned inside the block. The regular expression searches for an opening code tag with an optional lang attribute, and then matches the text between the opening and closing code tags. Inside the block the CodeRay.scan method is called. This method takes some text and a language as arguments and so we pass it $3 which matches the text between the code tags and $2 which matches the content of the lang attribute. It then calls .div on that to return the output wrapped in a div element. The :css => :class option tells CodeRay what type of CSS styling to do. CodeRay has a number of different options you can use to control the structure and look of the output and there is more information about this in the documentation.

Finally we need to create a stylesheet to set the styles for each of CodeRay’s classes. To provide a theme that looks like Railscasts, Ryan Bates has created a stylesheet that you can download from Github. We’ll take a copy of this file and add it to our application’s /public/stylesheets directory.

.CodeRay {
  background-color: #232323;
  border: 1px solid black;
  font-family: 'Courier New', 'Terminal', monospace;
  color: #E6E0DB;
  padding: 3px 5px;
  overflow: auto;
  font-size: 12px;
  margin: 12px 0;
}
.CodeRay pre {
  margin: 0px;
  padding: 0px;
}
 
.CodeRay .an { color:#E7BE69 }                      /* html attribute */
.CodeRay .c  { color:#BC9358; font-style: italic; } /* comment */
.CodeRay .ch { color:#509E4F }                      /* escaped character */
.CodeRay .cl { color:#FFF }                         /* class */
.CodeRay .co { color:#FFF }                         /* constant */
.CodeRay .fl { color:#A4C260 }                      /* float */
.CodeRay .fu { color:#FFC56D }                      /* function */
.CodeRay .gv { color:#D0CFFE }                      /* global variable */
.CodeRay .i  { color:#A4C260 }                      /* integer */
.CodeRay .il { background:#151515 }                 /* inline code */
.CodeRay .iv { color:#D0CFFE }                      /* instance variable */
.CodeRay .pp { color:#E7BE69 }                      /* doctype */
.CodeRay .r  { color:#CB7832 }                      /* keyword */
.CodeRay .rx { color:#A4C260 }                      /* regex */
.CodeRay .s  { color:#A4C260 }                      /* string */
.CodeRay .sy { color:#6C9CBD }                      /* symbol */
.CodeRay .ta { color:#E7BE69 }                      /* html tag */
.CodeRay .pc { color:#6C9CBD }                      /* boolean */

There is a CSS class for each type of token in the highlighted source code and the colours above can easily be changed to whatever look you need for your application.

To get the stylesheet to work we need to add a reference to it in the head section of the application’s layout file.

<%= stylesheet_link_tag 'blog', 'coderay' %>

With all that done we can edit an article and when we look at it we’ll see the highlighted text.

Using Textile

Syntax highlighting is often used with a simple markup language such as Textile or Markdown. To use Textile in our article we can wrap the coderay method in the show view in a call to textilize.

<%= textilize(coderay(@article.content)) %>

We don’t want the code that’s returned by the coderay method to be parsed by RedCloth so what we have to do is modify the method slightly, wrapping its output in a notextile element so that it isn’t parsed.

def coderay(text)
  text.gsub(/\<code( lang="(.+?)")?\>(.+?)\<\/code\>/m) do
    content_tag("notextile", CodeRay.scan($3, $2).div(:css => :class))
  end
end

If we edit the article again and add some markup code like this:

The piano is a musical instrument played by means of a keyboard that produces sound by striking steel strings with felt hammers. The hammers immediately rebound allowing the strings to continue vibrating at their resonant frequency. These vibrations are transmitted through a bridge to a soundboard that amplifies them.

 * item
 * item 2

<code lang="ruby">
def hello
  puts 'Hello, world!'
end
</code>

The article will now have a Textile-generated unordered list in it along with the syntax-highlighted code.

That’s it for this episode on syntax highlighting. CodeRay makes it easy to add syntax highlighting to code snippets in your Rails applications and is well worth a look if you need that functionality.