Railscasts - Syntax Highlighting

#207

Mar 29, 2010

23 comments

Syntax Highlighting

Here I talk about three popular choices for syntax highlighting in Rails: CodeRay, Ultraviolet and Pygments.

Tags: views plugins

Download (30.9 MB, 9:09)

alternative download for iPod & Apple TV (24.4 MB, 9:09)

Read it on ASCIIcasts

Resources

<%= textilize(coderay(@article.content)) %>

# config/environment.rb
config.gem "coderay"
config.gem "RedCloth"

# application_helper.rb
def coderay(text)
  text.gsub(/\<code( lang="(.+?)")?\>(.+?)\<\/code\>/m) do
    content_tag("notextile", CodeRay.scan($3, $2).div(:css => :class))
  end
end

# syntax_benchmark.rb
require "rubygems"
require "benchmark"
require "coderay"
require "uv"

path = __FILE__
content = File.read(__FILE__)

# run it once to initialize
CodeRay.scan("print 'hello'", "ruby").div(:css => :class)
Uv.parse("print 'test'", "xhtml", "ruby", true, "amy")

Benchmark.bm(11) do |b|
  b.report("coderay") do
    50.times { CodeRay.scan(content, "ruby").div(:css => :class) }
  end
  
  b.report("ultraviolet") do
    50.times { Uv.parse(content, "xhtml", "ruby", true, "amy") }
  end

  b.report("pygments") do
    50.times { `pygmentize -f html "#{path}"` }
  end
end

.CodeRay {
  background-color: #232323;
  border: 1px solid black;
  font-family: 'Courier New', 'Terminal', monospace;
  color: #E6E0DB;
  padding: 3px 5px;
  overflow: auto;
  font-size: 12px;
  margin: 12px 0;
}
.CodeRay pre {
  margin: 0px;
  padding: 0px;
}

.CodeRay .an { color:#E7BE69 }                      /* html attribute */
.CodeRay .c  { color:#BC9358; font-style: italic; } /* comment */
.CodeRay .ch { color:#509E4F }                      /* escaped character */
.CodeRay .cl { color:#FFF }                         /* class */
.CodeRay .co { color:#FFF }                         /* constant */
.CodeRay .fl { color:#A4C260 }                      /* float */
.CodeRay .fu { color:#FFC56D }                      /* function */
.CodeRay .gv { color:#D0CFFE }                      /* global variable */
.CodeRay .i  { color:#A4C260 }                      /* integer */
.CodeRay .il { background:#151515 }                 /* inline code */
.CodeRay .iv { color:#D0CFFE }                      /* instance variable */
.CodeRay .pp { color:#E7BE69 }                      /* doctype */
.CodeRay .r  { color:#CB7832 }                      /* keyword */
.CodeRay .rx { color:#A4C260 }                      /* regex */
.CodeRay .s  { color:#A4C260 }                      /* string */
.CodeRay .sy { color:#6C9CBD }                      /* symbol */
.CodeRay .ta { color:#E7BE69 }                      /* html tag */
.CodeRay .pc { color:#6C9CBD }                      /* boolean */

< Previous Episode

Next Episode >

23 comments

1. Pavel Mar 29, 2010 at 00:53

Thanks!

Report as Spam

2. Nils R Mar 29, 2010 at 01:14

Nice one!

Though I don't need that syntax highlighting right now, i can definately make use of the benchmarking. Thanks again.

Report as Spam

3. Andrea Mar 29, 2010 at 01:25

Thanks Ryan, I was planning to build an app with this feature, so your screencast is definitely saving me some time.

Report as Spam

4. Emerson Lackey Mar 29, 2010 at 01:47

Ryan, I'm actually in the process of setting up my own blog (built on Rails of course) that will focus on tutorials, screen-casts, and my personal musings.

A few days ago I realized that I hadn't setup a syntax highlighter yet... long story short I stopped in Railscasts and ended up using CodeRay after looking at your CSS/Source HTML.

I managed to get everything setup, but not without some trouble. Funny that you just came out with this, always on target at Railscasts!

Report as Spam

5. Boblin Mar 29, 2010 at 02:12

Thanks for the screencast.

Report as Spam

6. Alerksandr Koss Mar 29, 2010 at 02:29

Thanks!

Another method (via vim):

http://gist.github.com/347644

It script save code highlight to html file in current dir.

Report as Spam

7. Brandon Wright Mar 29, 2010 at 02:30

Amazing how you always post something about what I just did or am about to do! I figured out your Coderay technique by checking out the source to Railscasts on Github but I always like listening to your explanation and comments.

Report as Spam

8. Matt Swasey Mar 29, 2010 at 07:27

My friend @gotascii and I wrote a rack middleware that uses the unofficial pygments api (http://pygments.appspot.com/) to highlight source code within a ruby app.

Check it out, the only lib it requires is nokogiri.

http://www.gotascii.com/2009/11/7/pygrack-pygments-highlighting-middleware

Report as Spam

9. hakunin Mar 29, 2010 at 08:19

A couple of notes.

1) In the regexp, if you are not going to be using a matching group, you can mark it as a non-matching group using '?:', ie /(?:this_group_wont_match)/, so you can use $2, $1, instead of $3, and $2. In your case it would look like this: /\<code(?: lang="(.+?)")?\>(.+?)\<\/code\>/m.

2) Changing code tag into div may not be a good idea if you're planning to have both inline and block code. code tag is by itself an inline element. To use block code it may be better to wrap code in pre.

Report as Spam

10. Kurt Werle Mar 29, 2010 at 09:16

Really slick. Thanks.

Report as Spam

11. grigio Mar 29, 2010 at 09:43

Does it also work with FCKEditor or another WYSIWYG editor?

Report as Spam

12. Memiux Mar 29, 2010 at 11:23

Thanks, as always you're great but there is one more thing...

The video quality of this episode seems to be slightly awful compared with the rest of them.

Report as Spam

13. Ryan Bates Mar 29, 2010 at 13:24

@Matt, using Rack middleware is an interesting solution for this. I haven't looked into it but does performance become a problem? I imagine it needs to look through every HTML page even if there is no <pre> block on it, and is it possible to cache the results?

@hakunin, thanks for the notes and very good points. I always forget the syntax for non matching parenthesis.

@grigio, I'm not very familiar with these editors, but you just need to determine what they generate for the code syntax and swap that out like I do with the <code> tag in this episode.

@Memiux, I'm experimenting with other video codecs because, unfortunately, the Animation codec which I normally use is very buggy in Snow Leopard. I hope to find a solution which is nearly as good quality as Animation.

Report as Spam

14. Fredrik Mar 29, 2010 at 14:30

@Rbates, Memiux

I dont have any problems at all with the quality over all. Though that could be related to me running Ubuntu 10.04.

What I do note though is the filesize. It's a bit larger than usual for a shorter length of screencast.
Every byte is worth saving, considering that not everyone is using high-performance connections.
I for example is stuck with 2mbit and love smaller stream-sizes.

Report as Spam

15. Gregor Mar 29, 2010 at 14:54

damn, I hoped so much you would continue covering new features in rails3.

Report as Spam

16. Danz Mar 29, 2010 at 23:03

Ryan, thanks for another good tutorial.

It's seem a good step for highlight search result, am I right?!
I thought the method looks similar.

Report as Spam

17. Tatogenylorne Mar 30, 2010 at 08:54

in nto work for meee

Report as Spam

18. David Waite Mar 30, 2010 at 10:43

I was wondering if there were any reasons you chose the ruby-based libraries over javascript libraries, other than the rails-specific target of the screen-cast series?

Report as Spam

19. Nathan Youngman Mar 30, 2010 at 12:48

Hey Ryan,
Thanks for the informative screencast. Looking at the benchmarks, I was wondering how much of Pygments slow performance is from shelling and how much is due to Pygments itself.

I'm not very familiar with this benchmarking stuff though (the next Railscast?:-). On my machine coderay is about the same, 0.24 total and pygmentize is 9.28.

Not to discredit Pygments, I wrote up the following snippet, which may not be that useful for Rails developers. It returns 0.496517 usec to run Pygments 50 times from Python. Not sure exactly how to get more consistent benchmarks between the two.

http://gist.github.com/349504

Report as Spam

20. 2 Mar 31, 2010 at 08:03

Report as Spam

21. Schiphol aankomsttijden Mar 31, 2010 at 09:06

Is there an update on the pygments yet? Would also like to know this info.

Report as Spam

22. AkitaOnRails Mar 31, 2010 at 09:36

Nice screncast, as usual :-) But one recommendation would be, instead of re-parsing the code snippet at every request and caching it in the view, it is better to add the parsing as a before_save hook in the model and store the final HTML already pre-rendered in another text column in your table. Then the view has only to display this column instead of the original and because of this, it won't matter which library is faster because you only have to run it once and not at every request.

Report as Spam

23. Michael van Rooijen Apr 02, 2010 at 05:43

Thanks for this one Ryan.
I've been having quite a hard time coming up with a good solution to have both textile and coderay work for one of my apps. This will definitely do the trick! :)

Report as Spam

24. Christopher Giroir Apr 07, 2010 at 10:29

Great video (as they all are).

I ended up using ultraviolet since I love the textmate themes anyway (I'm actually a Emacs+Mac guy myself). My blog isn't moved over from Wordpress to my own custom software yet, but once it is it's going to use ultraviolet.

Just so others know, Heroku has the required libraries for Ultraviolet already installed which make using it a snap :)

http://mxkelsin.heroku.com/2010/04/01/installing-ultraviolet-on-my-mac/

Report as Spam

25. mens suit Apr 07, 2010 at 19:11

You're a genius. I can use this on my apps.

Report as Spam

26. murphy Apr 16, 2010 at 05:52

Thank for covering CodeRay - you did an outstanding job of explaining its strengths, weaknesses, and how to use it.

Only one addition: There's a built-in option to combine called (require) "coderay/for_redcloth" which enables the textile-style @[lang]...@ and bc[lang] shortcuts for syntax highlighting.

Report as Spam

27. Kristian Apr 17, 2010 at 07:10

I have not been able to make it work with Rails 3.

I tried using your suggestions from the XSS screencast with combinations of html_safe and raw, but so far no luck. I can get it to output it escaped once or twice, but never unescaped!

Anyone made it work for Rails 3?

Report as Spam

28. Kristian Mandrup Apr 17, 2010 at 07:34

I got it to work with Rails 3.

def coderay(text)
 text.gsub!(/\<code(?: lang="(.+?)")?\>(.+?)\<\/code\>/m) do
 code = CodeRay.scan($2, $1).div(:css => :class)
 "<notextile>#{code}</notextile>"
 end
 return text.html_safe
 end

And to override the scaffold pre style, added background-color: black

.CodeRay pre {
  margin: 0px;
  padding: 0px;
  background-color: black;
}

Report as Spam

29. kib2 Apr 18, 2010 at 09:12

Hi,

just to let you know that I've build a new syntax highlighting engine in Ruby. It's called Prism and you'll find it here: http://github.com/kib2/Prism.

I've also made a little pastebin with it : http://prism-pastebin.heroku.com/.

The sample you gave renders the following: http://prism-pastebin.heroku.com/13

Please report me any bug/problem, see you.

Report as Spam

30. site tanıtım Jun 29, 2010 at 17:51

Site tanıtım araçları arasında yer alan sitler 2010 popüler ve kaliteli siteleri hizmetinize sunuyor. Siteler 2010 yılında tıtım yapıtıyor.

Report as Spam

31. strapping machine Jul 07, 2010 at 00:06

Leaving a comment is the biggest support to Blogger.Thanks!

Report as Spam

32. shrink wrap Jul 07, 2010 at 00:07

Thanks for sharing the information.It is definitely going to help me some time.

Report as Spam

33. dhan Jul 07, 2010 at 08:54

thanks.. i'll try..

keep share :)

Report as Spam

34. welding rod Jul 08, 2010 at 18:07

this was a really quality post.I wasn't aware of the many ripples and depth to this story until I surfed here through Google! Great job.

Report as Spam

35. welding electrode Jul 08, 2010 at 18:16

I remember I tried this a while ago. It brings back bad memories. Nothing good seems to happen the first time. How long did it take you? I look forward to your next story.

Report as Spam

36. leisse Jul 23, 2010 at 06:57

Thank you,I think its very useful.

Report as Spam

38. tiffany notes Jul 30, 2010 at 02:20

Great site. This could probably have the refactoring tag added t it.

Report as Spam

39. Batzooh Aug 07, 2010 at 15:16

To have the correct output with Rails 3.0.0.rc replace the following line :

content_tag("notextile", CodeRay.scan($3, $2).div(:css => :class))

by :

content_tag('notextile', CodeRay.scan($3, $2).div(:css => :class).html_safe)

(simply add html_safe)

Report as Spam

40. Batzooh Aug 07, 2010 at 15:19

Oops, sorry for make it in two times, also :

<%=raw textilize(coderay(@article.content)) %>

(simply add raw)

PS : Perhaps, could you join this to my precedent comment and delete this one ?

Report as Spam

40. precision casting Aug 08, 2010 at 01:53

At QS Machinery, our engineers team always work closely with you, from a idea to finished products, so We can provide Quality Steel Casting, lost wax casting, precision casting and downhole cable protector and so on.

Report as Spam

41. kinopyo Aug 09, 2010 at 01:12

To 53:@Batzooh

Thanks for your comment, finally I can let it run on Rails 3.0.0.rc.

But I also notice that you're using raw helper method, which I think is dangerous because any script in the article content will run. It's better to use sanitize helper method.

<%=sanitize coderay(@article.content) %>

Report as Spam

42. UGG Boots on sale Aug 10, 2010 at 18:43

Gooooooooooooooooooood luck ~~!!

Report as Spam

43. designer handbags Aug 11, 2010 at 07:06

<a href="http://www.buyonstore.com/">designer handbags</a>
The article is worth reading, I like it very much. I will keep your new articles.

Report as Spam

44. free directory list Aug 11, 2010 at 22:42

I really appreciate what you post.

Report as Spam

45. oppo Aug 15, 2010 at 08:39

This was really useful. Thanks Ryan!

Report as Spam

46. AVI to iPad Aug 19, 2010 at 00:55

This was really useful.

Report as Spam

47. wholesale new era hats Aug 20, 2010 at 20:08

Generally I do not post on blogs, but I would like to say that this post really forced me to do so, Excellent post!

Report as Spam

48. converse all star Aug 20, 2010 at 20:50

love converse all star,love yourself.High quality low price.It's fit for you.

Report as Spam

49. lina Aug 23, 2010 at 23:27

Any member of your group can post to your trip blog. This is a great way to share information with your team and your
supporters.<a href=http://www.hzzxdq.net>power strip</a> |<a href=http://www.jdxpwj.cn/booster-cable>booster cable</a> |
<a href=http://www.jdxpwj.cn/tow-rope>tow rope</a> |<a href=http://www.jdxpwj.cn/ratchet-tiedown>ratchet tiedown</a>

Report as Spam

50. lily Aug 23, 2010 at 23:37

Well done, I admire the valuable information you offer in your articles. I will bookmark your blog
and have my children check up here often. I am quite sure they will learn lots of new stuff here than
anybody else<a href=http://www.hzhtdq.cn>power strip</a> |<a href=http://www.hzhtdq.cn>extension cord</a>
|<a href=http://www.hzhtdq.cn>trouble light</a>

Report as Spam

51. louis vuitton shoes Aug 26, 2010 at 20:56

Thanks for sharing your article. I really enjoyed it. I put a link to my site to here so other people can read it. My readers have about the same interets

Report as Spam

52. rap Aug 29, 2010 at 08:55

Thank you,I think its very useful.

Report as Spam

53. replicahandbags Aug 29, 2010 at 23:02

I have always liked Outdoor movies, a child standing at the window, looked out from home

to the following. Will be able to see the staff busy figure, a huge white cloth has a

child hang up and soon will be able to see the movie.

Report as Spam

54. herve leger dress Aug 30, 2010 at 19:45

Yuyao Huaneng sanitary ware factory is specialized in production of plumbing fittings.

Report as Spam