#229 Polling for Changes

Imagine that you have a blogging application that allows its users to add comments to articles. The blog is very popular so new comments are added frequently. If someone comes to the site to read an article and its comments and afterwards decides to add their own comment it’s possible that by the time they come to add their new comment other new comments have already been added to the article making their comment incorrect or irrelevant. It would be helpful therefore if the article page kept itself updated and automatically added any new comments that were posted. We’ll show you how to do that in this episode.

Setting Up jQuery

We’re going to have to use JavaScript in order to update the page on the fly and we’re going to use jQuery instead of Prototype as our preferred JavaScript library. As our application is written in Rails 3 we can use the jquery-rails gem to make it easy to replace the Prototype files with jQuery’s.

To use the gem we just need to add a reference to it in our application’s Gemfile.

gem 'jquery-rails'

We can then set it up by running

$ bundle install

After Bundler has finished there will be a new generator available that we can run to set up jQuery in our application. When we run it it will remove the Prototype files, add the jQuery ones and then overwrite the application’s default rails.js file with a jQuery compatible one. (If you want to install the jQuery UI library along with jQuery you can pass a --ui option to install those files.)

$ rails g jquery:install
      <span class="passed">remove</span>  public/javascripts/controls.js
      <span class="passed">remove</span>  public/javascripts/dragdrop.js
      <span class="passed">remove</span>  public/javascripts/effects.js
      <span class="passed">remove</span>  public/javascripts/prototype.js
      <span class="passed">create</span>  public/javascripts/jquery.min.js
      <span class="passed">create</span>  public/javascripts/jquery.js
    <span class="failed">conflict</span>  public/javascripts/rails.js
Overwrite /Users/eifion/rails/apps_for_asciicasts/ep229/blog/public/javascripts/rails.js? (enter "h" for help) [Ynaqdh] Y
       <span class="forced">force</span>  public/javascripts/rails.js

One of the useful things about the jquery-rails gem is that it automatically overrides the defaults that are included when we use the line

<%= javascript_include_tag :defaults %>

in our layout file so we don’t need to make any changes here.

Setting up Polling

Now that we’ve set up jQuery in our application we can start to do the work necessary to get our comments to self-update. There are two ways that we can approach this problem. One way is to use polling. Our application can send an AJAX request to the server at a regular interval to see if the number of comments has changed. The other option is to use WebSockets to provide a permanent connection to the server and to push any changes to the user as soon as they become available. Polling is the easier of the two options as WebSockets are not supported by all browsers yet and would require more coding along with a library such as Cramp to get working. We don’t need instant updates when a new comment is posted, a delay of a few seconds is perfectly acceptable, and so we’ll take the polling approach.

We’ll write the JavaScript that will poll for changes in the application’s application.js file. This file is included in all of the app’s pages so we’ll need to ensure that the code that checks for updated comments is only run on pages that have comments on them. The polling code will look like this.

$(function () {
  if ($('#comments').length > 0) {
    setTimeout(updateComments, 10000);
  }
});

function updateComments() {
  $.getScript('/comments.js');
  setTimeout(updateComments, 10000);
}

The code above starts with a call to jQuery’s $ function. When this function is called with a function as an argument, as it is here, the code inside the function is run when the page’s DOM has loaded. In this case when the DOM loads we check that the current page is an article page by looking for an element with an id of comments. If that element is found then we use setTimeout to call a function called updateComments after a delay of ten seconds.

In the updateComments function we want to get any updated comments for the article. There are a number of different ways we could do this. We could get the server to return the new comments in JSON format and use that data to create the HTML for the new comments but the problem with that is that updateComments would need to generate the correct HTML for the new comments. It would be easier to generate the HTML on the server and send back some JavaScript that will update the page to add that HTML in the correct place.

To do this we use jQuery’s getScript function. If we pass this function a URL it will make an AJAX request to that URL and execute any script that is returned. We’ll use getScript to call the CommentsController’s index action with a JavaScript format. We’ll need to add some query string parameters to the URL so that we can say which comments should be returned, but we’ll come back to that later.

After getScript has completed we call setTimeout again so that updateComments is called again after another ten seconds. We could have used setInterval instead of setTimeout when we check that the comments element exists so that we don’t need to call it again here, but there is an advantage to taking this approach. If we used setInterval then updateComments would be called every ten seconds no matter how long the AJAX call took to reply. Taking this approach means that the request will be made ten seconds after the call returns so that we’re not placing as heavy a load on the server when it gets busy.

The CommentsController doesn’t currently have an index action so we’ll have to write one. It will need to fetch the appropriate comments using the parameters we pass in from the JavaScript code above but we’ll leave it empty for now.

def index

end

We’ll need a JavaScript template to go with the action. In it we’ll just put a piece of test code to make sure that the polling is working.

alert('Comments go here');

When we start up the application’s server and visit the page for an article now we’ll see the alert appear about ten seconds after the page loads which means that the JavaScript above is being returned from the server and executed on the client.

Fetching The Correct Comments

Now that we have this working the next step is to get the relevant comments. We need to add two parameters to the call to the CommentController’s index action: the id of the current article and a timestamp so that we know which comments to get.

function updateComments() {
  $.getScript('/comments.js?article_id=' + article_id + "&after=" + after);
  setTimeout(updateComments, 10000);
}

What we still need to work out in the code above is where we will get the values for the article_id and after variables that we haven’t yet defined. In this case we can generate the values in Rails and pass them to JavaScript from the HTML document. This application uses HTML 5 so we can make use of data attributes to add the data to the document.

We’ll add the attributes in to the view code for the ArticleController’s show action. Data attributes can have any name as long as it begins with data- so we’ll add a data-id attribute to the article’s wrapper div that has a value of the article’s id and in similarly in the wrapper div for each comment we’ll add a data-time attribute containing the comment’s created_at value, converted to an integer.

<% title @article.name %>

<div id="article" data-id="<%= @article.id %>">
  <%= simple_format @article.content %>
  
  <p><%= link_to "Back to Articles", articles_path %></p>
  
  <% unless @article.comments.empty? %>
    <h2><%= pluralize(@article.comments.size, 'commment') %></h2>
    
    <div id="comments">
    <% for comment in @article.comments %>
      <div class="comment" data-time="<%= comment.created_at.to_i %>">
        <strong><%= comment.name %></strong>
        <em>on <%= comment.created_at.strftime('%b %d, %Y at %I:%M %p') %></em>
        <%= simple_format comment.content %>
      </div>
    <% end %>
    </div>
  <% end %>
  
  <h3>Add your comment</h3>
  <%= render :partial => 'comments/form' %>
</div>

We can now go back to the updateComments function and set the values of the article_id and after variables from those data attributes. Getting the article’s id is straightforward: we simply get it from the element with an id of article. There could be a number of comments on the page and we need to get the last one so we’ll use the selector .comment:last to get it, then read its data-time attribute.

function updateComments() {
  var article_id = $('#article').attr('data-id');
  var after = $('.comment:last').attr('data-time');
  $.getScript('/comments.js?article_id=' + article_id + "&after=" + after);
  setTimeout(updateComments, 10000);
}

Now that we have values for both of these variables we can use them to fetch only the comments that have been created for that article after the timestamp for the last one on the page.

It’s a good idea when doing something like this to check the development log to make sure that it looks like everything is working as it should be. If we visit the articles page and then look at the log we’ll see that the the request for the comments JavaScript is being made every ten seconds or so as we expect and the parameters look to be correct too.

Started GET "/comments.js?article_id=1&after=1283173233" for 127.0.0.1 at 2010-09-02 19:58:53 +0100
  Processing by CommentsController#index as JS
  Parameters: {"article_id"=>"1", "after"=>"1283173233"}
Rendered comments/index.js.erb (0.4ms)
Completed 200 OK in 41ms (Views: 41.1ms | ActiveRecord: 0.0ms)


Started GET "/comments.js?article_id=1&after=1283173233" for 127.0.0.1 at 2010-09-02 19:59:07 +0100
  Processing by CommentsController#index as JS
  Parameters: {"article_id"=>"1", "after"=>"1283173233"}
Rendered comments/index.js.erb (0.4ms)
Completed 200 OK in 34ms (Views: 33.6ms | ActiveRecord: 0.0ms)

Now that we know that the parameters are being passed correctly we can use them in the CommentsController’s index action to get the new comments.

def index
  @comments = Comment.where("article_id = ? and created_at > ?", params[:article_id], Time.at(params[:after].to_i))
end

We can now update the associated JavaScript template so that it renders out the comments rather than just showing an alert. Before we do that, however, we’ll need move the code that renders a comment out from the article’s show view into a partial so that we can use it in the JavaScript template. We’ll move it into a new partial file called _comment.html.erb.

<div class="comment" data-time="<%= comment.created_at.to_i %>">
  <strong><%= comment.name %></strong>
  <em>on <%= comment.created_at.strftime('%b %d, %Y at %I:%M %p') %></em>
  <%= simple_format comment.content %>
</div>

Then in the show action we can just render the partial for the collection of comments.

<% title @article.name %>

<div id="article" data-id="<%= @article.id %>">
  <%= simple_format @article.content %>
  <p><%= link_to "Back to Articles", articles_path %></p>
  <% unless @article.comments.empty? %>
    <h2><%= pluralize(@article.comments.size, 'commment') %></h2>
    <div id="comments">
    <%= render @article.comments %>
    </div>
  <% end %>
  <h3>Add your comment</h3>
  <%= render :partial => 'comments/form' %>
</div>

We can update the comments index JavaScript template so that it generates the JavaScript to update the comments.

$('#comments').append("<%= escape_javascript(raw render(@comments))%>");

This code renders the collection of comments using the partial we have just written. We have to wrap the output from the partial in the raw method as by default Rails 3 will escape the HTML and we don’t want that to happen. We then have to call escape_javascript on the output from that command so that the HTML is safe to embed into a JavaScript string. The JavaScript that is sent back to the browser appends this string to the end of the comments div.

We can take a look at the page in the browser now to see if the comments update automatically and we get mixed results. While the comments do get updated we’re seeing the last comment added again and again.

This is happening because when we get the new comments the timestamp we pass in in only precise to a second. In this case we’re getting the comments after 13:00:33 on August 30th 2010, but if we look in the database the comment was actually left at 13:00:33.892041 and while this isn’t much of a difference it’s enough for it to be considered after the time passed to the query. To fix this we need to round up so that we get the comments left at least a second later than the current latest comment and so we’ll add 1 to the integer value received from the parameters in the query.

def index
  @comments = Comment.where("article_id = ? and created_at > ?", params[:article_id], Time.at(params[:after].to_i + 1))
end

Now we won’t see the final comment appearing repeatedly. We can test that the updating works by opening the article’s page in two browser windows, submitting a comment in one and seeing if it appears in the other.

After a few seconds the polling takes place and the comment appears in the other browser.

Updating The Count

While the new comment was added, the heading that lists the number of comments wasn’t updated to match. Let’s fix that next.

We can update the comments count in the same file we use to update the comments. At first the obvious thing might be to do something like this:

$('#comments').append("<%= escape_javascript(raw render(@comments))%>");
$('#article h2').text('<%= @comments.first.article.comments.size %> comments');

This approach will make a call on the database to get the number of comments, however, and as this code will be called frequently it makes sense to avoid this if we can. We can instead get the number of comments from the page by counting the number of elements on the page with a class of comment.

$('#comments').append("<%= escape_javascript(raw render(@comments))%>");
$('#article h2').text($('.comment').length + ' comments');

This won’t code won’t handle pluralization so it will only show “comments”, even if there’s only one comment, but we’ll leave that. As a final touch we’ll add a clause so that the JavaScript is only set if there are some new comments.

<% unless @comments.empty? %>
$('#comments').append("<%= escape_javascript(raw render(@comments))%>");
$('#article h2').text($('.comment').length + ' comments');
<% end %>

When we add another comment in one browser now the comments are updated in the other along with the count.

A Final Fix

There’s one small problem with our code that still needs fixing. In the updateComments function the value of the after variable is set from an attribute taken from the last comment on the page. If there are no comments for the article that’s being viewed then this value will be blank so we’ll set a default value of zero in this case so that all of the comments are returned.

$(function () {
  if ($('#comments').length > 0) {
    setTimeout(updateComments, 10000);
  }
});

function updateComments() {
  var article_id = $('#article').attr('data-id');
  if ($('.comment').length > 0) {
    var after = $('.comment:last').attr('data-time');
  }
  else {
    var after = 0;
  }
  
  $.getScript('/comments.js?article_id=' + article_id + "&after=" + after);
  setTimeout(updateComments, 10000);
}

There are other improvements we could make to this part of the code. For example we could make the update interval dynamic so that if the last comment was added a long time ago, say over an hour, the polling frequency is reduced. This way less strain will be placed on the server when articles that haven’t had comments added recently are being viewed.

We could also spice up the user interface a little by adding highlighting to indicate when new comments are added to the page. Alternatively we could add a link that says that new comments are available and only show them when it is clicked.