#168
Jun 29, 2009

Feed Parsing

Learn two different techniques for parsing an RSS feed using Feedzirra in this episode!
Download (18.1 MB, 10:31)
alternative download for iPod & Apple TV (13.5 MB, 10:31)

Resources

sudo gem install pauldix-feedzirra
script/generate model feed_entry name:string summary:text url:string published_at:datetime guid:string
rake db:migrate
# config/environment.rb
config.gem "pauldix-feedzirra", :lib => "feedzirra", :source => "http://gems.github.com"

# script/console
FeedEntry.update_from_feed("http://feeds.feedburner.com/railscasts")
FeedEntry.count

# models/feed_entry.rb
class FeedEntry < ActiveRecord::Base
  def self.update_from_feed(feed_url)
    feed = Feedzirra::Feed.fetch_and_parse(feed_url)
    add_entries(feed.entries)
  end
  
  def self.update_from_feed_continuously(feed_url, delay_interval = 15.minutes)
    feed = Feedzirra::Feed.fetch_and_parse(feed_url)
    add_entries(feed.entries)
    loop do
      sleep delay_interval
      feed = Feedzirra::Feed.update(feed)
      add_entries(feed.new_entries) if feed.updated?
    end
  end
  
  private
  
  def self.add_entries(entries)
    entries.each do |entry|
      unless exists? :guid => entry.id
        create!(
          :name         => entry.title,
          :summary      => entry.summary,
          :url          => entry.url,
          :published_at => entry.published,
          :guid         => entry.id
        )
      end
    end
  end
end
<!-- articles/index.html.erb -->
<div id="recent_episodes">
  <h3>Recent Railscasts Episodes</h3>
  <ul>
  <% for entry in FeedEntry.all(:limit => 10, :order => "published_at desc") %>
    <li><%= link_to h(entry.name), entry.url %></li>
  <% end %>
  </ul>
</div>

RSS Feed for Episode Comments 18 comments

1. sthapit Jun 29, 2009 at 00:53

always enjoy your screencasts. thanks!


2. Kieran P Jun 29, 2009 at 04:32

Been using FeedZirra for some time now. It's been really great. But long/complex urls on Ruby Enterprise Edition have caused problems recently.


3. Michael Jun 29, 2009 at 04:59

Thanks again mate!


4. Ricky Jun 29, 2009 at 06:26

Thanks Ryan

any idea how to combine multiple feeds into one master feed?


5. Nils R Jun 29, 2009 at 06:52

It is starting to become kinda creepy … it's as if you were reading my mind. Always coming with stuff I need just in time!

Love your screencasts!


6. RailsCasts Fan Jun 29, 2009 at 10:21

Thank you Ryan!
That's exactly what I needed!
So far, I used Nokogiri. The performance of XML processing is really better than Nokogiri and than Hpricot?


7. Tim Matheson Jun 29, 2009 at 10:51

I second the notion that Ryan has ESP. His screencast's always seem to be right on target with what I am working on. We joke that he has bugged our office space :D.
Thanks Ryan another great episode.


8. Gavin Jun 29, 2009 at 11:16

Great! Thanks for this Ryan

ps - I think feedzirra is supposed to rhyme with Godzilla... it's a joke


9. Julien GENESTOUX Jun 29, 2009 at 13:06

Anybody dealing with RSS parsing for a lot of feeds and who don't want to have the hassle of polling should check out http://superfeedr.com

Let me know what you think! (You included Ryan ;))


10. Orion Engleton Jun 30, 2009 at 00:48

This is awesome .. I had to do this a couple weeks ago.. I had the same moving parts as you but not as clear and concise!! Thanks for totally showing me the right way to do this!
I agree with Nils.. when I first saw the title I had deja vu!


11. Nacho Jun 30, 2009 at 06:43

I'm getting errors installing it on a Windows box because of curl. Is there a way to install curl on windows?


12. wrax Jun 30, 2009 at 22:54

@ryan what problems were you having with updating? I cant seem to get it to work either. It fetches and parses fine. Anyone else getting this?

http://pastie.org/530375


13. bob d Jul 03, 2009 at 17:57

I found that this plugin works very well for any xml, but the documentation is not available. Just do the following:
- freeze the feedzirra gem
- in the parser directory clone the rss and rss_entry files, rename them and change the mappings to match you xml structure
- locate feedzirra.rb and add references to your new parser files
- in feed.rb update @feed_classes array with your new parser

In general you should be able to use the example files as reference. Also make sure to update the specs.


14. Pooja Gupta Jul 07, 2009 at 04:30

Is there a way to install curl on windows? does feedzirra work with windows or not ?


15. Jason Cohen Jul 07, 2009 at 07:47

Can you give guidelines for how you would unit-test this code?

For example, can you use a local file URL to ensure it properly parses and stuffs in the database various test data?

Thanks.


16. Joris Jul 26, 2009 at 12:27

Feedzirra needs information from the stored feed entries to get the update working. You can read more on this here: http://groups.google.com/group/feedzirra/browse_thread/thread/6eb16d9a6d4d168e#


17. Michael Aug 14, 2009 at 18:27

Does anyone have a code sample for a daemonized version of this? I'm not quite sure how to get the initial statement to run only once within daemon_generator's while clause.

I used the code as is, but it will never terminate since it will never complete the while($running) loop because of the update loop.


18. anon_anon Dec 02, 2009 at 17:26

You might want to look at vtd-xml, the latest and most advanced xml processing model, far better than DOM or SAX

<a href="http://vtd-xml.sf.net">vtd-xml</a>


19. Valentin Jan 06, 2010 at 03:26

Might be a bit late to post here, but I can't get this to work at all...
 
I followed step by step but get this error:

NoMethodError: undefined method `update_from_feed' for #<Class:0x103804fb8>

I don't really see where I am going wrong, as I have that method defined in the model ...

Thanks!


20. Brad May 17, 2010 at 00:36

I'm a bit of a newbie to all this and after a few hours of experimenting today, I couldn't for the life of me work out how to the Feedzirra in Cron with Whenever. Any ideas?


21. Nike pas cher Jul 13, 2010 at 00:40

thank you for your resources.


22. iPhone Ringtone Maker for mac Jul 20, 2010 at 19:09

ah ha .so ^^^^


23. Chirantan Jul 23, 2010 at 04:20

How to parse custom feed attributes? There is a way to do that for entries using

Feedzirra::Feed.add_common_feed_entry_element("wfw:commentRss", :as => :comment_rss)

But I want the same feature for the feed object, not just its entries. Is there a way to do that? Like say

Feedzirra::Feed.add_common_feed_element("geo:lat", :as => :latitudes)

Help!


25. optical fiber Cable Aug 01, 2010 at 23:59

I am very appreciate the issue that it can benefit to the low income guys, thanking for your sharing, it is wonderful.


25. solar collector solar water heater Aug 02, 2010 at 00:03

It's one of the most important facto to fix this time.


26. timberlandbootsuk Aug 02, 2010 at 02:07

we provide our buyers with an efficient and manageable procurement process covering every phase of the international supply chain and

streamlining trade channels. Also welcome wholesaling, feedback now!


27. authentic nike shoes Aug 17, 2010 at 20:01

I am very appreciate the issue that it can benefit to the low income guys, his is awesome .. I had to do this a couple weeks ago.. I had the same moving parts as you but not as clear and concise!! thanking for your sharing, it is wonderful.


28. Wholesale hats Aug 20, 2010 at 20:30

Took me awhile to read all the comments, but I really love the article. It proved to be very useful to me and I am sure to all the commenters here! It's always nice when you can not only be informed, but also entertained! I'm sure you had fun writing this article. Comfortably, the article is really the sweetest on this precious topic.


29. jimmy choos Aug 21, 2010 at 01:09

Thank you for sharing!


30. Wholesale Electronics Aug 25, 2010 at 01:52

Discount Wholesale Electronics, Wholesale Cell Phones, Electronic Gadgets and More from the Best Dropship Wholesaler


31. silly bandz Aug 25, 2010 at 03:30

<a href="http://www.toptoys2trade.com/power-balance-wholesale-2-c-40/ "> power balance</a>


32. louis vuitton shoes Aug 26, 2010 at 21:02

Thanks for sharing your article. I really enjoyed it. I put a link to my site to here so other people can read it. My readers have about the same interets


33. WiFi Cell Phones Aug 30, 2010 at 20:17

thanks for your sharing the post, i like it so much for the useful information.


34. snow boots Aug 30, 2010 at 20:55

I'm getting errors installing it on a Windows box because of curl. Is there a way to install curl on windows?


35. levis belts Sep 01, 2010 at 20:54

Good article! Thank you so much for sharing this post.Your views truly open my mind.

Add your comment:

(SKIP THIS ONE)

(required)

(not shown)


(use pastie or gist for code)

sponsored by:
if you want to help:
required:
Get Quicktime Player
Give Back to Open Source