#22
Apr 23, 2007

Eager Loading

One way to improve performance is to cut down on the number of SQL queries. You can do this through eager loading. Learn all about it in this episode!
Download (22.7 MB, 4:37)
alternative download for iPod & Apple TV (9.7 MB, 4:37)
Task.find(:all, :include => :projects)
Task.find(:all, :include => [:projects, :comments])
Task.find(:all, :include => [:projects, {:comments => :user}])

RSS Feed for Episode Comments 38 comments

1. Jake Apr 23, 2007 at 00:29

I always get excited when I see the (1) next to railscast in my RSS reader. Another great tip!


2. skycn Apr 23, 2007 at 00:31

so good ,thank you .
love it!


3. sintaxi Apr 23, 2007 at 00:40

I often refer back to these casts while I work. Thank you.


4. riki Apr 23, 2007 at 01:40

Thank you! These are great.


5. Rutra Apr 23, 2007 at 01:47

You are doing fantastic job, Thank you!


6. dima Apr 23, 2007 at 02:40

Thank you for your astonishing effort.
You give us a great educational place.

To all my new rails colleagues I recommend your screen casts as an easy to understand knowledge mine.


7. Ted Apr 23, 2007 at 06:17

Great stuff as usual.

I'd like to have seen the before and after results of including comment and user counts on the task list and how the database queries were affected when you added them to the find include.


8. Kevin Apr 23, 2007 at 06:20

Just to echo the praise above, thank you! I too look forward to seeing an update in my RSS reader; keep up the great work.


9. Ryan Bates Apr 23, 2007 at 07:47

Thanks everyone for the encouraging comments.

@Ted: I think you'll only see a boost in performance if you're displaying the comment and user name for each task in the list. Each task will need an additional query to fetch the comments, and each comment would need another query to fetch the user name. If you're displaying a significant number of tasks with comments then this can get out of control. Eager loading will reduce this all to one big query with lots of joins.

That said, if you're just displaying the number of comments and not the entire comment content, you won't see much of a boost in performance with eager loading because it will be fetching more information from the database (the content of the comment) and not just the number of comments. I cover this problem in the next episode.


10. Pablo Apr 23, 2007 at 07:55

Excelete trabajo, disfruto mucho viendo estos railcast y espero muy entusiasmado cada vez que hay uno nuevo.
Felicitaciones !!!

Excelent Work, I enjoy a lot seeing those railscast, Im awaiting very enthusiastic each time that there is a new one.
Congratulation !!!


11. Chris G Apr 23, 2007 at 08:39

Again, very helpful!

Railscasts has recently become my favourite TV show, im not kidding! Awesome job Ryan!


12. Hugh Apr 23, 2007 at 09:47

Nice screencast.

Do you ever get name clashes with eager loading? IE, since a project and a task both have the column 'name', when using eager loading you might get a Invalid SQL exception.


13. Xilo32 Apr 23, 2007 at 10:01

Hey Ryan,

  Thank you for putting out great screencasts. Great work. Great quality. Awesome contribution to the community.


14. David Parker Apr 23, 2007 at 10:14

Ryan,

I really enjoy your screencasts. Keep up the great work!


15. Ryan Bates Apr 23, 2007 at 10:53

@Hugh,

Good point, I should have mentioned this in the screencast. You can avoid name clashes by specifying the table name in front of any mentioned column name. For example:

Topic.find(:all, :include => :project, :order => 'topics.name')

You can do the same in the conditions parameter and anywhere else.

Note to self: add some code formatting in comments.


16. Wandschrank Apr 23, 2007 at 13:40

You open my eyes. I am a better human now.


17. Luis Felipe Hurtado Apr 23, 2007 at 15:26

Como dice mi abuelita: "si vas a hacer algo, hazlo lo pero bien hecho". Este trabajo es excelente y de mucha calidad.


18. Daniel Steinsland Apr 23, 2007 at 16:27

I love Railscasts, it's a simple no fuzz approach to learning rails. I do wish I could watch the podcast on my Apple TV however. Just a thought. Keep up the good work!!!


19. Ryan Bates Apr 23, 2007 at 16:36

Railscasts are coming soon to an AppleTV or iPod near you.


20. Rebort Apr 23, 2007 at 16:52

Love the screencasts and the recent additions of the site.

Have you thought about doing screenshots or having sample code up next to each video?

The reason I ask -- I'm trying to remember something you did with a User model and the =|| operator, but I can't remember which video it was in!

Anyway -- thanks for sharing the knowledge and putting out a great site!


21. Ryan Bates Apr 23, 2007 at 16:58

@Rebort,

Great ideas, thanks. I'm trying not to add to the amount of work for each episode, but I'll give it some serious consideration.

The ||= operator is described in the very first episode on caching with instance variables.


22. Pepe Apr 24, 2007 at 00:04

Congratulations ;)
Ryan can you explain the link_to xxx, xxx_path(xxx)? I tried it and it didn't work.


23. Ryan Bates Apr 24, 2007 at 07:36

@Pepe, the "_path" method is called a named route. This is generated automatically when you are doing a RESTful design. I'll try to cover it in a future episode.

Great idea, thanks.


24. Shaliko Apr 24, 2007 at 08:26

Thank you!

Always great pleasure to studying your video. And always Malreaux them.


25. Jack Apr 24, 2007 at 16:54

Great cast as always!


26. Jack Apr 24, 2007 at 16:55

Have you ever considered getting more coders on board to contribute railcasts? I ask purely out of greed, I just can't get enough or RoR screencasts :)


27. RuMiL Apr 24, 2007 at 19:44

You rulez Ryan!
I love the railscasts, even when I don't speak english, I can understand and learn a lot more on every episode.

:)


28. Dougal Apr 24, 2007 at 21:32

Like many others, I get excited when ever my RSS reader shows there is a new screencast. Keep up the good work Ryan. Very helpful.


29. Ryan Apr 24, 2007 at 22:18

I enjoy the screencasts. Some of them I find to be a little too basic, but I often get something worthwhile out of them. I do, however, like the path you're taking--it seems like the topics are getting more and more in-depth.

As a future request, I would love to see a screencast or two on modules and mixins. I'm starting to get into meta-programming, but I find it really, really confusing in the beginning.

Keep up the good work!


30. J. Kyle Apr 24, 2007 at 23:43

Much thanks again and again and again, Ryan!


31. Warren Henning Apr 24, 2007 at 23:47

Just wanted to say that I appreciate you taking the time to make these screencasts.

Information delivered in little bite-sized chunks is often much more palatable than massive cram sessions.


32. Grant Neufeld Apr 25, 2007 at 05:31

Clear, concise, and very useful. Thanks for your excellent Railscasts.

What are you using to get the pretty-printed live display of your development.log in the Terminal?

What are you using to get command-key strokes displayed in that little popup near the top right?

Thanks!


33. Bryce Apr 25, 2007 at 08:20

Thanks for the great screencasts. It's like having another friend coding next to you. I've used a lot of your tips to clean up the code I'm working on and it's really helped out my development a ton.

Thanks,
Bryce


34. Ryan Bates Apr 25, 2007 at 08:25

@Jack, maybe someday I'll get more contributors to Railscasts, but at the moment I like to keep things simple and controllable.

@Ryan, an episode on modules and mixins is a great idea. I'll add it to my list. :)

@Grant, the live display of the development.log file is just coming from when I run "script/server", I guess mongrel is automatically set up to display this. If it's not displaying it for you, you can run this command (probably won't work on Windows)

tail -f log/development.log

The keystrokes are being displayed using KeyCastr.

http://stephendeken.net/software/keycastr/

Thanks for the encouraging comments folks!


35. Andy Apr 26, 2007 at 08:24

Simply wonderful. Your screencasts have become part of my rails learning toolkit, offering quick, clear and genuinely useful tips. Thanks.


36. Trueque Apr 27, 2007 at 02:04

Wonderful. This really better than any tutorial.

Thanks!!


37. Grant Neufeld Apr 27, 2007 at 08:09

Thanks for the answers. KeyCastr is a very good idea for screencasts.

I’ve been using Locomotive on Mac OS X for my Rails development, so I hadn’t been calling script/server directly. I had also just been opening my logs in Textmate or TextWrangler, so hadn’t realized there was pretty-printing that would pick up in tail.

Thanks again!


38. InMan Apr 27, 2007 at 10:19

I have code in helper:
  def page_methods(page)
    str = " "
    str += ( link_to blabla ) unless page.first?
    str += ( link_to blublu ) unless page.last?
    str
end

I found that first? and last? makes new query to database. How can i tell to use already made page query where i fetch all data.


39. Ryan Bates Apr 27, 2007 at 15:02

@InMan,

Are you using some form of pagination here? I can't tell why it is querying with just that code. I suggest starting a thread on http://railsforum.com instead of discussing it here. That way you can post more code. I'll try to respond there if I know the answer.


40. InMan Apr 28, 2007 at 04:07

I forgot to say that i am using plugins:
acts_as_tree
acts_as_list :scope => :parent_id


41. Horatio Apr 30, 2007 at 02:34

I'll say it again...this is awesome! I have been digging to find an example of this in practice for a while! More specifically the multiple levels of including. Great job!


42. supaspoida May 16, 2007 at 16:33

Just wanted to say that your Railscasts have been helping me tremendously while trying to learn rails. I've worked my way out of several tough spots to grasp using them, thanks!

I had one question about this episode though. I am working on a gallery type page for my portfolio, and I am using eager loading to call up categories, projects, and images all at once. I am wondering if there is a way to limit just one of the models returned? For example I only want to retrieve 5 projects per page. Where would I put the limit condition? This is what I have in my category model right now:

def self.portfolio
  find_all_by_parent_id(2, :include => { :projects => :images }, :conditions => 'images.default = true' )
end

Thanks!


43. Ryan Bates May 17, 2007 at 10:27

Very good question. AFAIK the limit clause in SQL is very limiting, and I don't think it's possible to do this with an SQL LIMIT (find :limit parameter). See if fetching all of the projects gives you decent performance. If not you could try not fetching the projects and doing a separate SQL query for each category, fetching the limited projects. This is a good way to go if you don't have many category models.

If you want to do this all in one query, I'm sorry I don't know how. I'm sure it's possible with a custom SQL query, but I'm not proficient enough with SQL to know.


44. sfty Aug 24, 2007 at 14:18

now that rails edge has query caching, eager loading may not always be the optimal solution, esp. with :select not working, one has to pull every column of all the tables :include'd.

see technoweenies "trick":
http://activereload.net/2007/5/23/spend-less-time-in-the-database-and-more-time-outdoors
http://pastie.caboo.se/63741

me thinks it would be really cool to have a follow-up on this episode. :)


45. sfty Aug 24, 2007 at 14:57

in context with my previous post, here I have another nice link concerning the problems with eager loading:

http://darwinweb.net/article/Free_Form_Manual_Eager_Loading


46. Matt Aug 27, 2007 at 08:49

Great episode! For anyone using eager loading on large databases, to get a huge performance boost you should set the column to index on the database of the attribute you're including, e.g. setting the task_id column in the projects table to be an index column using the sql command
ALTER TABLE `projects` ADD INDEX ( `task_id` )


47. Oscar Del Ben Oct 17, 2007 at 08:32

Thanks


48. shruthi Nov 11, 2007 at 17:25

Thank you very much. Its been of great help


49. smacak Nov 19, 2007 at 13:32

One word for you. Amazing! Thanks for these great screencasts. Keep up the good work.


50. Captain Zhan Mar 19, 2008 at 08:20

Simple but nice. Thank you for it.


51. Avishai Mar 27, 2008 at 20:58

Excellent tutorial :-)

Is it possible to get one piece of related data with out actually making a model for it? For example, I have Places and Countries, but I don't need/want a Country model, so I did Place.find(:all, :joins => 'LEFT JOIN countries ON places.country_id = countries.id'). The query executes, but I can't figure out how to access the resulting country name. Does anyone know?

Thanks!


52. Vasanth Sep 21, 2008 at 05:26

Thank you very much Ryan!


53. croolie Nov 07, 2008 at 11:06

You can do this on the fly in Lore (lore.rubyforge.org), and by configuration, even.

On the fly:

  User.all.join(Project).using(:project_id).entities

Or by configuration (eager load every time)

  model User < Lore::Model
    table :user, :public
    aggregates Project, :project_id
  end

User.all.entities # (now automatically joins referencesd project)


54. harini Dec 24, 2008 at 11:45

Your episodes help me a lot..You are simply great.


55. animal shaped rubber bands Jul 26, 2010 at 21:01

[url=http://www.toptoys2trade.com/power-balance-wholesale-2-c-40/ ]power balance[/url],


56. tiffany notes Jul 30, 2010 at 00:48

Great site. This could probably have the refactoring tag added t it.


57. sell silver serving sets Aug 03, 2010 at 03:56

I have been following you blog for quite some time now…..I just wanted to say how excited and happy I am for you. I can’t wait to see what you come up with.


61. financial coaching Aug 06, 2010 at 03:02

Great Buddy how you know about this in very details, it is really very interesting and knowledgeable things.


61. michaeljordanshoes Aug 08, 2010 at 20:12

I think it is a good title, it is so wanderful!


61. contractor bonds Aug 09, 2010 at 09:56

I generally do not post in Blogs but your weblog forced me to, amazing work.. beautiful.


61. ugg adirondack boot Aug 10, 2010 at 01:25

It is my pleasure to read this page,I look forward to reading more.


62. jordansneaker Aug 10, 2010 at 18:23

Hello kitty


63. free directory list Aug 11, 2010 at 22:05

I really appreciate what you post.


64. free directory list Aug 11, 2010 at 22:05

I really appreciate what you post.


65. free directory list Aug 11, 2010 at 22:06

I really appreciate what you post.


66. free directory list Aug 11, 2010 at 22:17

I think i have same problem too


67. emrah eren Aug 13, 2010 at 16:34

hello thank you good information


68. mbt shoes sale Aug 14, 2010 at 17:33

I don't think it's possible to do this with an SQL LIMIT (find :limit parameter). See if fetching all of the projects gives you decent performance.


69. ugg adirondack ii Aug 15, 2010 at 20:35

Good post.You did a good work,and offer more effective imformation for us!Thank you.


70. north face jackets Aug 15, 2010 at 20:35

This is a really quality post.I find this information through Google. Great job.


71. moncler store Aug 15, 2010 at 20:36

I am so excited about this page. Thanks for the wonderful post .


72. uggs sale Aug 15, 2010 at 20:37

I am the first time on this site and I am really enthusiastic about and so many good

articles. I think it’s just very good.


73. uggs sale Aug 15, 2010 at 20:37

Thank you very much! I am delighted of your lovely post.


74. air jordan retro 13 for sale Aug 19, 2010 at 22:38

Thanks for this excellent and simple way to use presenter objects. They're one of the best Rails resources I know and extraordinarily well prepared.


75. ugg adirondack Aug 23, 2010 at 00:01

I am so excited about this page. Thanks for the wonderful post .I bookmarked this site for further reading.


76. China Camcorder kaufen Aug 25, 2010 at 19:33

Efox-shop Grosshandel Tablet Laptop, Tablet PC, und Touchscreen Tablet Laptop in <b><a href="http://www.efox-shop.com/dc-camcorder-c-133.html">http://www.efox-shop.com/dc-camcorder-c-133.html</a></b>
Efox-shop Grosshandel<b><a href="http://www.efox-shop.com/dc-camcorder-c-133.html">China Camcorder kaufen</a></b>, Handy ohne Vertrag, Handy ohne Simlock,Sciphones i68, Touchscreen Handys und Handy Zubeh?re von Elektronik- Grosshandel in http://www.efox-shop.com/dc-camcorder-c-133.html


77. louis vuitton shoes Aug 26, 2010 at 23:13

Thanks for sharing your article. I really enjoyed it. I put a link to my site to here so other people can read it. My readers have about the same interets


78. snow boots Aug 31, 2010 at 01:59

I too look forward to seeing an update in my RSS reader; keep up the great work.


79. louis vuitton sunglasses Sep 01, 2010 at 21:15

Good article! Thank you so much for sharing this post.Your views truly open my mind.

Add your comment:

(SKIP THIS ONE)

(required)

(not shown)


(use pastie or gist for code)

sponsored by:
if you want to help:
required:
Get Quicktime Player
Give Back to Open Source