Hey Ryan, great video once again. Thanks a lot for doing this, this has helped me out tremendously with answering questions and helping people get started.
Great information. I had to migrate in something like this:
change_column :users, :crypted_password, :string, :default => nil, :null => true
change_column :users, :password_salt, :string, :default => nil, :null => true
If I didn't, mysql would complain about a blank password.
If you find that your scheduled thinking sphinx task is running but your index isn't updating then you probably need to add PATH and possibly SHELL variables to the top of your crontab file because cron doesn't load your users ennvironment.
I've had nothing but problems implementing counter_cache in an ajaxy application. The changes are simply not propagated to the in-memory object as expected.
I have ended up going with a hybrid solution, where I use the counter cache only on pages where the count value cannot change (such as an index of articles with comment counts, explicitly using article.comments_count ). On pages where the count can change, such as an article detail page with a new comment form and an ajaxy updated comments block, I use article.comments.length.
Project has many Events,
Events HABTM Users,
Users HABTM Events.
In the Project view, I can successfully show the checkbox, but the checkbox name is definitely wrong and can't pass back correct params to update the Project.
In the console, I can confirm the models are set up correctly:
Please do an episode on the new nested forms patch, introduced in Rails 2.3. All examples I have seen so far are for 1-M associations. Please demonstrate for both 1-1 and 1-M. I guess it is really no difference apart from a hasOne instead of hasMany in the model, but just for completions sake, would be nice with both examples included ;)
Thanx for some great screencasts...
an issue i came across is when using nginx as a front for live server. nginx conf file is set to redirect all the /javascripts/ request to the public/javascripts folder.
this means the call to the dynamic files will never return the correct file.
gotcha : if you are using this with Rails 2.3.2, the request has been discreetly removed so it won't work, as per http://github.com/rails/rails/commit/41af606db385abe429888c5aca8b2e86c8830c24
To add it again manually, follow the instructions at: http://github.com/rails/request_profiler/tree/master
Good luck and thanks again to Ryan for these great contributions!
In Ryan's example, there is only one query executing in the background. In your example, you are doing 2 queries. On more heavy joins/queries, it will matter that you can do only one query and let the DB handle the optimisation.
Some asked about the :current argument passed to edit_user_path. It is used in the URL.
See http://api.rubyonrails.org/classes/ActionController/Routing.html and scroll down to Named Routes.
In this same example, if you replace
edit_user_path(:current)
with
edit_user_path(current_user.username)
Then the URL will be something like
http://localhost:3000/users/ryanb/edit
First of all: Thanks for the great tutorials. I was basically thrown into cold water concerning the use of ruby on rails and they help a lot.
But now I am having some unexplainable trouble with the "moving to partial" part of this tutorial...
Have a look at this thread:
http://railsforum.com/viewtopic.php?pid=103348#p103348
What's funny, is when the javascript broke, it didn't add the HTML to the bottom. However, the Ruby part worked, so the comment was created. Then, after the bug in the rjs was found, and you said it was working, the page only showed two comments but the comment count showed three, because it was pulling from the db. Then, when you loaded the page in Firefox (i.e. the page was refreshed) everything was working fine. Anyway, just a little nuance, and possibly a topic of client side and server side scripting/writing to the db.
Hi. I love the screencasts. However, I just watched the episodes on SQL injection and XSS, and it looked like your code was susceptible to those attacks. However, I could be totally wrong, because I am very new to RoR.
The above issues notwithstanding, you don't actually need to use JS to search for the DIVs you want to unhide.
Instead of setting "display: none" on every element, just add a class to it like "adminContent".
The main CSS file should specify that *.adminContent should have display: none.
It should also specify that a common parent, say body, when assigned an isAdmin class, should affect its children:
body.isAdmin *.adminContent {
display: block;
}
The dynamic JS would then need to simply add the "isAdmin" class to the body tag and the browser will take care of the rest due to CSS cascading and a higher specifity score of the above rule that unhides than the generic one that hides. It's also significantly faster than doing it manually with JS dom searching if you have a large DOM tree.
Glad to see more people paying attention to this technique! I've been presenting on this (which I've been calling progressive caching) for a few months now, and have a couple of blog posts:
I noticed that when I tried this and error came back saying the method "request.relative_url_root" was undefined. Apparently this was moved out of the request object and into the AbstractRequest object, but what I did instead was change
Thanks for the snippets everyone. I've forked Ryan's code and implemented the cookie solution: http://github.com/lemonbbq/railscasts-dynamic-page-caching/
It seems to work quite well as rails doesn't even get hit on the cached requests. Like the session cookie, the caching cookie doesn't have an expiry date set. This means when the user quits their browser, both cookies are cleared and the user is signed out normally.
Currently the user's name and admin status are stored in separate cookies. I'd like to combine these (possibly along with other attributes) into a single cookie using JSON. Everything works fine on the rails side, but I'm having problems getting JavaScript to unescape the JSON string - the spaces get replaced by plus signs. I'll have to work on this.
Integrating with Authlogic should be a breeze: one would simply need to add callbacks for after_save, after_destroy and after_persisting to set and destroy the caching cookie.
@Michael: You raise an interesting point about the performance hit from using cookies. From YUI's results it seems that only cookies larger 500 bytes really slow down the response time. Surely the increase in performance from page caching would outweigh the effect of slightly larger cookies. Though I agree that the increased complexity would be nice to avoid.
(responding to the concerns of some people, again longer since several commentators voiced concerns)
There seems to be a BIG misunderstanding about security in a web application.
Hiding (JS) code or links does not provide any security. Why?
Well, let's look at an analogy in the real world. This argument needs two steps.
1) Is your house more secure because you don't tell anyone you hid the key under the second big stone from the left?
2) If you just said yes, would you still say so if I change the scenario by saying "if there are hundreds of people searching around your house for a way in 24 hours a day, 7 days a week?
My point is, in a world of a village hiding links on the webpage is like hiding the key under the stone and may be "secure" enough in that environment.
On the Internet, where hundreds of thousands of automated (often hijacked) computers probe all reachable servers and services all the time (which does not mean they are all on *your* server at the same time, but some of them always are probing your systems!), this is NOT secure in ANY way.
On the Internet, the only way to achieve security is by securing the services themselves, not by hiding that those services exist.
In this context it means you add a before_filter where you check if there is a user with the appropriate rights logged in in the current session. If you do that you don't need to care if anyone finds your links. Finding admin-links in a Rails application is VERY easy - unless you deviate from standards such as common naming conventions and REST significantly - which in turn makes your life so much more difficult in the long run.
Perhaps this is my unfamiliarity with JavaScript showing, but I'd be concerned with the security of this -- what's to prevent somebody from using a client-side override to enable the hidden admin links? Or do you not care if the links become active because other checking will prevent them from working if the user isn't an admin?
@Brian: There won't be any flicker with THIS example in (even mildly) modern browsers.
Main reason: the Javascript is loaded from the HEAD part of the page, and that means it is fully loaded and executed BEFORE the page is loaded and rendered. Of course, because of the wait for the DOM-ready event in the Javascript the actual work done by the script happens at about the time the good old <body onLoad="..."> would execute, but there is so little to do for the script execution will be instantaneous for all practical purposes.
I guess you still could get a flicker, but only if the layout change was major, i.e. the page has a complicated layout with lots of elements in it, and a lot of areas on the page would be impacted. In THIS example only thus far empty areas are affected, no other elements already on the page change in any way.
(sorry for posting sooo muuuch, but I think it's valuable content???)
...and another excellent performance presentation, this one also from a YUI guy:
http://www.slideshare.net/natekoechley/high-performance-web-sites-2008 (Cookies starting on slide 57)
@Duncan - to "1": you cannot use session cookies at all from Javascript, since for security purposes they are sent as HTTP-only cookies (not just by Rails but by nearly all servers these days). If you want to access something from JS, you must set a "regular" cookie.
I used this technique two years ago for different reasons:
When you log out and go back in the browser using the page cache you go back to pages where you are logged in. The problem was I had pages loading dynamic content only accessible while logged in in the session, and going back to such a page would bring up the server-generated code for "logged in" from the browser's page cache, but the AJAX would produce a failure.
So I turned to the store-login-info-in-extra-cookie solution proposed in the comments above.
However, I removed the code again, among other things (like complexity) because of this...
The disadvantage of any cookies used as a replacement for browser-side storage is that they are uselessly sent back to thee server with each and every HTTP request. Apart from the bytes going over the wire, I believe I saw a presentation (from the YUI guys in the collection of great tech. videos in the YUI theater) where this was identified as having a significant (meaning noticeable overall) performance impact. One of the 14 main pieces of advice for website performance is to reduce the number of cookies as much as possible.
I suggest not to use cookies for client-side storage, as tempting as it is. There already are tools (e.g. a module in Dojo, persist-js, a module planned for by and for YUI in the near future) trying to give a consistent API for the many client-side storage implementations (ranging from Flash to HTML5-type global and local storage in Firefox 3(.5)).
The problem with hacks is they tend to persist and byte you later, when you forgot you had implemented them. So let's use *real* client-side storage if you must, especially since the problem you are trying to solve is not THAT pressing.
Thanks Ryan...great screencast. I'm curious how this looks in production. Is there a slight "jump" or flicker to the page as it appears to have loaded, and then a second update is made? Or is not noticeable. Thanks!
Brian
One small suggestion to Pete's suggested script. I move the find script that inserts .gitignore into blank dirs BEFORE running git init. This avoids placing .gitignores into the .git directory. Harmless but not very clean.
Here's the gist
http://gist.github.com/141509
@Pawel, Storing the username in a cookie and parsing it in JavaScript is a great idea, although it is quite specific to this solution. Here I try to give a more generic solution that will work for any kind of dynamic content (such as something that may not be dependent on the current user's actions).
I'll play around with this a bit more and possibly cover the cookie solution in a future episode.
Andy: Search engines? Other than that I don't care about it either.
Meanwhile, if you don't care about customer not having JS why do you need page caching in the first place? You can make all links as JS calls and reload just required part(s) of the page (http://vh-daf.ru/service -- it is in Russian, but try to click on the top submenu and you get the idea). Just because of 0.5% (or less) weirdos who still want to press reload button? :)
In fact on this site you can disable JS and still get (almost) the same result, but I would not say it is easy for more complicated cases and I have done non-JS links for Google, not for humans.
A bit confusing topic -- hard to realize where the border line is between traditional and JS design.
Perfect timing!!
Thanks Ryan
Hey Ryan, great video once again. Thanks a lot for doing this, this has helped me out tremendously with answering questions and helping people get started.
Wonderful wonderful screencast!
I would also like to see a screen cast on being an openID provider
Great information. I had to migrate in something like this:
change_column :users, :crypted_password, :string, :default => nil, :null => true
change_column :users, :password_salt, :string, :default => nil, :null => true
If I didn't, mysql would complain about a blank password.
We've seen quite a few episodes about authentication, I think that authorization is an important followup (it's a little daunting to me).
Am i the only dumb noob who can't get this working...
i did everything as shown in the vid but when i send a new search i fail with this error message:
Missing template products/_product.erb in view path app/views
Your timing couldn't have been better as I was just about to play around with OpenID for Authlogic today. Thanks a lot ;)
Hi, Ryan
Thank you for another one god screencast!
Had you any experience with single sign on with Shibboleth? Is it makes sense to do screencast about it?
Thanks,
Igor
email fetch works with yahoo openid?
Awesome as always.
Keep rocking.
Hi Ryan,
Amazing video... Keep it up.
Thanks,
Madan Kumar Rajan
If you find that your scheduled thinking sphinx task is running but your index isn't updating then you probably need to add PATH and possibly SHELL variables to the top of your crontab file because cron doesn't load your users ennvironment.
I've had nothing but problems implementing counter_cache in an ajaxy application. The changes are simply not propagated to the in-memory object as expected.
I have ended up going with a hybrid solution, where I use the counter cache only on pages where the count value cannot change (such as an index of articles with comment counts, explicitly using article.comments_count ). On pages where the count can change, such as an article detail page with a new comment form and an ajaxy updated comments block, I use article.comments.length.
Solution for #67: http://ho.race.hk/blog/?p=143
Project has many Events,
Events HABTM Users,
Users HABTM Events.
In the Project view, I can successfully show the checkbox, but the checkbox name is definitely wrong and can't pass back correct params to update the Project.
In the console, I can confirm the models are set up correctly:
>> proj = Project.find(:first)
>> proj.events[0].user_ids
=> []
>> proj.events[0].user_ids = [1,2]
=> [1, 2]
however, how can I name the check_box_tag?
The following returns an unknown key: user_ids error when submit the form:
<%= check_box_tag "proj[event_ids][user_ids][]", user.id, event.users.include?(user) %>
sorry, I meant in the NEW Action you need to add that
I found the solution to the create action error: line 17 nil.comments.
In the create action you need to add:
def new
@commentable = commentable_find
@comment = Comment.new
end
How do you automatically add new files? What is the point of adding a directory if the directory isn't monitored for new files?
Hi Ryan,
Please do an episode on the new nested forms patch, introduced in Rails 2.3. All examples I have seen so far are for 1-M associations. Please demonstrate for both 1-1 and 1-M. I guess it is really no difference apart from a hasOne instead of hasMany in the model, but just for completions sake, would be nice with both examples included ;)
Thanx for some great screencasts...
an issue i came across is when using nginx as a front for live server. nginx conf file is set to redirect all the /javascripts/ request to the public/javascripts folder.
this means the call to the dynamic files will never return the correct file.
What can we do in this case?
gotcha : if you are using this with Rails 2.3.2, the request has been discreetly removed so it won't work, as per http://github.com/rails/rails/commit/41af606db385abe429888c5aca8b2e86c8830c24
To add it again manually, follow the instructions at: http://github.com/rails/request_profiler/tree/master
Good luck and thanks again to Ryan for these great contributions!
@Mike,
In Ryan's example, there is only one query executing in the background. In your example, you are doing 2 queries. On more heavy joins/queries, it will matter that you can do only one query and let the DB handle the optimisation.
Some asked about the :current argument passed to edit_user_path. It is used in the URL.
See http://api.rubyonrails.org/classes/ActionController/Routing.html and scroll down to Named Routes.
In this same example, if you replace
edit_user_path(:current)
with
edit_user_path(current_user.username)
Then the URL will be something like
http://localhost:3000/users/ryanb/edit
hello i am using this cast for my application but am getting some error near
prawnto :prawn => { :top_margin => 75 }
like undefined method `prawnto'
if dont put prawnto it gave me missing layout error
please help me
Hey guys, I'm getting a:
undefined method `record' for #<User:0xb68aa708>
error. Is this cause I'm on Rails 2.2.2 still?
/usr/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/attribute_methods.rb:260:in `method_missing'
btw - awesome screencast as usual!
Thanks!
First of all: Thanks for the great tutorials. I was basically thrown into cold water concerning the use of ruby on rails and they help a lot.
But now I am having some unexplainable trouble with the "moving to partial" part of this tutorial...
Have a look at this thread:
http://railsforum.com/viewtopic.php?pid=103348#p103348
What's funny, is when the javascript broke, it didn't add the HTML to the bottom. However, the Ruby part worked, so the comment was created. Then, after the bug in the rjs was found, and you said it was working, the page only showed two comments but the comment count showed three, because it was pulling from the db. Then, when you loaded the page in Firefox (i.e. the page was refreshed) everything was working fine. Anyway, just a little nuance, and possibly a topic of client side and server side scripting/writing to the db.
Hi. I love the screencasts. However, I just watched the episodes on SQL injection and XSS, and it looked like your code was susceptible to those attacks. However, I could be totally wrong, because I am very new to RoR.
Anyway, I love your simple and RESTful solutions.
Is this better than using a daemon, or just an alternative? And does this create a crontab conflict if you have several apps using the same gem?
How do you find all Products that do not have any categories?
I have a query that works but I'd like to have that in a named_scope.
Any ideas?
img_shells_with_out_users = ImageShell.find_by_sql 'SELECT * FROM image_shells INNER JOIN image_shells_users ON (image_shells_users.image_shell_id!=image_shells.id)'
Hey Ryan,
Great episode, as usual.
I was wondering how much of this you can achieve with the recently added support for caches_action ..., :layout => false.
Regards
The above issues notwithstanding, you don't actually need to use JS to search for the DIVs you want to unhide.
Instead of setting "display: none" on every element, just add a class to it like "adminContent".
The main CSS file should specify that *.adminContent should have display: none.
It should also specify that a common parent, say body, when assigned an isAdmin class, should affect its children:
body.isAdmin *.adminContent {
display: block;
}
The dynamic JS would then need to simply add the "isAdmin" class to the body tag and the browser will take care of the rest due to CSS cascading and a higher specifity score of the above rule that unhides than the generic one that hides. It's also significantly faster than doing it manually with JS dom searching if you have a large DOM tree.
Can you give guidelines for how you would unit-test this code?
For example, can you use a local file URL to ensure it properly parses and stuffs in the database various test data?
Thanks.
Glad to see more people paying attention to this technique! I've been presenting on this (which I've been calling progressive caching) for a few months now, and have a couple of blog posts:
http://www.culann.com/2009/04/progressive-caching
and
http://www.viget.com/extend/progressive-caching-in-depth/
Is there a way to install curl on windows? does feedzirra work with windows or not ?
I noticed that when I tried this and error came back saying the method "request.relative_url_root" was undefined. Apparently this was moved out of the request object and into the AbstractRequest object, but what I did instead was change
#{request.protocol + request.host_with_port + request.relative_url_root + request.path}"
to (on line 151 of open_id_athentication.rb)
#{request.protocol + request.host_with_port + request.request_uri()}"
and I got around that error message
i tried this example but gave me error like
530 5.7.0 Must issue a STARTTLS command first. g25sm12608420wag.43
please give me some solution
Thanks for the snippets everyone. I've forked Ryan's code and implemented the cookie solution: http://github.com/lemonbbq/railscasts-dynamic-page-caching/
It seems to work quite well as rails doesn't even get hit on the cached requests. Like the session cookie, the caching cookie doesn't have an expiry date set. This means when the user quits their browser, both cookies are cleared and the user is signed out normally.
Currently the user's name and admin status are stored in separate cookies. I'd like to combine these (possibly along with other attributes) into a single cookie using JSON. Everything works fine on the rails side, but I'm having problems getting JavaScript to unescape the JSON string - the spaces get replaced by plus signs. I'll have to work on this.
Integrating with Authlogic should be a breeze: one would simply need to add callbacks for after_save, after_destroy and after_persisting to set and destroy the caching cookie.
@Michael: You raise an interesting point about the performance hit from using cookies. From YUI's results it seems that only cookies larger 500 bytes really slow down the response time. Surely the increase in performance from page caching would outweigh the effect of slightly larger cookies. Though I agree that the increased complexity would be nice to avoid.
About Security
(responding to the concerns of some people, again longer since several commentators voiced concerns)
There seems to be a BIG misunderstanding about security in a web application.
Hiding (JS) code or links does not provide any security. Why?
Well, let's look at an analogy in the real world. This argument needs two steps.
1) Is your house more secure because you don't tell anyone you hid the key under the second big stone from the left?
2) If you just said yes, would you still say so if I change the scenario by saying "if there are hundreds of people searching around your house for a way in 24 hours a day, 7 days a week?
My point is, in a world of a village hiding links on the webpage is like hiding the key under the stone and may be "secure" enough in that environment.
On the Internet, where hundreds of thousands of automated (often hijacked) computers probe all reachable servers and services all the time (which does not mean they are all on *your* server at the same time, but some of them always are probing your systems!), this is NOT secure in ANY way.
On the Internet, the only way to achieve security is by securing the services themselves, not by hiding that those services exist.
In this context it means you add a before_filter where you check if there is a user with the appropriate rights logged in in the current session. If you do that you don't need to care if anyone finds your links. Finding admin-links in a Rails application is VERY easy - unless you deviate from standards such as common naming conventions and REST significantly - which in turn makes your life so much more difficult in the long run.
Perhaps this is my unfamiliarity with JavaScript showing, but I'd be concerned with the security of this -- what's to prevent somebody from using a client-side override to enable the hidden admin links? Or do you not care if the links become active because other checking will prevent them from working if the user isn't an admin?
@Brian: There won't be any flicker with THIS example in (even mildly) modern browsers.
Main reason: the Javascript is loaded from the HEAD part of the page, and that means it is fully loaded and executed BEFORE the page is loaded and rendered. Of course, because of the wait for the DOM-ready event in the Javascript the actual work done by the script happens at about the time the good old <body onLoad="..."> would execute, but there is so little to do for the script execution will be instantaneous for all practical purposes.
I guess you still could get a flicker, but only if the layout change was major, i.e. the page has a complicated layout with lots of elements in it, and a lot of areas on the page would be impacted. In THIS example only thus far empty areas are affected, no other elements already on the page change in any way.
(sorry for posting sooo muuuch, but I think it's valuable content???)
...and another excellent performance presentation, this one also from a YUI guy:
http://www.slideshare.net/natekoechley/high-performance-web-sites-2008 (Cookies starting on slide 57)
I found the presentation about the impact of cookies on website performance:
http://yuiblog.com/blog/2007/03/01/performance-research-part-3/
@Duncan - to "1": you cannot use session cookies at all from Javascript, since for security purposes they are sent as HTTP-only cookies (not just by Rails but by nearly all servers these days). If you want to access something from JS, you must set a "regular" cookie.
I used this technique two years ago for different reasons:
When you log out and go back in the browser using the page cache you go back to pages where you are logged in. The problem was I had pages loading dynamic content only accessible while logged in in the session, and going back to such a page would bring up the server-generated code for "logged in" from the browser's page cache, but the AJAX would produce a failure.
So I turned to the store-login-info-in-extra-cookie solution proposed in the comments above.
However, I removed the code again, among other things (like complexity) because of this...
The disadvantage of any cookies used as a replacement for browser-side storage is that they are uselessly sent back to thee server with each and every HTTP request. Apart from the bytes going over the wire, I believe I saw a presentation (from the YUI guys in the collection of great tech. videos in the YUI theater) where this was identified as having a significant (meaning noticeable overall) performance impact. One of the 14 main pieces of advice for website performance is to reduce the number of cookies as much as possible.
I suggest not to use cookies for client-side storage, as tempting as it is. There already are tools (e.g. a module in Dojo, persist-js, a module planned for by and for YUI in the near future) trying to give a consistent API for the many client-side storage implementations (ranging from Flash to HTML5-type global and local storage in Firefox 3(.5)).
The problem with hacks is they tend to persist and byte you later, when you forgot you had implemented them. So let's use *real* client-side storage if you must, especially since the problem you are trying to solve is not THAT pressing.
Thanks Ryan...great screencast. I'm curious how this looks in production. Is there a slight "jump" or flicker to the page as it appears to have loaded, and then a second update is made? Or is not noticeable. Thanks!
Brian
Thanks for Posting . Fine Job
One small suggestion to Pete's suggested script. I move the find script that inserts .gitignore into blank dirs BEFORE running git init. This avoids placing .gitignores into the .git directory. Harmless but not very clean.
Here's the gist
http://gist.github.com/141509
@gUI, yes.
@Pawel, Storing the username in a cookie and parsing it in JavaScript is a great idea, although it is quite specific to this solution. Here I try to give a more generic solution that will work for any kind of dynamic content (such as something that may not be dependent on the current user's actions).
I'll play around with this a bit more and possibly cover the cookie solution in a future episode.
Andy: Search engines? Other than that I don't care about it either.
Meanwhile, if you don't care about customer not having JS why do you need page caching in the first place? You can make all links as JS calls and reload just required part(s) of the page (http://vh-daf.ru/service -- it is in Russian, but try to click on the top submenu and you get the idea). Just because of 0.5% (or less) weirdos who still want to press reload button? :)
In fact on this site you can disable JS and still get (almost) the same result, but I would not say it is easy for more complicated cases and I have done non-JS links for Google, not for humans.
A bit confusing topic -- hard to realize where the border line is between traditional and JS design.
It's a technique. You may need this in situations where cookies/JS is not a viable or elegant solution. Calm down.