#337 Capistrano Recipes pro
Get the most out of Capistrano by writing specific recipes with ERB templates. Here I show how to deploy to a blank VPS by running just a few Capistrano commands.
-
watch on iTunes
-
follow on Twitter
-
follow on Facebook
-
subscribe to RSS feed
Sometimes Chef just seems so heavy. Using Capistrano for provisioning would means one less tool to learn...
What problems does Chef solve that can't be addressed by Ryan's Capistrano approach? I know chef is good for large deployments, but for small/medium sized systems this Capistrano approach seems great.
This is my question too!
Does it make sense to invest time to learn another tool, when capistrano seems to be able to make any task on any number of servers?
Well I'll answer my own question. :-) I dug into Puppet last week - the tools it has for creating users, installing packages configuring systems etc would take a lot of time to rebuild from scratch in Capistrano. So I've become a Puppet user & believe its worth the time to learn. But this is a complicated tool, and it still seems like there is a need for a simpler deploy tool than Chef or Puppet.
Mixing capistrano and puppet/chef could be an option. As you said, creating users , installing packages and ensuring they stay installed are what tools like puppet/chef are made for. On the other hand configuring anything app specific i.e. nginx host, unicorn and so forth is easier using capistrano because it is more closely related to the application.
Wow, this is great. I usually use apache+passenger and a MySQL database. Tried to read up on the differences and benchmarks, but it's still not clear to me: is it just a matter of personal preference or is this setup (postgres, nginx, unicorn) really faster/better?
Significantly so.
I'm pretty curious myself. I'm getting ready to do some hard profiling on our multiple apps at work to try and find the best setup for performance.
Anyone have any benchmark information on using Unicorn/nginx over Passenger/Apache?
Unicorn is significantly faster than Passenger. A well-tuned MySQL install performs similarly to a well-tuned Postgres install.
Nginx is generally much simpler to setup than Apache. It's also faster and less resource hungry.
Apparently there’s only a significant advantage in performance if you’ve got fast clients, otherwise the opposite is the case: http://www.ruby-forum.com/topic/1822610
pg is transactional and just generally pretty nice.
Henk,
One think I learned, from watching a nodejs presentation actually (http://tv.adobe.com/watch/max-2011-develop/nodejs-and-you/) is that nginx is non-blocking whereas Apache is. The presenter covers this with a slide and illustration pretty early on, but nginx accepts the browser request, hands it to unicorn, then takes the next one whereas (he explains) Apache waits until the request is finished.
Hum, just wondering, I don't know nginx much, but as the unicorn socket has the app name in it, shouldn't the upstream be something like unicorn_<%= application %> so that there can be more than one ?
Looks like it is:
https://github.com/railscasts/episode-337/blob/master/blog-after/config/recipes/templates/nginx_unicorn.erb
upstream unicorn { server unix:/tmp/unicorn.<%= application %>.sock fail_timeout=0; }I think he means the:
upstream unicorn {to
upstream unicorn_<%= application %> {Otherwise you'd have a lot of "upstream unicorns" loaded. I wondered this myself, and was going to test it. If no one else posts, I'll tell what I find out.
From what I can tell, you would need to change it to include the application name.
Ok, thanks for testing that :-)
I found this blog post on GitHub about their transition from nginx/haproxy/mongrel to nginx/unicorn. Sounds intriguing.
https://github.com/blog/517-unicorn
A heads up that if you add '--yes' to the end of the add-apt-repository lines, it will not prompt you to add the repo in the default install on Ubuntu 11.10.
David,
Excellent, thank you for that.
For some reason when it runs the bootstrap it prompts for a sudo password since each command in the script is attempting to sudo, but does not accept my input when typing. (In fact, in my zsh term it displays the password as I type) Has anyone else had this issue? I enter it at the beginning as Ryan does when prompted; I would think it would be passed through. Ubuntu is the default server install in my case.
[xxx.xxx.xxx.xxx] sh -c 'rbenv bootstrap-ubuntu-11-10'[out :: xxx.xxx.xxx.xxx] [sudo] password for deployer:
I have the same issue, couldn't resolve it.
So I ran the command manually and commented it out.
It is because the script is using it's own sudo command:
https://github.com/fesplugas/rbenv-installer/blob/master/bin/rbenv-bootstrap-ubuntu-11-10
Capistrano has its own sudo helper which types in the password for you. Normally you can't pass input directly.
I just copied rbenv's bootstrap file and made it into one line so I can bootstrap myself.
I just ran into this problem again..
I ended up changing the script in recipes/rbenv.rb, the line that runs the bootstrap to:
run %q{sed "s/sudo/sudo -p 'sudo password: '/g" $HOME/.rbenv/plugins/rbenv-installer/bin/rbenv-} + rbenv_bootstrap + " | bash"I change the sudo to the format capistrano wants and run it through bash.
Thanks, that helped a lot!
Use visudo and edit the sudoers file. Make sure the admin or sudo group, which ever your deploy user is in, has the NOPASSWD option:
%sudo ALL=(ALL) NOPASSWD:ALL
You can add the following method to rbenv.rb:
Then you can just use the method to invoke rbenv wherever you need to ...
Hi guys,
does anyone know how to automate mysql installation?
I'm stuck when that blue screen pops up and asks for mysql password!
I found great resource here about streaming data from a capistrano run helper: http://errtheblog.com/posts/19-streaming-capistrano
Here's what you're looking for: https://github.com/cmer/shoestrap-example/blob/master/helpers/default#L162
You can use a preseed file to automate the mysql installation, Ubuntu (10.10)
you can use something like (not tested)
run "#{sudo} debconf-set-selections /tmp/mysql_preseed"
run "#{sudo} apt-get -y install mysql-server"
The preseed file itself (you can run this through erb) or replace
the erb code with your mysql password:
mysql-server-5.1 mysql-server/<%= lv_password %> <%= lv_password %>
mysql-server-5.1 mysql-server/<%= lv_password %> <%= lv_password %>
mysql-server-5.1 mysql-server/start_on_boot boolean true
I switched from capistrano to sprinkle, because sprinkle can be programmed to verify the deployment and I don't have to duplicate
capistrano recipes for each Rails project.
Made a mistake with the preseed file. Should read
mysql-server-5.1 mysql-server/root_password_again password <%= lv_password %>
mysql-server-5.1 mysql-server/root_password password <%= lv_password %>
mysql-server-5.1 mysql-server/start_on_boot boolean true
Hi Carl, gummybears - thx for help.
In the end I found the solution for installing mysql by sending the password via capistrano when the blue screen appears- its quite simple. Here is the solution:
Also, for those that are interested - user creation can also be automated via capistrano (that is the only thing Ryan did manually in this episode). You can check that at this link.
It's a bit complicated, but in the end you don't even have to ssh to machine prior to running capistrano recipes.
I hope this is useful to someone :)
Thanks this worked very nicely. :)
I recently released a very simple alternative to Chef if anyone's interested: https://github.com/cmer/shoestrap
Ryan,
Awesome episode, would you suggest using this approach to setup vagrant for development?
A little note: Instead of using the
set_defaultmethod here (or Capistrano's own_cset), a similar effect can be achieved by using the second parameter to thefetchmethod like so:directories_to_create = fetch(:directories_to_create, [])If I now use
set(:directories_to_create, Dir[...])that gets used, otherwise it uses the second parameter as the default (in this case just an empty array).Has anyone been able to get the nginx :install recipe to work on Debian Linux? There is no add-apt-repository command available.
You have to install the python-software-properties package ;)
Ryan allowed me to Gemify these recipes. Check it out here.
I added a few of my own changes such as setting up a firewall, setting the timezone to UTC and a task for tailing the logs. I plan on adding the Apache, Passenger and MySQL recipes from episode 335 as well.
I'm still trying to wrap my head around these concepts. Of the lot,I think your gem looks much more approachable. Would it be in any way possible to run this against a local vagrant install?
Ok, I'm getting there but now the problem is that it requires a password for the deployer.
hmmmm...I just put up a repo with scripts that run without the needing sudo on the deploy user, but mine aren't gemified.
https://github.com/mcmoyer/chef-istrano
Might have to follow suit and gemify them...that really makes them handy
Great episode, thanks Ryan.
One thing, maybe because I'm not familier with Capistrano, I'm curious about what will happen if I run "deploy:install" task twice, for example, in order to add a new server.
I assume Capistrano will try to install everything both to the new one and which has had those already.
If so, will it be OK(no problems as its consequence)? or otherwise is there anyway to avoid it, such like running a task on a specific server?
You can limit what servers will execute a task by passing a HOSTS environment variable like this:
Thanks Chris, now things look clear to me:)
Ryan thanks for the excellent deployment series.
I'm attempting to deploy to Linode following these recipes and get the following error when I try to cold deploy:
Suggestions in right direction will be greatly appreciated.
My guess is you are probably not using your id_rsa.pub key for the remote git repo. I had a different key set for a repo and got the same error. All I did was add my id_rsa.pub key to the remote git repo and everything worked fine. I think there is a way to also do
ssh-add, but I was less than successful with getting that to work.About once a month you more than make up for the $9. Very impressed good sir, very impressed. Thank you for everything!
I really like the Capistrano recipes approach to deployment. But as configured, the deployment only works from the master branch on the github repo. What steps would need to be taken to deploy from another branch, i.e. a staging branch, onto a staging app on the same linode?
You can set the branch. To deploy from a branch called
stage, I would doset :branch, "stage"Just in case someone is interested to know, in Ubuntu versions 11.10 (Oneiric Ocelot) and onwards the "admin" group is not going to be provided by default. The power that be are favouring the "sudo" group instead of "admin" group. If you're upgrading from a lower version, your "admin" group will persist but on a fresh install "admin" group will be missing.
So if you get an error that "admin" group does not exist, simply replace it with "sudo".
adduser deployer --ingroup sudoThank you, I was wondering why the group didn't exist.
If you check
cat /etc/sudoersyou might find that the "admin" group is there. In that case just rungroupadd adminand continue as usual.I use
cap ryan:batesto solve all my problems :)Thank you!
Do I really need an external git repository for this?
I'm not being able to find decent resources on a setup with only my machine and my vps...
can't get it to work...
I'm not using a clean clone of this, but when installing nginx it will simply stop waiting the user to press [ENTER] and... well, nothing more..
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com] Stable
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com]
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com] Stable version of nginx.
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com]
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com] More info: https://launchpad.net/~nginx/+archive/stable
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com]
** [out :: ec2-23-22-22-218.compute-1.amazonaws.com] Press [ENTER] to continue or ctrl-c to cancel adding it
I tried getting dropbox through capistrano there too but it doesn
t seem to be possible too because I'd need to stop dropboxd daemon after linking the accounts, doesnt seem to be possible...the screencast was great... I`m not very confident in the approach of using it to set up the VPS tho... it seems anything that will provide a prompt that's not to input text will just stop the scripts forever
or I could be doing something wrong, were you guys able to get the cap deploy:install to work?
me 2, same problem
here's the code I used on the task to get it to work!(found it around github..)
Nice man that works a treat
This worked for postgre on Ubuntu 12.04
task :install, roles: :db, only: {primary: true} do run "#{sudo} add-apt-repository ppa:pitti/postgresql",:pty => true do |ch, stream, data| if data =~ /Press.\[ENTER\].to.continue/ #prompt, and then send the response to the remote process ch.send_data(Capistrano::CLI.password_prompt("Press enter to continue:") + "\n") else #use the default handler for all other text Capistrano::Configuration.default_io_proc.call(ch,stream,data) end end run "#{sudo} apt-get -y update" run "#{sudo} apt-get -y install postgresql libpq-dev" endThis code repeats 3 times. I've made
press_entermethod:I don't like to press Enter every time, so I've used
ch.send_data( "\n")code.Then you can use it like that:
Just add
y...FYI: If you look in the source, Capistrano::CLI.ui just links to a Highline object. So you can do
Capistrano::CLI.ui.ask("Enter something")instead of CLI.password_prompt :)Add the following method to base.rb ...
Now use it in all recipes (3 in the example code) where 'add-apt-repository' is being used...
add_apt_repository 'ppa:nginx/stable'Thanks for for the screencast Ryan and thanks everyone for the helpful comments. I was able to deploy to AWS EC2 Ubuntu 11.10 (ami-baba68d3) using this script with a few minor changes.
A. Not sure if this is good practice but I used the default ubuntu user rather than creating a new user with admin rights. Couldn't get it to work with a new user, I think because AWS relies on keypair rather than password authentication for ssh. I suspect there is a better solution here, I'm not an expert on this.
B. In deploy.rb I used ssh_options to specify the location of my AWS keypair file:
C. Added the solution from Breno Santos Salgado (above) in postgres, nginx and nodejs recipes to prevent the deployment getting stuck when the system asks you to "Press [enter] to continue".
D. Changed Ubuntu version number on line two of rbenv.rb for the version of Ubuntu I used:
Thanks Ryan ....Anyone have a recipe for rmagik or imagemagik?
This worked for me on Ubuntu 12.04:
So I discovered an odd problem with using
Capistrano::CLI.password_prompt "PostgreSQL Password: "I spent most of the day trying to figure out why I was getting this error
FATAL: password authentication failed for user "DBuser"at the
bundle exec rake RAILS_ENV=production db:migratestage of my
cap deploy:cold.Turns out that when I use special characters when prompted for the db user password during
cap deploy:setup, the cold deploy fails. But if I use a password without any special characters then the cold deploy works.I don't know why this would be the case, so if anyone has any ideas as to why special characters can't be used, I'd love to be enlightened.
Thanks for mentioning this. I ran into the exact same thing.
Thanks Ryan, that's brilliant!
So, I've been struggling to figure out what is failing.. Everything installs and setups up fine...code is deployed to the correct location. Both unicorn and nginx are started, but when I hit the site all I get is the Welcome to nginx screen. I even doubled checked that the default site for nginx was gone...any ideas where I could look to figure out what is happening?
You forgot to delete the default site from nginx.
There are two options:
1. Do it manually:
the commands you have to enter:
ssh_to_your_server:
ssh deployer(or_which_user_u_have_on_server)@server_ipcommands_on_server:
sudo rm /etc/nginx/sites-enabled/defaultsudo service nginx restartto autostart it when server restarts:
sudo update-rd.d unicorn_yourapp_nameOr compare your nginx.rb script to (at minute: 7:27):
nginx.rb from RyanB
if its correct run:
cap deploy:installShould also have "set :normalize_asset_timestamps, false" in deploy.rb
Hi. Thanks for another awesome podcast. I have a task to solve. I need different configurations in "/config/recipes/templates/nginx_unicorn.rb" per host. How can I use different variables for e.g. server_name?
If you are getting a
gem not installederror, then do checkout the railscasts on zero downtime deployment. You need to send aUSR2 signaland setpreload_app=true. More details here at stackoverflowToday I had a big problem with deploying on remote machine. Many hours spent on problem with hanging Capitrano.
When I did
cap deploy:installall required packages were installed properly except one -ruby 1.9.3-p327.Capistrano hangs forever on executing
rbenv install 1.9.3-p327with messageInstalling ruby-1.9.3-p327....I've used second ssh connection with
toprunning for checking numer of users logged in. I've detected that messageInstalling ruby-1.9.3-p327...still exists after number of logged in users decrease from 2 into 1.What is the reason? SSH timeout session breaks connection.
How to solve the problem? My solution is:
1) Just edit file
/etc/ssh/sshd_configand add following lines:Parameter
ClientAliveIntervalshould have very small number to make pinging very often.18*100 = 1800 seconds means 30 minutes of session timeout (enough I think).
2) Then restart ssh service:
3) Try to deploy:
It works now for me :-)
If you not, additionally try to edit file
/etc/ssh/ssh_config:Then restart ssh service.
I hope it will be helpfull for somebody.
SSH Timeouts - avoid ssh disconnection with Capistrano:
1) Add file
config/recipes/ssh.rb2) Edit
config/deploy.rbfile and add:load "config/recipes/ssh"It works when you call:
or
NOTE 1 - you can call
cap ssh:installmany times with no worries that file/etc/ssh/sshd_configincludes stringClientAliveInterval 60many times.It's safe because following code:
NOTE 2 - Additionally I've made the same kind of protection in file
config/recipes/rbenv.erb. So now I can callcap deploy:installmany times (well, it's not recommended but sometimes it's usefull if you tests deploying process many times).should be:
Thanks :D this help me.
I ran in to problems with the rbenv install on Ubuntu 12.04. I changed the bootstrap line to version 12-04 and it still didn't work. I fixed it like this:
run "rbenv #{rbenv_bootstrap}"run "#{sudo} apt-get -y install python-software-properties build-essential"With Ubuntu 12.10 you need to install software-properties-common because add-apt-repository is not in python-software-properties.
You get the following message (for our googlers):
Anyone else having a problem when running cap deploy:setup and getting a chmod error :
failed: "sh -c 'chmod g+w /home/deployer/apps/blog /home/deployer/apps/blog/releases /home/deployer/apps/memDownDash/shared /home/deployer/apps/blog/shared/system /home/deployer/apps/blog/shared/log /home/deployer/apps/blog/shared/pids'" on 192.168.119.131
chmod:
"changing permissions of `/home/deployer/apps/blog/releases'
** [out :: 192.168.119.131] : Operation not permitted"
chmod:** [out :: 192.168.119.131] changing permissions of `/home/deployer/apps/blog'
** [out :: 192.168.119.131] : Operation not permitted
chmod:
** [out :: 192.168.119.131] changing permissions of `/home/deployer/apps/xxx/shared'
** [out :: 192.168.119.131] : Operation not permitted
chmod:
** [out :: 192.168.119.131] changing permissions of `/home/deployer/apps/blog/shared/system'
** [out :: 192.168.119.131] : Operation not permitted
chmod: changing permissions of `/home/deployer/apps/blog/shared/log': Operation not permitted
chmod: changing permissions of `/home/deployer/apps/blog/shared/pids': Operation not permitted
I setup a user in the admin group, and have "set :use_sudo, false"
While this tutorial is awesome the way it is, it falls flat at one thing that is more than not going to happen to you
Please address Capistrano
@ryanb could you see how to do that please.
In an act of shameless self promotion, I'd like you to promote http://pulsar.nebulab.it
You can use it to organize capistrano recipes and configurations. Maybe you'll find it useful.
First sign in through GitHub to post a comment.