Michael van Rooijen's Profile

Sidekiq runs as a separate process like Resque, Delayed Job, Qu, QueueClassic, etc. GirlFriday runs in the same process as your app server.

You'll want to provision the server(s) with Chef (or similar tools like Puppet) to prepare it for deployment. Then use Capistrano to do the actual deployment(s).

^ pro-tip.

Heroku is good in a lot of situations. However, what disappoints me a little is the lack of JRuby and Rubinius support. This is where EngineYard shines a little more. A lot of Ruby developers probably aren't aware how much more concurrency you can gain using a VPS running JRuby/Rubinius with Puma/Trinidad/TorqueBox rather than running Thin or Unicorn.

For example, TorqueBox (a JRuby-specific App Server) is a "real" application server that runs independently (it's practically a whole infrastructure, not something that just binds to a port), and you deploy your application to this app server. This one single instance uses threads (JVM, not MRI) for everything you would normally have to spin up processes for with MRI (which translates to Dynos on Heroku).

There have been people that took their worker farm (120 worker processes) from around 60gb in memory usage (yes, 60 gigabyte) spread across like 10 dedicated servers, down to about 1gb memory usage on a single server by switching from process-based scaling to thread-based with JRuby or Rubinius. That is huge. Not only does it scale well, you reduce a lot of costs, the infrastructure becomes more simple and imagine running 120 workers on Heroku, the bill goes through the roof. It really makes me wonder why there aren't more people supporting JRuby and Rubinius. (Note that this also applies to concurrent web requests of course, not just workers).

However, I definitely agree you when it comes to being more at ease when deploying to Heroku, simply because you cannot do any administrative tasks. When something breaks, needs updating, and what not, Heroku is the one taking care of it all while you sleep. It mainly depends on your application's requirements. If you have a basic app that doesn't really need a lot of concurrency (be it in the web process, or worker process) then it's all good. However, once you have use cases where you run large worker farms or tremendous amounts of traffic, it might change your game completely.

These are all interesting discussions, I guess basically it comes down to picking the right tools for the job and dealing with certain tradeoffs on each side. But it'd be good to let everyone know it's not just MRI, Thin and Unicorn out there.

What'd I'd love to see from Heroku is (real) support for JRuby (now) and Rubinius (once RBX hits 2.0), perhaps 1GB of ram per Dyno (at a higher price) and the ability to use TorqueBox flawlessly. I think this would be a total game changer for deployments and power per Dyno.

tl;dr:

• Check out Ruby Implementations:
  • JRuby
  • Rubinius
• Check out thread-enabled app servers:
  • Puma
  • Trinidad
  • TorqueBox (this is pretty much one-of-a-kind, a beast, not comparable to any other Ruby app server to date)
• @Heroku: Please add real support for JRuby and TorqueBox.

While I like how the #delay method looks in the code, automatically serializing/deserializing objects, it does feel somewhat dirty. I recall it being fairly slow when working with large objects, and in most cases I think providing a hash of options along with the model's id would be faster. But, please do correct me if I'm wrong about that. I think that the less data you store in the database, faster the retrieval and serialization/deserialization will be when just using Struct classes, but it does mean you need to write a bit more code for everything you want to queue up. And, if you're going to do that then you might want to consider using Resque instead. I've used both a few times, and they both have their pros and cons. Note: I haven't tried out the new Delayed Job 3 that was recently released.