Railscasts - Custom Daemon

#129

Sep 29, 2008

Custom Daemon

Creating a custom daemon to handle background tasks is surprisingly simple. In this episode I will make a daemon to handle dynamically scheduled tasks.

Tags: plugins

Download (9.9 MB, 7:40)

alternative download for iPod & Apple TV (8.4 MB, 7:40)

Resources

sudo gem install daemons
script/plugin install git://github.com/dougal/daemon_generator.git
script/generate daemon mailer
RAILS_ENV=development lib/daemons/mailer_ctl start

# lib/daemons/mailer.rb
while($running) do
  mailing = Mailing.next_for_delivery
  if mailing
    mailing.deliver
  else
    sleep 15
  end
end

# mailings_controller.rb
def deliver
  Mailing.update(params[:id], :scheduled_at => Time.now)
  flash[:notice] = "Delivering mailing"
  redirect_to mailings_url
end

# models/mailing.rb
def self.next_for_delivery
  Mailing.first(:conditions => ["delivered_at IS NULL AND scheduled_at <= ?", Time.now.utc], :order => "scheduled_at")
end

def deliver
  update_attribute(:scheduled_at, nil)
  sleep 10 # placeholder for sending email
  update_attribute(:delivered_at, Time.now)
end

< Previous Episode

Next Episode >

39 comments

1. David Trasbo Sep 29, 2008 at 04:27

This solution looks great! It seems to be a good alternative to Starling and Workling. Thanks for the screencast.

2. AC Sep 29, 2008 at 04:33

Very interesting. Thanks!

3. Kevin Sep 29, 2008 at 05:05

Ryan,
As part of your series on background processing, have you considered covering daemon_controller (http://github.com/FooBarWidget/daemon_controller/tree/master) by the Phusion guys and how it might tie in with some of these other approaches?

Thanks and keep the great screencasts coming! :)

4. Ryan Bates Sep 29, 2008 at 07:17

@Kevin, I'm planning to cover Monit or God to help manage daemon processes in the near future. I may cover daemon controller there too. Thanks for bringing this up.

5. Jpemberthy Sep 29, 2008 at 08:21

thanks Ryan, I was using the OpenWFEru scheduler, but I preffer this method, this is exactly what I was looking for.

6. Wilson Sep 29, 2008 at 08:57

Ryan, nicely done. I'm very pleased to hear you are planning a future episode on monit and god. Thanks

7. fred c Sep 29, 2008 at 10:18

Great episode.

Just a quick question - does this method load the rails env when the job is picked up?

Thanks.

8. Douglas F Shearer Sep 29, 2008 at 13:10

Fred C - DG does load the rails env when the job is picked up. By default this is production, but you could change it to whatever you wished.

9. Ryan Bates Sep 29, 2008 at 13:24

@fred, the Rails environment will be loaded when the daemon starts. At that point it will stay loaded for each job.

10. Jon Sep 29, 2008 at 13:50

Hi Ryan. Thanks for great screencasts! I will need one of this solution for background processing, but I'm not sure witch one i should go with. My application will receive xml file via http upload. It's quit time critical data and it has to processed as soon as possible. I can have a daemon that process the files in a directory or go with the solution like workling/starling. /Jon

11. aleco Sep 29, 2008 at 14:37

Ryan, in case you're planning to add a followup:

1) using this method is tricky if you have more than one schedule, e.g. one every 15sec, one daily and one weekly schedule. Unless you can live with the memory load of having 3 separate daemons (which you usually don't want).

2) add a auto startup/shutdown to the deploy scripts, I guess stopping via :before_update_code might be appropriate.

12. Ryan Bates Sep 29, 2008 at 15:15

@Jon, if you need a job to be processed on demand then it sounds like Starling + Workling will be a good solution for you.

Another question to ask yourself is how frequently this task will be called. If you expect it to be only once an hour or less then you may want to try Backgroundjob as that will start/stop the Rails process each time. No need to have something constantly running in the background if you're not using it constantly.

@aleco, are you referring to recurring scheduled tasks? This solution was primarily designed for dynamically scheduled tasks where the frequency is not known up front. What you're referring to is definitely possible with one daemon, but the logic can get a little bit tricky in determining what task needs to be run at what given time.

You may want to look into BackgrounDRb for this. It has some cron-like scheduling which will work much better for this kind of thing.

Regarding the deployment, I honestly haven't tried doing this yet. Ideally you should just add a "lib/daemons/mailer_ctl restart" call to your deploy.rb restart task. I'm not certain if this works, if not then I think the fault is with the generator and should be fixed.

13. aleco Sep 29, 2008 at 15:32

Using "restart" will do the trick, but if you use the daemon for longer DB tasks, it might be a good idea to stop the daemon before deploying and restarting it afterwards. Because otherwise you might end up migrating a DB you're updating from the daemon at the same moment. This is why I suggested to use :before_update_code and :after_update_code.

Oh, another hint: most people will want to add

RAILS_DEFAULT_LOGGER.auto_flushing = 1

otherwise they will spend hours trying to find out why their production log isn't updated, assuming their daemon code is buggy. The explanation is that production log uses MAX_BUFFER_SIZE = 1000 and no auto flushing, meaning that it will only write to disk once it has 1000 bytes in buffer.

14. Bill Sep 29, 2008 at 15:37

Hi Ryan, thanks for the screencast.
Just an info. What about the simplest solution? I mean, what do you think to create a method in a given class and then class that method every X minutes by a cron job?
Is there any pros to go with a daemon or stuff like that instead of a cron job? (obviously for methods that can be called every X minutes)
What's the pros and cons of each approach?
(now i'm using a cron job :) )

15. Brian Sep 29, 2008 at 17:32

This came at a perfect time for me. I'm now successfully using this method but wanted to comment that I had a horrible time debugging my daemon. It seems the log file spits out a lot of confusing errors. Sample:

*** below you find the most recent exception thrown, this will be likely (but not certainly) the exception that made the application exit abnormally ***
#<MissingSourceFile: no such file to load -- action_mailer/ar_mailer_helper.rb>
*** below you find all exception objects found in memory, some of them may have been thrown in your application, others may just be in memory because they are standard exceptions ***
#<NoMemoryError: failed to allocate memory>
#<SystemStackError: stack level too deep>
#<fatal: exception reentered>
#<LoadError: no such file to load -- active_support>

The confusing part is that, at least for my env, all of those can be ignored. The more confusing part is that I couldn't figure out how to get a backtrace out of my daemon :( So I spent a good chunk of time on the errors shown in the log only to later realize they were red herrings. I ended up having to do a lot of inspecting of my daemon and finally got it working.

Anyway, it was frustrating. Hopefully the debug situation will get better? All that said, this is a great solution for the problem I had :-D So was worth the trouble.

16. Rong Sep 30, 2008 at 02:59

As you said in the "Advanced search" screencast it would be nice to clean up the search table from time to time. Is this a good way doing that? I mean shall I have a sleep time for about 24 hours?

17. Matt Sep 30, 2008 at 21:19

Thanks, Ryan! I've been using this for a couple of months now to do some pretty hairy background processing and it's been working well. I'm looking forward to seeing your screencast on monitoring these types of processes with God and Monit. Keep it up!

18. Rajesh Oct 03, 2008 at 05:44

Hi Ryan,

I get this error message wheen i try to start the Daemon on windows machine.

http://gist.github.com/14555

19. Nanda Oct 03, 2008 at 13:02

Ryan,
I was able to start the daemon fine, changed code while($running) loop and saw the changes. But now I can't stop or restart the daemon using lib/daemons/whatever_ctl stop command. I had to force kill the process from terminal to stop the daemon.

20. Carl Oct 03, 2008 at 21:51

@Rajesh,

As far as I know Daemons won't work on Windows, so you'd probably have to create a service and work through that (a much bigger problem than doing it the *nix way). I found this to give you a bit of help getting started, but really it should just tell you why it is better to use Linux or OS X for some things:

http://rubyforge.org/forum/forum.php?thread_id=14841&forum_id=320

21. Anthony Ettinger Oct 05, 2008 at 19:20

My lame ISP (dreamhost) won't let me install gems.

22. Rajesh Oct 06, 2008 at 05:56

#20

Thanks Carl.

23. Melvin Ram Oct 14, 2008 at 00:26

I'm wondering how much memory usage this approach uses. BackgroundRB is the method I've had experience with but it took like 140mb of memory to run. Any ideas on how this approach is on memory?

24. Melvin Ram Oct 14, 2008 at 12:02

I followed the instructions but when I start the daemon, I get this error:

http://pastie.org/292348

Any ideas on how I might get it working?

25. lolcatz Oct 15, 2008 at 07:22

@Melvin: i have the same issue...

26. Nick g Oct 15, 2008 at 08:17

I also have the same issue as Melvin and lolcatz. Running on linux.

All I put in the daemon code were puts statements and it doesn't run. I have installed all the gems it complains about.

27. lolcatz Oct 15, 2008 at 20:37

@melvin, @nick:
i completely removed daemons.yml and lib/daemons/gc_ctl.

here's my script/daemons and lib/daemons/gc.rb http://pastie.textmate.org/293636

removing options/daemons.yml solved my problem.

i know it's cheesy but works like a charm.

28. warhammer time card Oct 16, 2008 at 02:57

thanks bubby!

29. sthapit Oct 16, 2008 at 11:22

I'm unable to stop or restart the daemon. Anyone figure out how to fix this?

The log file also has this:
#<NoMemoryError: failed to allocate memory>
#<SystemStackError: stack level too deep>
#<fatal: exception reentered>
#<Errno::ENOENT: No such file or directory - /home/sthapit/workspace/rezz/log/mailer.rb.pid>

30. Melvin Ram Oct 22, 2008 at 23:09

lolcatz, that didn't work for me

31. mialsk Oct 28, 2008 at 23:04

why not check out <a href="http://www.3zoom.com">Warhammer gold</a> at 3zoom? i think it might be better.
<a href="http://www.3zoom.com">Warhammer online gold</a>,
<a href="http://www.3zoom.com/war-all/cheap-powerleveling-5-all.html">warhammer online powerleveling</a>,
<a href="http://www.3zoom.com/war-all/cheap-powerleveling-5-all.html">Warhammer Powerleveling</a> as well
<a href="http://www.3zoom.com/war-all/cheap-account-5-all.html">Warhammer online account</a> 
 <a href="http://www.3zoom.com/war-all/cheap-account-5-all.html">Warhammer online accounts</a> 
 <a href="http://www.3zoom.com/uswarhammer-all/warhammer-account-5-all.html">Warhammer account</a> 
 <a href="http://www.3zoom.com/uswarhammer-all/warhammer-account-5-all.html">Warhammer accounts</a>

32. Tam Kbili Nov 02, 2008 at 11:31

Thanks for posting the topic, it's great idea! I'm getting this in the log file smiliar to comments above, any ideas?
# Logfile created on Sun Nov 02 11:16:11 -0800 2008 by /
*** below you find the most recent exception thrown, this will be likely (but not certainly) the exception that made the application exit abnormally ***
#<Gem::Exception: can't activate activesupport (= 2.0.2, runtime), already activated activesupport-2.1.0>
*** below you find all exception objects found in memory, some of them may have been thrown in your application, others may just be in memory because they are standard exceptions ***
#<NoMemoryError: failed to allocate memory>
#<SystemStackError: stack level too deep>
#<fatal: exception reentered>
#<LoadError: no such file to load -- active_support>

33. Adam Nov 16, 2008 at 06:05

This is fits my scenario perfectly. I did run into a problem in using this with Capistrano however. I get daemon buildup over time. Every time I do a "cap deploy" I end up with a few more daemons running around.

Any idea why this might be happening?

http://pastie.org/316048

34. wow gold Nov 16, 2008 at 19:16

<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> Buy WAR gold </a>
<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> Cheap Warhammer Gold </a>
<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> Buy warhammer online gold </a>
<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> cheap warhammer online gold </a>
<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> Buy warhammer gold </a>
<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> buy warhammer online gold </a>
<a href="http://www.gamevive.com/Warhammer/cheap-Warhammer-gold/Warhammer-Anlec(Oceanic)-24-2093-Order.html"> buy cheap warhammer gold </a>
<a href="http://www.gamevive.com/Warhammer/Warhammer-Gold/Cheap-Warhammer-Gold.html
"> War Gold </a>
<a href="http://www.gamevive.com/Warhammer/Warhammer-Gold/Cheap-Warhammer-Gold.html
"> Buy Warhammer Gold </a>
<a href="http://www.gamevive.com/Warhammer/Warhammer-Gold/Cheap-Warhammer-Gold.html
"> Warhammer Online Gold </a>
<a href="http://www.gamevive.com/Warhammer/Warhammer-Gold/Cheap-Warhammer-Gold.html
"> Warhammer Gold for sale </a>
<a href="http://www.gamevive.com/Warhammer/Warhammer-Gold/Cheap-Warhammer-Gold.html
"> Cheap Warhammer Gold </a>
<a href="http://www.gamevive.com/Warhammer/Warhammer-Account/Cheap-Warhammer-Account.html
"> Warhammer Accounts </a>
<a

35. wow gold Nov 16, 2008 at 19:52

I'm looking forward to seeing your screencast on monitoring these types of processes with God and Monit.

36. warhammer time card Nov 17, 2008 at 21:54

<a href="http://www.bysneaker.com/"> Air Jordan Shoes</a> 
<a href="http://www.gmbar.com/">WoW Gold</a> 
<a href="http://www.gmbar.com/">WoW Power Leveling</a> 
<a href="http://www.gamevive.com/">WoW Gold</a> 
<a href="http://www.gamevive.com/">WoW Power Leveling</a> 
<a href="http://www.gmbar.com/"> Warhammer Time Card </a> 
<a href="http://www.gmbar.com/"> Warhammer CD Key </a> 
<a href="http://www.gmbar.com/cheap-war-gold.php"> Warhammer Gold </a> 
<a href="http://www.gmbar.com/cheap-war-gold.php"> Warhammer Online Gold </a> 
<a href="http://www.gmbar.com/cheap-war-gold.php"> Warhammer Accounts </a> 
<a href="http://www.gmbar.com/cheap-war-gold.php">Warhammer Power Leveling</a> 
<a href="http://www.gmbar.com/cheap-war-gold.php"> Warhammer Online Key </a> 
<a href="http://www.gmbar.com/cheap_war_gold.php"> WAR gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold.php"> warhammer gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold.php"> warhammer online gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold.php"> Buy WAR gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold.php"> Buy warhammer gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold.php"> Buy warhammer online gold </a> 
<a href="http://www.gmbar.com/war-accounts.php"> WAR Accounts </a> 
<a href="http://www.gmbar.com/war-accounts.php"> warhammer Accounts </a> 
<a href="http://www.gmbar.com/war-accounts.php"> warhammer online Accounts </a> 
<a href="http://www.gmbar.com/war-accounts.php"> Buy WAR Accounts </a> 
<a href="http://www.gmbar.com/war-accounts.php"> warhammer Accounts for sale </a> 
<a href="http://www.gmbar.com/war-power-leveling.php "> Warhammer Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling.php "> Warhammer Online Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling.php "> War Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling.php "> Buy Warhammer Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling.php "> Warhammer PowerLeveling </a> 
<a href="http://www.gmbar.com/war-cdkey.php"> Warhammer CD Key </a> 
<a href="http://www.gmbar.com/war-cdkey.php"> Warhammer online CD Key </a> 
<a href="http://www.gmbar.com/war-cdkey.php"> Warhammer Timecard </a> 
<a href="http://www.gmbar.com/war-cdkey.php"> Buy Warhammer Time Card </a> 
<a href="http://www.gmbar.com/war-cdkey.php"> Warhammer 60 days Time Card </a> 
<a href="http://www.gmbar.com/cheap-war-gold-eu.php"> Warhammer EU Gold </a> 
<a href="http://www.gmbar.com/cheap-war-gold-eu.php"> Warhammer EU Power Leveling </a> 
<a href="http://www.gmbar.com/cheap-war-gold-eu.php"> Warhammer EU CD Key </a> 
<a href="http://www.gmbar.com/cheap-war-gold-eu.php"> Warhammer EU Accounts </a> 
<a href="http://www.gmbar.com/cheap_war_gold_eu.php"> Cheap Warhammer Gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold_eu.php"> Cheap Warhammer online gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold_eu.php"> Buy Cheap Warhammer Gold </a> 
<a href="http://www.gmbar.com/cheap_war_gold_eu.php"> Buy WAR Gold </a> 
<a href="http://www.gmbar.com/war-accounts-eu.php"> Cheap WAR Accounts </a> 
<a href="http://www.gmbar.com/war-accounts-eu.php"> Cheap warhammer Accounts </a> 
<a href="http://www.gmbar.com/war-accounts-eu.php"> Cheap warhammer online Accounts </a> 
<a href="http://www.gmbar.com/war-accounts-eu.php"> Buy Cheap WAR Accounts </a> 
<a href="http://www.gmbar.com/war-power-leveling-eu.php"> Cheap Warhammer Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling-eu.php"> Cheap Warhammer Online Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling-eu.php"> Buy War Power Leveling </a> 
<a href="http://www.gmbar.com/war-power-leveling-eu.php"> Warhammer EU Power Leveling </a> 
<a href="http://www.gmbar.com/war-cdkey-eu.php"> buy Warhammer CD Key </a> 
<a href="http://www.gmbar.com/war-cdkey-eu.php"> buy Warhammer online CD Key </a> 
<a href="http://www.gmbar.com/war-cdkey-eu.php"> cheap Warhammer CD key </a> 
<a href="http://www.gmbar.com/war-cdkey-eu.php"> warhammer time card </a> 
<a href="http://www.gmbar.com/war-cdkey-eu.php"> Warhammer prepaid time card </a>

37. wow gold Nov 17, 2008 at 21:57

wow gold
http://www.gamevive.com/

warhammer gold
http://www.gmbar.com/

Air Jordan Shoes
http://www.bysneaker.com

warhammer power leveling
http://www.gmbar.com/war-power-leveling.php

Custom Daemon

39 comments

Add your comment: