Please read for an updated status on RailsCasts:
Learn more or hide this
Nice introduction to QC - thanks heaps. What's with that video poster though? ;-)
I'm definitely considering the jump to Postgres, it would be really nice if there was a PG engine for Faye.
So, the only advantage of QC over DJ is that in uses some internal postgres hooks and doesn't hit database every n seconds like DJ, right?
In that case DJ is still far more advanced and feature-rich and i see no reasons considering QC over it.
While I admit that QC does not have all of the features that delayed jobs boasts, I would also admit that it does not have all of the shortcomings as well. QC takes job locking quite serious and has proven to be more robust in high-volume scenarios. For instance, take a look at the PL/pgSQL functions that queue_classic uses to lock a job. All of the code in that function is tuned to ensure quick access to jobs while guaranteeing that no two workers can lock the same job. While the chance that delayed job will allow multiple workers to lock the same chance is low, it will happen eventually --especially in production environments that churn through tens of millions of jobs per day
Also, as Ryan Bates mentioned, queue_classic is not dependent on active_record. This means a whole lot when you want to write simple ruby programs that utilize a message queue.
Finally, we use queue_classic at Heroku because we have production systems that have daily message throughput in the tens of millions. We needed a queuing system that was reliable and easy to reason about; queue_classic satisfied both of these conditions.
Feel free to reach out if you have any more questions on how queue_classic might be able to help you out. @ryandotsmith or email@example.com
Awesome ! but, anyone can tell if queue_classic load the whole application in the workers ? I'm running on a small VPS and it's hard to have 80mb of RAM used by background job workers... just for send registration email (like DJ)
Answer to your question is in the screencast (between 6:30-6:50) and QC documentation; simply write your own worker "binary" (weird to use that term for a script file...) that only loads what's needed.
I'm experiencing issue with require "your_app"
what it should be required? the name of my application does not work
It must run with rake or "./bin/worker" ? (both does not work)
Ok so here we need to require mailers classes & action_mailer only, work great
You mentioned that the main advantage of queue_classic over something like resque would be that you don't need a daemonized process running all the time. I'm curious what you would use to poll the queue then? Wouldn't you daemonize the rake task you had running throughout the screencast? Or would you run that or the lightweight worker script from cron?
resque needs redis running all the time. that's the daemon in question. queue_classic only needs the database.
Really nice and handy episode (as usual !), keep up the good work.
Another queue system worth trying is sidekiq, it's quite close of resque but doesn't spawn a process per worker. It uses celluloid under the hood that would worth an episode too !
I wonder how easy it would be to use this with Apartment. Apartment currently supports DJ. Also, it would be cool to see an episode on Apartment. It'd fit right into the current Postgres theme.
I don't see any PostgreSQL specific features here.
MySQL and any another RDBMSes can be used to achieve the same thing AFAICT.
PostgreSQL specific features used include: SELECT FOR UPDATE NOWAIT and LISTEN/NOTIFY. Take a look at the PL/pgSQL function that is called to lock a job. https://github.com/ryandotsmith/queue_classic/blob/master/sql/ddl.sql
SELECT FOR UPDATE NOWAIT
Cool. Oracle also supports SELECT FOR UPDATE NOWAIT. It also has a feature named Database Change Notification that is similar to LISTEN/NOTIFY.
Ah yes. Oracle.... Oracle certainly has these features and more. However, I typically exclude Oracle from comparisons when talking about Ruby related programming. Oracle has so many features but they are out of reach for simple projects that exist in a non-enterprise environment.
Does loading the QC functions once with a migration work for dev environments? We have it set up so we drop and reload them around db:schema:load and load them after db:test:load.
We are working to add support to allow symbols in job params (e.g. hash keys or values) when enqueuing jobs in QC (by turning them into strings before encoding with OkJSON).
We have a queue for sending emails, so we'll have to investigate using workers that aren't running the full environment for those to save memory. Great idea.
One thing to watch out for when using QC (or any background job manager) with Rails is jobs enqueued from after_create / after_save callbacks that use the new record. We've found that you must use after_commit instead to ensure that the record has been committed to the DB before the worker tries to find it by ID.
You are right about your input/comment - One thing to watch out for when using QC (or any background job manager) with Rails is jobs enqueued from after_create / after_save callbacks that use the new record. We've found that you must use after_commit instead to ensure that the record has been committed to the DB before the worker tries to find it by ID.
I am interested to see Ryan Smiths response or comments on this too.
How does one use queue_classic with rubber/capistrano? It seems like rake qc:work needs to run, just not sure where to put that command (in deploy.rb?) Thanks!
I needed a:
ENV["QC_DATABASE_URL"] = "postgres://localhost/mailer_development" if Rails.env.development?
ENV["DATABASE_URL"] = "postgres://localhost/mailer_development"
The two differences:
1) Only set the url in development so when I deploy to Heroku the production database url gets picked up.
2) Use QC_DATABASE_URL instead of DATABASE_URL. Apparently Rails actually uses DATABASE_URL instead of database.yml if DATABASE_URL is present. This will cause your tests to start failing, despite the Rails.env.development? guard, due to the initializer running prior to Rails.env getting set to "test". This is confusing, but put a puts Rails.env into the initializer and watch the output when you run rake test and you'll see "development" and then "test" as Rails runs your tests.
Hope that helps someone. Great Railscast as always Ryan!
thanks! ENV["DATABASE_URL"] = "postgres://localhost/mailer_development" doesn't work for me as well.
but i get "undefined method `load_functions' for QC::Queries:Module" when trying to migrate, why is this?
take a look at :
I think the names have changed to :
Thanks Yoda! I have this problem and also my postgres installation is kind of weird so was all messed. but thanks to you i can narrow down the prob and its now fixed!
How can I use queue_classic to send emails? I can pretty much make queue_classic call any method and Queue it successfully, but when it comes the time to call an ActionMailer method or a class method that wraps an ActionMailer call, it simply Doesn't work!
is there an additional configuration needed?
PS. I can see the method being queued in the queue_classic_jobs table and then being removed, but it basically does nothing(does not send the e-mail) :(
Sure I have used it to send email, and it works great.
It was pretty simple. The send_confirmation looks up the user, and then gets their email address and whatever other information needed.
You may want to enable delivery errors in your development.rb config file. Maybe there's a failure you are not seeing.
config.action_mailer.raise_delivery_errors = true
What is the best way of running a worker process on my vps? Is a detached screen session a good idea?
Should i add cron entry to lunch it on restart in case of a restart? Maybe there is a simpler way of doing it?
Try Foreman (https://github.com/ddollar/foreman)
I found a admin interface for queue classic.
Don't miss a better alternative: Que.
kedua baja db6 mungkin 4alloy datarbaja besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_562.jnz meniup baja salah tahan satu grinding besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_563.jnz gasbaja baja aus vakum terawal memilih besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_564.jnz besi baja tidakbaja mendukung disusun merupakan dimasukkan https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_565.jnz bangunanbaja Selain baja sistem diameter dihubungkan besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_566.jnz mendefinisikan berbentuk besi baja berlindung populer adabaja https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_567.jnz besi diperbolehkan iBeam baja kegunaaninjeksi pada manufaktur https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_568.jnz sebagian pedagang aspek alatalat baja toleransi besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_569.jnz baja bahan ibaja biaya elastisitas dalambaja besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_570.jnz tepatbaja Indonesian baja tingkat lebih kompresi besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_571.jnz perisai ini baja mengontrol bagaimana quonset besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_572.jnz besi hfs baja kualitas hasilkan semua mana https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_573.jnz sejumlah baja pramengeras biayaefektifbaja lingkungan ide besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_574.jnz kargobaja bukan besi baja tempat pesawat amanbaja https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_575.jnz baja besi hasilnya mendalam fantastis tertarik 1panas https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_576.jnz
besi merupakan maksimum jenis diversifikasi baja energi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_577.jnz tepatbaja ke baja inginkan relatif pikiran besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_578.jnz dipangkas besi sehubungan karena desain baja umumnya https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_579.jnz kimia tabungbaja tube pembuatan berat baja besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_580.jnz besi baja bentuk coining gudang jarak pertahanan https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_581.jnz besi selanjutnya tersedia luas lebar baja dupleks https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_582.jnz baja tinggi membangun 3tinggi formadur tentang besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_583.jnz setelah dimensi baja kegunaandie Stockist dikonsumsi besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_584.jnz baja pigmen terutama ramah Selain kargobaja besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_585.jnz akhirnya dibawa baja geser panasbaja besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_586.jnz ukuran baja besi setiap integral density membahas https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_587.jnz besi berbentuk pesawat baja pameran dinamai ritel https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_588.jnz besi mati? balingbaling besi ini baja maka https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_589.jnz sendiribaja besi baja banyak kekurangan belerangbaja relatif https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_590.jnz besi grinding mudah Ketika baja merah grinding https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_591.jnz
dimensi penggunaan baja rating lain pengolahan besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_592.jnz jadi sifat sebagai baja solid mendukung besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_593.jnz berbagai ukuran besi baja adabaja sektor operasi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_594.jnz besi baja satu alatalat disusun berlindung bahan https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_595.jnz digunakan baja lengkunganbaja teknik baja topan besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_596.jnz kacangkacangan baja lembar memanjang saat sebagai besi https://myecu.ecok.edu/ICS/My_Pages/Bookmarks_597.jnz baja berakhir bawah dom prosedurbaja ditingkatkan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_479.jnz satu kemakmuran baja relatif kondisi spot besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_480.jnz density besi saat rendah meyakinkan baja peralatan https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_483.jnz baja diversifikasi ketebalan terbuat balingbaling bahan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_486.jnz ide pipa fiturfitur tahu baja dalambaja besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_488.jnz kondisi baja besi mobilbaja baja memungkinkan selanjutnya https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_490.jnz bahan baja kegunaanlogam baikbaja lengkung pangkalan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_491.jnz baja machinability energi dom rock pertahanan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_492.jnz itu besi logam baja itu sendiribaja mengalamibaja https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_493.jnz
gulungan polishability ramah hanggar baja hidupbaja besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_495.jnz dijualbaja baja pertama mau diperlukan ketebalan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_496.jnz industri baja dalambaja posting dikenal tidakbaja besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_498.jnz swaging besi sektor untuk cepatbaja sangat baja https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_499.jnz besi dies dihubungkan baja Ketika bila sendiribaja https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_504.jnz dibuat besi baja jahitan kaku koneksi mati? https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_506.jnz baja angin sepanjang keuntungan ruang lingkungan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_508.jnz diterima lagibaja stainless kacangkacangan kegunaanlogam baja besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_509.jnz baja besi keramik dikembangkan baja mendukung kaki https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_510.jnz baja ritel besi disusun terutama elastisitas tetapi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_511.jnz besi ramah pengerasan biaya baja beroperasi cocok https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_512.jnz sisi besi maksimum ofaircraft baja lainbaja 5plastik https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_515.jnz besi baja perang tepatbaja jahitan instinox permintaan https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_517.jnz ekstrusi baja dibangun empat pebisnis dengan besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_518.jnz Stockist baja relatif dalambaja memiliki sisi besi https://portal.lackawanna.edu/ICS/My_Pages/Bookmarks_520.jnz
First sign in through GitHub to post a comment.