#313 Receiving Email with Mailman pro
Jan 02, 2012 | 11 minutes |
Tools
The Mailman gem makes it easy to receive email in a Rails application. Here I show how to write a script to run Mailman in its own process and pull down mail from a POP3 account.
- Download:
- source codeProject Files in Zip (91 KB)
- mp4Full Size H.264 Video (27.6 MB)
- m4vSmaller H.264 Video (13.5 MB)
- webmFull Size VP8 Video (15.2 MB)
- ogvFull Size Theora Video (32.3 MB)
This is quite interesting. Can you do a similar screencast for handling incoming faxes and then routing them appropriately?
Thanks.
Bharat
Incoming faxes? Wouldn't you need to have like a phone number or something? I don't think many people have the need to handle faxes.
I for one would be interested in learning how to handle inbound faxes. :-)
Wow, who knew! I can't picture how this could work, but seems interesting!
You can skip
require "rubygems"
since Ruby 1.9I've been working on email marketing for a while now and I have to say that Gmail is the 1st email client that I use to test my emails, but then I have to test in the painful Outlooks... because the majority of clients uses Outlook (Link).
Making emails "work" in Outlook 2007 & 2010 (Windows versions) is painful, because M$ switched the render engine from IE to Word (join your voice at Link) and the Word engine doesn't follow some web standards, so be ready for some headaches.
Important lessons:
* be careful when sending a lot of emails, your email address can get blacklisted very fast
* use several email clients to test your emails
* forget CSS positions
* use a spam tester, to test your email's spam value
* add an unsubscribe link to your email, because it's mandatory (SPAM-CAN laws)
* ...
Some useful resources:
Link
Link
Hey Nuno, I created a tool to help people to test HTML e-mails.
The tool, called Puts Mail uses Premailer to check HTML warnings and to make the CSS inline. ;)
But I think that your concern is more about the screencast Sending HTML Email, no?
Cheers,
Pablo Cantero
You want to be careful rescuing
Exception
as it catches things like Ctrl-C and other signals. This is because For example:Try pressing Ctrl-C or killing the PID. It really sucks having to
kill -9
because the killer never knows if they're causing some sort of state corruption.I recommend rescuing from
StandardError
and possiblyTimeout::Error
if you are hitting some external service. Another option is to do something like this:any tips for using this on Heroku?
From https://github.com/titanous/mailman/issues/19
Write a rake task that calls Mailman::Application.run and then use a heroku 'worker' to do this. Your rake task needs to be 'jobs:work'
http://devcenter.heroku.com/articles/queueing
Hi, I am fairly new to both rails and heroku. Based on ur suggestion, I researched quite a bit on how to create rake task and call Mailman::Application.run, but was not able to find any good examples. I'd very much appreciate if you can share some sample code around this. Thanks!
Unfortunately if the app uses one dyno 24x7, the app cannot use the free quota for the worker as well. :/
Have you tried using the Mailgun or CloudMailIn add on?
I am using cloudmailin, and just started playing with mailgun.
Both are very easy to setup and use.
Thing I like about cloudmailin: its advance feature allows directly storing email attachment on s3.
Thing I like about mailgun: its routes and actions.
Would it be difficult to configure the server to start with foreman rather then a script?
HI Ryan, I've noticed that you have used a few environment variables in the recent episodes. Do you have any opinions on how best ways of setting them? IE - one config file vs in each environment file? Maybe this could be a short episode on how environment variables work and boot sequences?
Thanks!
I would also love to see a video on this.
Any tips on daemonizing this thing?
Use Procfiles with foreman gem? In production export as upstart job.
Any ideas on how to use multiple pop3 mailboxes for polling?
Any advantages of Mailman over Activemailer?
By checking the docs Action Mailer - Receiving Emails you'll see that no receivers are provided, i.e. nor POP3 or Maildir built-in support.
You need to somehow trigger the ActionMailer recieve method every time an email is received, e.g.
Mailman solves that for you but ActionMailer doesn't, you would need to write your own Maildir Guard plugin and/or a POP3 polling daemon to call UserMailer.receive
How would you handle having different configurations? Let's say a User logs into the system. Under their Account Settings, they can put in their pop/smtp settings and then use ActionMailer and MailMan with those settings vs having one configured for global use. One of the reasons why I ask is because the SendMail app within linux keeps getting flagged as spam and/or never reaches it's destination. However, this doesn't happen when using a SMTP server. I know that this is a bit off topic in general.
I'd love to see a take on this that removes Mailman from the equation and handles emails that are "delivered" to the app via a POST request (a la Sendgrid's Post API).
I just found a promising cloud service cloudmailin
+1 for this
can anybody please help me. how to configure mailman on production.
should i need to keep my production terminal allways open after typing script/mailman_server.rb
I've tried setting up the maildir access (using postfix as my email server), but am having permission problems. The mailman script can't access the maildir messages. If I chmod the maildir, the new messages that get written aren't chmodded, so that doesn't work.
Any ideas on a workaround?
I created a mail account for the user my webapp is running with. I forward (selected) e-mails from another account to this one, so mailman can pick them up and process.
I tried to use the Daemon gem to run the mailman_server in the background. However, I cannot get it to recognize the rails environment, so I cannot access my models from the mailman_server.
The only way I can get it to run in the background is using:
nohup ./script/mailman_server &
But because there are no start and stop commands, it is difficult to implement in a deployment strategy.
Next try: God to manage the mailman_server
Did you set Mailman.ignore_stdin to true? I read somewhere that this must be set to run it as a daemon.
Yes, I did...
check out my comment below. I've provided an example which daemonizes the mailman server in test / dev and production environments.
Hey all. After watching this episode I integrated Mailman into one of our apps. I had to deal with many of the issues described above like daemonizing, testing and so on. I've put together a guide at
http://dansowter.com/mailman-guide/
Hope it helps someone.
Dan
It sure does Dan. Awesome. I got it working now.
I find it difficult to setup the server,so I want to see your amazing blog~
But I can't open the link above.
Can you send the link or your article to me?
my email adress: dqaria gmail.com
Thank you!
Hello Dan,
I had bookmarked that link some time ago but your website appears to be down.
Is there a replacement site where your content is hosted? It is a great article and I'm sure those of us who read it found it useful.
Thank you much.
I'm following along with this and I am having an odd issue that I cant figure out when saving the ticket to the database. Instead of it writing the To or From field as just the email address, its writing it in the form of "--- !seq:Mail::AddressContainer - email@address"
Anyone else ran into this?
Turns out it had something to do with the mail client I was using to send the email from. (Microsoft Outlook for Mac)
Has anyone else had issues and found a resolution when using Outlook or any kind of multipart/HTML email and have it format nicely. How did you handle email signatures with images?
Hello! I don't have Railscasts Pro, so I havent't seen the episode, but thought you guys might find this useful:
https://github.com/johncant/mailman-rails
It's a fairly basic Rails integration Gem for Mailman. It makes it easier to define your email router across your application, for instance from mailers or models. It also includes some rake tasks for daemonization which solves some of the problems mentioned in these comments.
I'm having problems to show the tickets after pop3 ativation.
Encoding::CompatibilityError in Tickets#index
Showing /home/caio/RubymineProjects/tickets/app/views/tickets/index.html.erb where line #17 raised:
incompatible character encodings: UTF-8 and ASCII-8BIT
Extracted source (around line #17):
14:
15: <%= ticket.Subject %>
16: <%= ticket.From %>
17: <%= ticket.Body %>
18: <%= link_to 'Show', ticket %>
19: <%= link_to 'Edit', edit_ticket_path(ticket) %>
20: <%= link_to 'Destroy', ticket, method: :delete, data: { confirm: 'Are you sure?' } %>
Rails.root: /home/caio/RubymineProjects/tickets
Application Trace | Framework Trace | Full Trace
app/views/tickets/index.html.erb:17:in
block in _app_views_tickets_index_html_erb___2407065071369385486_24584740'
each'app/views/tickets/index.html.erb:13:in
app/views/tickets/index.html.erb:13:in
_app_views_tickets_index_html_erb___2407065071369385486_24584740'
index'app/controllers/tickets_controller.rb:7:in
I'm googling a lot but I've found nothing...
Hi @caiolufer, I am also facing a problem similar to this. Can you let me know how you solved this?
I tried to decouple the receiving of a ticket from the further processing using observers.
For that I added an observer to tickets:
This works fine when adding a new ticket through the browser. However when I send an email to my application I'll get the following exception:
E, [2013-03-19T20:19:50.352625 #8296] ERROR -- : uninitialized constant Ticket
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/inflector/methods.rb:230:in
block in constantize'
each'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/inflector/methods.rb:229:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/inflector/methods.rb:229:in
constantize'
constantize'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/core_ext/string/inflections.rb:54:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:210:in
observed_class'
observed_classes'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:203:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:223:in
observed_classes'
observed_classes'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-3.2.12/lib/active_record/observer.rb:96:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:219:in
initialize'
new'/Users/tummam/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/singleton.rb:141:in
/Users/tummam/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/singleton.rb:141:in
block in instance'
synchronize'/Users/tummam/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/singleton.rb:139:in
/Users/tummam/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/singleton.rb:139:in
instance'
instantiate_observer'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:86:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:59:in
block in instantiate_observers'
each'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:59:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activemodel-3.2.12/lib/active_model/observing.rb:59:in
instantiate_observers'
block (2 levels) in class:Railtie'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-3.2.12/lib/active_record/railtie.rb:117:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/lazy_load_hooks.rb:36:in
instance_eval'
execute_hook'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/lazy_load_hooks.rb:36:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/lazy_load_hooks.rb:43:in
block in run_load_hooks'
each'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/lazy_load_hooks.rb:42:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activesupport-3.2.12/lib/active_support/lazy_load_hooks.rb:42:in
run_load_hooks'
'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/activerecord-3.2.12/lib/active_record/base.rb:720:in
/Users/tummam/Documents/Rails/ticketing/app/models/ticket.rb:1:in
<top (required)>'
block (2 levels) in 'script/mailman_server:19:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/router.rb:66:in
instance_exec'
route'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/router.rb:66:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/message_processor.rb:19:in
process'
block in get_messages'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/receiver/pop3.rb:41:in
/Users/tummam/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/net/pop.rb:665:in
each'
each_mail'/Users/tummam/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/net/pop.rb:665:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/receiver/pop3.rb:40:in
get_messages'
block in polling_loop'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/application.rb:118:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/application.rb:115:in
loop'
polling_loop'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/application.rb:115:in
/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/application.rb:75:in
run'
run'/Users/tummam/.rvm/gems/ruby-2.0.0-p0/gems/mailman-0.6.0/lib/mailman/application.rb:7:in
script/mailman_server:16:in `'
To me it looks like the observer is the wrong idea to handle further processing. However as I envision it to be rather complex I wanted to decouple it from the receiving part of the application.
Can someone help me out with the observer problem or suggest a better method to do it?
Anyone know how to add stopping/starting the script/mailman_server to Rubber?
This was a lot of help for me: http://dansowter.com/mailman-guide/
Unfortunately, Dan's site seems to be down. I sure do wish I'd saved his post as a PDF or something, it's brilliantly written and simple.
I'm confused about a few things.
Will this work with Rails 4?
You specify
require: false
for the mailman gem because you don't want it loaded in Rails but as a separate process. Is this how mailman is meant to be used? No where on mailman's userguide does it mention this.There is no script directory in Rails 4? Sure I can just create it, but I imagine there is a better place to put stuff like the
mailman_server
file?I can't get past the step where you create a new record using the email contents.
I get the error
type_cast': can't cast Mail::AddressContainer to string
.Full stack trace here.
I take it you are using Postgres? Likely your message is returning arrays for "to" and "from" and Postgres is throwing an error because it expects a string. Try message.to.to_s or message.from.first
Thanks! I am using Postgres and you were right, converting the arrays from
message.to
andmessage.from
to strings or getting the first elements solves the issue.Hi, I am following this tutorial. I have followed all above steps.
Manual steps before configuring the pop3 server were running all fine. Like: chmod +x script/mailman_server and cat mailman_test.eml | script/mailman_server
But at the last step when I run script/mailman_server in the terminal, I got the error:
Users/nandini/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/net/pop.rb:552:in `connect': SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed (OpenSSL::SSL::SSLError)