#214 A/B Testing with A/Bingo

Thanks for this nice introduction to rails A/B testing, there was also a talk some time ago about this topic that can be reached here:

http://rubyconf2009.confreaks.com/19-nov-2009-17-15-how-tdd-missed-the-point-introducing-edd-nathaniel-talbott.html

Ryan,

Thank you so much for doing this episode!

I really appreciate all the hard work you put in each week.

this is What I needed When I needed it, Thanks Ryan!!!

You must've read my mind! I've been trying to implement A/Bingo lately and kept getting the 'frozen array' error, so thanks for the patch.

Another excellent screencast!

I'm wondering if the code in the third code box should read:

    <%= title signup_title %>

instead of:

    <% title "Free Sign up" %>

tester

Not a huge deal but the code in the git repo is different than what was shown in the Railscast.

I think the git repo had some type of todo list, while the code in the cast was a signup form.

Awesome screencast as usual. Haven't needed to do any A/B testing so far, but this seems like the perfect excuse to start.

A/Bingo is not a bad solutions, seems to be similar to vanity.

Anyways I'll stick with vanity thanks for the awesome screencast.

Thank you Ryan for another great episode.

One thing is noticed was that you did not spent a lot of time on the difference between split testing and multivariate testing.

What you presented was a multivariate test, but if the viewer is new to testing and the terminology, a few words about the types of tests might help clear up the field.

Otherwise great simple introduction.

Ive not tried this yet but if you are caching you products page for example. Would the ab testing still work or would you need a workaround

Thanks for the post Ryan, great as usual.

I'v stared to think that it's still Sunday when checked your site and noticed that that's no now railscast :(

So ryan is there a way to do multiple scenarios like for example ....try green image or try blue image or try red image and see which one provides the best conversions....

Everything seems to work except counting conversions. With caching enabled, even though I've excplicitly NOT cahced the action, I only see this the first time, not subsequent calls:

Cache hit: Abingo::Experiment::body_background_white::alternatives ({})
  Abingo::Alternative Update (78.8ms) UPDATE `alternatives` SET conversions = conversions + 1 WHERE (`alternatives`.`lookup` = '5343424bdcd7cc178fb422651f97fb95')

...and the test results don't get updated. Do you know how to prevent that from happening?

Great screencast, as usual! I'm considering A/Bingo for our own A/B testing purposes, and was just wondering if it's still people's preferred framework (after - possibly - Vanity), or have any other players entered the Rails A/B testing market in the 12 months since this railscast was first aired? Thanks!

Hey, Great screencast.

I am however getting an error when trying to use a block:

When I use this:

<% ab_test("signup_test", ["START YOUR MEMBERSHIP","SIGN UP", "CLICK HERE"]) do |button_text| %>
  <%= button_to button_text, :new_user_registration, {:method=>:get, 'data-theme'=>'mp', :onclick=>"_gaq.push(['_trackEvent', 'Users', 'Sign Up']);"} %>  
<% end %>

I get this:

undefined method `block_called_from_erb?' for #<#<Class:0x105245f30>:0x105243ca8>

Any ideas what that means? Have you seen that before?

Thanks for all the great work and all the help.

thanks for good tips in advance.
I have a question.
what is DSL?