#111 Advanced Search Form

Dude, I would never thought, nor prolly would have ever thought about making a search a resource. Was this an original idea or something that someone else hinted at for you?

Once again Ryan, Bravo.

Hi Ryan,

The scaffolded controller only had new and show, right? Wouldnt the search form invoke the non-existent create action on the server? Have you modified the controller code for the above to make it work?

Cheers,
Aditya

Uhh... like it... ...that simple.

I've done something similar to store it for customized RSS Feeds.
But I have it to simplify with a search model.

Thank you Ryan
Rafael

Great screencast.

But please, please never ever use floats in real applications when it comes to money. I don't want to have rounding errors on my bank account in the future.

Cheers,
Maciej

@taelor, it's not an original idea. I heard about it elsewhere but don't remember where.

@Aditya, the nifty_scaffold generator will generate the create action automatically if there's a new action.

@Maciej, good point! I normally use the decimal type for prices, but thought a float would be okay in this case since this isn't really a price. It's just a number that's used to search prices. I wouldn't be performing calculations on this value.

@Ryan, well that's nifty! Yes it absolutely is.

How would you implement pagination?
I've done something similar, the only difference is the search class generates a conditions block passed to another model. This makes it easy for me to tack on pagination.

Hello!

Why don't you create a simple virtual model (non-activerecord class) with instance attribute parameters?

@Dandre, good question. What I would probably do is pass the page number into the Search model and call paginate inside the search model. This opens up the ability of storing pagination parameters in the searches table (such as how many results are displayed on a page).

Since we have a model dedicated to searching, to me it seems best for it to handle as much of the searching as it can.

@Avalon, that's a possibility, but the problem is then we have no way to fetch the results again. There's no way to bookmark the URL, or add pagination if you need to. Storing the search in the database allows you to do this. However it really depends on the requirements for your app.

Amazing!, Ryan but I emailed you yesterday! lol

Thanks a lot, you don't know how much for this one!!

Episode 112 could show us how to make this search polymorphic, cause I would like a search for products, one for comments, other for posts...

In the show page I'm getting:

Couldn't find template file for restaurants/_restaurant

Where line of error is:

<%= render :partial => @search.restaurants %>

What could be wrong, anything else needs to be added?

@Marcos, I'm not sure how a polymorphic association would help in that case. I'd probably just make a separate search model for each one (ProductSearch, CommentSearch, etc.). Unless you need the same attributes for each one but they will likely need to be different. Then if you have some duplication you can move that out into a generic Search module.

@Jose, yep, this will look for a partial under "restaurants/_restaurants.html.erb", so you should make one there which has your restaurant display. Alternatively you can use a "for" loop to loop around the restaurants array instead of using a partial.

For those who may not know, if you don't want to save your searches in the database, but would like all benefits of using an AR model (like validations), you can use virtual attributes:

class Search < ActiveRecord::Base
  attr_accessor :keywords, :category_id, : minimum_price, :maximum_price
  validates_presence_of :keywords
  # blah...
end

Everything works the same, just doesn't save anything. And no need for a migration or 'searches' table in db.

Instead of the complicated conditions why not use criteria_query http://agilewebdevelopment.com/plugins/criteria_query

Ryan, this screencast looks like it's going to be a HUGE help in making some of my resources more restful.

Thanks for the great work

I've also thought about the off-database model.
In current solution:
- You can easly run out of the free id (it is not so uncommon as it may appear to be, hard to solve and detect) - the deleted ids are not reused
- Also easy DOS attac (run out of ids).

Run out of id can be solved but it requires in fact stop of service.

He Ryan,

Great article..i am going to used it for sure in my new rails project.

You talked about storing the search results of the user in the database. I looked arround the net, but with no result. Do you have some links where i can find information about this kind functionality?

Grtz..remco

I find it ironic how easily you can implement a search as complex as this, but there is no search implemented on this site!

D'oh!!

Thanks Ryan, Today I started working on a search engine for my application and this railscast is exactly what I need.

PS Does someone have a paginated version of the form to let me see? ;)

@Bizzy, yep! planning to add searching to Railscasts, just haven't gotten to it yet. :)

@Ryan
Why on keyword_conditions do you use the variable keywords twice? Is once not enough?

Thanks

It's still me
I have noticed that in the code you post here as a resource you have made all the methods (unless find_products") private. However, in the railscast all are public. With private methods it looks like the search doesn't work. Am I missing something on my app or they must be public?

Thanks again

Thanks for this screencast!
It's going to be so usefull! I'm going to use it with sphinx search engine!

I think the _conditions methods need to be public because the the 'methods' method doesn't return private methods. Which means you can't run grep on them.

Another great plugin to make writing complex conditions easy:

RailsWhere (http://code.google.com/p/railswhere/)

I think that using this plugin makes the code a lot more readable that writing seperate functions for each condition and then using metaprogramming fu to join them all together.

Example rewrite of the Search class using RailsWhere.

class Search < ActiveRecord::Base

  def products
    @products ||= find_products
  end

  def find_products
    Product.find(:all, :conditions => to_conditions)
  end

  private

  def to_conditions
    w = Where.new
    w.and "products.name LIKE ?", "%#{keywords}%" unless keywords.blank?
    w.and "products.price >= ?", minimum_price unless minimum_price.blank?
    w.and "products.price <= ?", maximum_price unless maximum_price.blank?
    w.and "products.category_id = ?", category_id unless category_id.blank?
    w.to_s
  end

end

Ryan, I love your screecasts. You do a wonderful job, but I have one gripe. The intro makes me want to commit suicide. The clickety click of a keyboard, the big BLAHMMMMM noise, etc. Consider changing it? Please?

Love,
d

@Andrew : thanks RailsWhere looks interesting

Spot on Ryan!

I've found the acts_as_ferret plugin very good for plaintext searching.

I've found one pitfall with your approach though: if you want to exchange searches between users ? If somebody does a search in Google they can easily pass that url to me and I can enjoy the same results. That's why I think that having complete query strings in the url is not that bad idea.

If you change the condition_parts method to read:

private_methods(false).grep(...

It will work just fine.

Apologies for the incorrect code. It should be fixed now.

@Devin, planning to make a new intro soon. Sorry about that. ;)

@Tomek, since the search id is in the URL, it's still possible to share the searches with anyone.

This is a valuable resource!
Love it. thanks

Rails newbie here. Thanks so much! Your railscasts are amazing. What should my partial page look like? I can't seem to get the partial to display my search results correctly.

Nevermind...I got the search results working! Just forgot my for loop!

Thanks for the cast!

That's an amazing solution! :)

I've adopted a similar approach but using a MultiCondition object (http://github.com/weppos/activerecord-multiconditions/).

The main advantage is that I can queue illimitate conditions and I can decide which conditions should be queued for which methods.
Using a solution based on method names might cause some trouble when I would use, for instance, all conditions in action A and all conditions - 1 in action B.

Great job Ryan,
I never thought I will saw such a beautiful code for multiconditional search.

If your Product model "has_many :colors" how would you also add the ability to to return only products that come in a certain color - assuming you have a text field in your search form named "color" and the Color model has a field named "name"?

I figured it out. You need to add in an ":include" to the "find" like so:

def find_products
  Product.find(:all, :conditions => conditions, :include => :colors)
end

and then you use:

def color_conditions
    ["colors.name = LIKE", "#{color}"] unless color.blank?
end

Marcos, Ryan and Greg, please, check out a chunk of code i wrote tonight:

http://wild-tatar.livejournal.com/41283.html#cutid1

i use it to search anything:

Search.new(:source => 'members', :assoc => {:profile => {:height => 100..190}}).result
or
Search.new(:source => 'members', :assoc => {:favourites => true}).result
or
Search.new(:source => 'members', :root => {:first_name => 'LIKE Dennis%'}).result
or combination of anything of the above

Follow-up: i realize it still needs a lot of improvement.

In possibly-not-so-distant future I plan like to move it inside AR::Base.find to accept search clauses from :conditions, to make associations' table_name discovery more generic, etc. I would be very grateful for any kind of constructive feedback.

Ryan: Surely, the id is a perfect way to reference the search but if the user changes the search the others can see the change too which might be undesireable.

Love the screencast. Just to say I did something similar, the difference is that mine returned a sql string and a 'human string'.

  def self.create_search(search, admin = false)
    @search = search
    @admin = admin
    @human_string = 'all properties'
    dept_condition self.methods(false).grep(/_conditions$/).map { |m| logger.debug "Searching #{m} "
      self.send(m) }
    return @condition.join(' AND '), @human_string
  end

with one of the private methods

  def self.price_range_conditions
    if @search[:min].to_i > 0 and @search[:max].to_i > @search[:min].to_i
      @criteria << replace_bind_variables("#{table_name}.`price` BETWEEN ? AND ?", [@search[:min].to_i, @search[:max].to_i])
      @human_string += ", costing between " + number_to_currency(@search[:min]) + ' and ' + number_to_currency(@search[:max])
    end
  end

with both the sql and human string I then do the find and let the user save the search into the database for later use if neaded.

PS keep up the good work.

Lately I have been using a search controller, which I use the new action from to start of a complex search. I have been working on a search engine at work, which needed the most complex search code I have ever had to write. Finally I came up with this: http://www.styleless.nl/search.rb

It's still really slow, takes about a minute before the search is complete (any hints anyone? ;)). I'm still refining it at the moment, but I'm going to take this to a search model after watching this :)

Thanks (again) for a great screencast.

I have found using squirrel with some modifications to resource_search allows for much of the above, with little addititional work. With squirrel, there is little reason to ever build your own SQL WHERE clauses manually again.

I can't tell you how grateful I am for this: I spent weeks beating my head against various search options, and none of them seemed to make any sense for my needs until I finally found this, and suddenly I have everything I need!

I do have a question, though. How would I write the conditions for a boolean field? I have several boolean (true/false) fields in my tables, and I want users to be able to check a box in a search field when a field should be true. How should I do this?

Many thanks!!

Thanks for this screencast.

I use this searchmethod and the result is presented in combination with wicegrid (http://redmine.wice.eu/api/wice_grid/index.html) and it works great.
The method for making a SQL condition is awesome.

How do you make this kind of thing drop the :conditions => conditions thing in the case that the user doesn't fill in any of the form. Actually, what I'm really doing is adding GeoKit in here, so I want to do a search which uses the :origin => and :within => stuff. It works fine as long as somebody searches using the conditions, but if you search JUST with origin and within, it throws a syntax error because of an AND with nothing in front of it.

I hope someone can help me.
I don't want to use the nifty scaffolding approach. What does it generate in the new and show methods?
Thanks.

Hello!
Thanks for all this great tuts.
Keep up!
I have question about your data base in this tutorial. How she luck like in the moment of creating nifty_scaffold. Please tell me!
Best regards and thanks again!

What if my search model uses a HABTM relation.

Where I select multiply options from a checkbox list and want to use them in my search.

Do my model have a string field containing the id's like (11,54,32) or do I create a separate link table?

suppose that my checkbox list is a existing model.

Any ideas?

Thanks to Greg's suggestion about including different models. helped me out.

Any hints on what that partial page looks like? I'm getting some newbie associations mixed up and I can't show my results. Thanks for all the screencasts.

Correct me if i'm wrong here but this code appears to have a little bug. conditions_parts is called twice and hence the methods function is invoked twice. I noted however, due to my application breaking, that methods does not always return an array of methods in the same order btween calls (ie its return value is not have consitent ordering). This breaks the coonection between filter_options and filter_clauses, meaning incorrect values get passed to the final query "sometimes"..

further to my above cooment. I was getting incorrect search results until I fixed it. herse the code change i made which means methods is only called once..

def filter_parts
  @parts ||= methods.grep.....
  return @parts
end

Ryan, great screencast! I really loved idea storing the search filters in database. What I am thinking is, if my model has really many fields (strings, integers, booleans) and I really want to provide a full advanced search to the user on every field, will I be then copying every field to the search model? Is there are way to reuse the fields but still keep the separate model?

Ryan, i followed ur screen cast tens of times but dont get it working. The problem is that resulting SQL, mixes up the corresponding values. Do you know what might be wrong? (Rails 2.2.2)

Example SQL i get to logs: SELECT * FROM `properties` WHERE (properties.price_unit = 'buy' AND properties.offer = 'USD' AND properties.area_unit = 'once' AND properties.kind = 'meter' AND properties.period = 'apartment')

So, those values are mixed up and in wrong order. Please, could u tell how I can find the problem.

Thanks!

Hello,

Ryan, many thanks for your excellent video tutorials.

Im a total ruby/rails beginner and I do not get this search to work. Would it be possible to paste all the code which concerns this search?

I would like to have a look at the:
- search controller
- _products.erb partial

It would also make this tutorial a bit more beginner-friendly if those where included.

regards

Thanks for the excellent cast.
The various search methods in the search model can be now replaced by named_scope definitions. Is it right?

Nadav

Hi Ryan. I am very new to Ruby on Rails. Advance search is what I was looking to implement. I did try it out but facing some problems. when i render the partial I get an error sayin template missing. I create the partial but what do i put in the show method??

can you pls help

I wonder what codes is in Project Controller? Can anyone provice me with some cource codes about advanced search!

Hey Ryan, shouldn't the code for maximum_price_conditions say " <= " instead of the " >= " in the screencast?

Oh I see you fixed it in the code here in the post... ;-)

Fantastic idea. It's one of those things were "there's gotta be a better way" ... and this is it.

I recommend those who aren't so hot on their "partials" to look at this:

http://guides.rubyonrails.org/layouts_and_rendering.html#rendering-collections

... at least, if you're like me and that "so let's render a partial" made you go "whoa, WTF?",
then the above link is for you ;)

GaJ

第三䒈

Am i the only dumb noob who can't get this working...

i did everything as shown in the vid but when i send a new search i fail with this error message:

Missing template products/_product.erb in view path app/views

Looks good but what happen when your relation its has_and_belongs_to_many in one of the attributes ?

So what is the user's URL, from outside of the application? Is it RESTful? Even though maybe form generated? (I don't read Ruby, sorry, I'm a PHP Head.)

Could you show us how to change the search model to a non-database backed model? For example, let's say we implement this as you show it in the screencast, then we decide to make search.rb a class that does not inherit ActiveRecord::Base.

I got the same problem as Markus

Missing template products/_product.erb in view path app/views

so I created the partial _product.erb as the following

<% form_for @product do |f| %>
  <%= f.error_messages %>
  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>
  <p><%= f.submit "Submit" %></p>
<% end %>

.. and now I get the error below

Called id for nil, which would mistakenly be 4 -- if you really wanted the id of nil, use object_id

When I display in the debugger for the view search/show the content of @search.products I got the correct entries in the array. It seems like the partial cannot link it to the variable @product.

I am using Rails 2.3.4 and I tried also to pass the parameters :collection and/or :as etc. when calling the partial but I got always the same error.

I also tried to call the partial products/index which expects a collection @products and still the same error.

Please help me I am new to Rails and I am really stuck with this variable passing between the view of one controller (searches/show) and the partial of another controller (products/_product.erb.)

Cheers,
Houcine

@Houcine:

I simply used The following code as my partial, in place of in your case @product.name (as you put it :name) I used inventory.item and inventory.part_number

I saved this file as _inventory.html.erb in my app/views/inventories folder.

I didn't adjust either (searches or inventories controller)

For those with experience using partials is this a good way of working with these files? also how did you deal with displaying "no results found?"
-------
<p>
<strong>Record:</strong> <%= link_to "#{inventory.item}", inventory %>
  
  
</p>
<p>
  <strong>Part Number:</strong>
  <%=h inventory.part_number %>
</p>

<p>
</p>

<% form_for @product do |f| %>
  <%= f.error_messages %>
  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>
  <p><%= f.submit "Submit" %></p>
<% end %>

.. and now I get the error below

Called id for nil, which would mistakenly be 4 -- if you really wanted the id of nil, use object_id
----------------

For people who are getting the aforementioned error message you are missing a = in the code.

<% form_for @product do |f| %>

should be

<%= form_for @product do |f| %>

and you should no longer get an error message!

It took me a good half an hour to figure this out

i've got this kind of problem:

ArgumentError in Searches#show
Showing app/views/searches/show.html.erb where line #1 raised:
wrong number of arguments (0 for 1)
Extracted source (around line #1):
1: <%= render :partial => @search.advertisements %>

any idea how to fix it??
yes i've got a partial _advertisements in app/views/advertisements
and it looks like that
http://pastie.org/757620
thx for any help
regards

The for loop in the searches show action is going to be:

<% for product in @search.products do %>

I have a question regarding how to save the search model that contains a field with multiple checkboxes: http://groups.google.com/group/rubyonrails-talk/browse_thread/thread/50915dfb233d62bc#

Please click on the link and help. Thanks.

Ryan,

I encountered a problem with the conditions_options method when trying to compare a field to an array of values. The flatten would recursively flatten my argument (an array) and i'd end up with too many options for my conditions. I made the following change:

## MY FIELD
def state_conditions
   ["state in (?)", state] unless state.to_s.blank?
end

## CHANGED CODE HERE
def conditions_options
     opts = []
     conditions_parts.each do |condition|
        condition[1..-1].each{|cp| opts << cp}
     end
     opts
end

Other than that, I really liked this approach. I added a few extra methods to use the same model to generate and search through SQL or my SOLR index.

Good job.

The flatten options does break the use of "IN (?)" type searches...

ActiveRecord::PreparedStatementInvalid: wrong number of bind variables (2 for 1) in: <some field> IN (?)

Adding the optional level of 1 to flatten fixes this.

Got a problem with the show view,
Only get the attributs stored in the model search
that's the partial:
[CODE]
<p>
  <strong>Keywords:</strong>
  <%=h @search.keywords %>

</p>
<p>
  <strong>Rezepthauptgruppen:</strong>
  <%=h @search.rezepthauptgruppe_id %>
</p>
<p>
  <strong>Rezeptuntergruppen:</strong>
  <%=h @search.rezeptuntergruppe_id %>
</p>
<p>
  <strong>Preis Pro Portion:</strong>
  <%=h @search.preis_pro_portion %>
</p>
<p>
  <strong>Kj:</strong>
  <%=h @search.kj %>
</p>
[/CODE]

Thats the model
[CODE]
def rezept
@rezepte ||= find_rezepte
  end
  
private

 def find_rezepte
Rezept.find(:all, :conditions => conditions)
 end

def keyword_conditions
  ["rezepte.rezept LIKE ?", "%#{keywords}%"] unless keywords.blank?
end

def preis_pro_person_conditions
  ["rezepte.preis_pro_portion <= ?", preis_pro_portion] unless preis_pro_portion.blank?
end

def kj_conditions
  ["rezepte.kj <= ?", kj] unless kj.blank?
end

def hauptgruppen_conditions
  ["rezepte.rezepthauptgruppe_id = ?", rezepthauptgruppe_id] unless rezepthauptgruppe_id.blank?
end

def untergruppen_conditions
  ["rezepte.rezeptuntergruppe_id = ?", rezeptuntergruppe_id] unless rezeptuntergruppe_id.blank?
end

def conditions
  [conditions_clauses.join(' AND '), *conditions_options]
end

def conditions_clauses
  conditions_parts.map { |condition| condition.first }
end

def conditions_options
  conditions_parts.map { |condition| condition[1..-1] }.flatten
end

def conditions_parts
  private_methods(false).grep(/_conditions$/).map { |m| send(m) }.compact
end
[/CODE]

thats the log:

Processing SearchesController#show (for 127.0.0.1 at 2010-09-15 18:38:20) [GET]
  Parameters: {"id"=>"30"}
  ?[4;36;1mSearch Columns (0.0ms)?[0m ?[0;1mSHOW FIELDS FROM `searches`?[0m
  ?[4;35;1mSearch Load (0.0ms)?[0m ?[0mSELECT * FROM `searches` WHERE (`searches`.`id` = 30) ?[0m
Rendering template within layouts/application
Rendering searches/show
  ?[4;36;1mRezept Load (0.0ms)?[0m ?[0;1mSELECT * FROM `rezepte` WHERE (rezepte.preis_pro_portion <= 5 AND rezepte.rezepthauptgruppe_id = 4 AND rezepte.kj <=800) ?[0m
Rendered rezepte/_rezept (0.0ms)
Rendered rezepte/_rezept (0.0ms)

what did I wrong?

Thanks
TimE

That works flawlessly!
Thanks

Note: It appears that Nifty Generators are no longer viable with Rails 3.1+. I love the idea behind them of being so minimal and non-intrusive. I'm hoping Ryan has some time to give them some love, but in the meantime, be warned.

I posted a bug report on github for the 3.2 issue I noted, but it looks like the problems go back for quite some time.

This episode has been updated to Rails 5 as a blog post Advanced Search Form in Rails 5