#307 ElasticSearch Part 2 pro

Dec 12, 2011 | 16 minutes | Plugins, Search

This final part on ElasticSearch and Tire will show how to make more complex search queries, customize the indexing, and add facets.

Click to Play Video ▶

Download:
source codeProject Files in Zip (84.1 KB)
mp4Full Size H.264 Video (40.1 MB)
m4vSmaller H.264 Video (19.9 MB)
webmFull Size VP8 Video (23 MB)
ogvFull Size Theora Video (50.6 MB)

tfl almost 13 years ago

thanks again for a great railscast.

One thing I am still missing and that is how to full text search over multiple models. I know that there is the gem ultrasphinx but maybe you know other ways to do this

Karel Minarik almost 13 years ago

You can search across multiple indices (= model indices, eg. "articles" and "comments") with the Tire lower-level DSL. You can extract that into a separate class, search across the models and merge results manually by score, etc.

No built-in multimodel search a la ThinkingSphinx, at the moment...

You can see a discussion about this topic at https://github.com/karmi/tire/pull/131

Marcello Rocha almost 13 years ago

@tfl:

You can commit different documents to the same index. ES will store the type in a special key named _type.

@rbates:

Awesome screencast and with great timing too. My company is switching from Solr to ES just as I type. =)

Ali Anwar almost 13 years ago

@rbates:
I loved it, this is amazing in quality and timing. the last few pro screencasts have proven themselves to be very useful in my current work.

A question, could you give an example as how to use fuzzy search? and how to apply "phonetic token filter"?

Although I'm googling around, but I would appreciate any hint, since this what's stoping me now from jumping into elasticsearch/tire to replace sphinx/thining_sphinx.

Again, many thanks!

Brian almost 13 years ago

@rbates:

Any tips for testing with these gems?

Nicolai Seerup almost 12 years ago

phillyslick over 11 years ago

+10 - the Wiki entries on testing are definitely lacking

Bob Walsh almost 13 years ago

So, can ElasticSearch be hosted on Heroku? Or do you need to host it on say AWS and then live with latency issues? (thinking about a <1k set of fairly small records...)

Kyle Decot almost 13 years ago

I'm also interested in getting this to work on Heroku. Let me know if you manage to get it up and running!

Karel Minarik almost 13 years ago

There are some "ES as service" providers, but no Heroku addon, yet.

However, the latency issues are not particularly interesting here, since Heroku runs on AWS, so you should get a very small latency when you set up your ES nodes at AWS.

Leo Gallucci over 11 years ago

Update:
Bonsai Elasticsearch

lmbacelar almost 13 years ago

I am having problems in re-indexing a model, since I am using kaminari instead of will_paginate.
The pagination method for Kaminari is "page" instead of "paginate".
Can I set this up anywhere on the Tire gem or do I have to define a paginate method on each searchable models, that point to the page method?
Thanks

Karel Minarik almost 13 years ago

@Imbacelar: Search for "kaminari" in issues at Github, there are plenty of examples which will get you started.

lmbacelar almost 13 years ago

Got it. Need to provide a paginate method on each searchable Class.
Thanks for the tip (@Karel) and for the great screencasts (@Ryan).

schneider-th almost 13 years ago

I also got it working, but Rails can't recognize dates anymore:

"Object must be a Date, DateTime or Time object. "2011-12-22" given."

But as you can see, it is in the normal date format.

Edit: Found the error myself. Rails wanted an extra ".to_date"

Karel Minarik almost 13 years ago

Hey all,

thanks to Ryan for the awesome screencasts. I've published a couple of refactorings and suggestions at https://github.com/karmi/railscasts-episodes/compare/master.

If you like elasticsearch and Tire, the commits are well worth the close look:

Ches Martin over 12 years ago

Thanks for taking the time to make these improvements, and especially to give the detailed explanatory commit messages Karel, they're really helpful.

Travis about 12 years ago

Thanks so much.. i was having anxiety attack with massive codebase thinking about re-adding all those as new methods :S

Ryan Siddle about 12 years ago

Thank you for refactoring this code. The associated objects are especially useful rather than masking each attribute behind a method.

I was having the same thoughts as Travis!

schneider-th almost 13 years ago

Two quick comments:

1) If I try to reindex my objects, rails throws the following at me:
rake aborted! undefined method 'paginate' for #<Class:0x00000106e32310>
I'm using Kaminari, so maybe it's because of that.

2) After including query do and boolean do I don't get any of my objects shown. Neither with nor without search term. I also don't get any errors, they just don't show up.

The first one is not a problem, but the second one really bothers me, so any help would be appreciated.

Karel Minarik almost 13 years ago

Hi,

ad 1) yes, that's a Kaminari related issue. I guess it's trivial to define a paginate method, but nevertheless, there's a patch including the paginate methods for Kaminari based projects already.

ad 2) I don't understand the issue here, ie. what is "including" query do. Best to submit an issue at Github.

Jimish Jobanputra almost 13 years ago

Ryan,

Can you do a pro episode on using advanced search with elastic search?

http://railscasts.com/episodes/111-advanced-search-form-revised

Ravindra almost 13 years ago

Great Tutorial.

But I want to know how we can build relavant search engine?

Ex: If we are searching for keyword "beer" if there is not results for the keyword beer it should atleast return relavant results such as "Alcohol","Whisky".

Another Example would be Keyword "mobile phones" it should return results as "Nokia", "Samsung" and so on.

Tony Marschall almost 13 years ago

Is it possible to use cancan or other authorisation methods with elasticsearch or tire?

ragrawal over 12 years ago

Is there a way to fetch all documents. I don't want pagination

Nekolus over 12 years ago

I am looking for a auto suggest, as a user types in. How do i do that?

scott hamilton about 12 years ago

Does anybody know how to include the author's name in the facet results. I'm trying to figure out if I can avoid looking up the Author for each facet:

          ruby
        
Author.find(facet['term']).name

I was trying to see if I could get the facet to return more than one value like this:

          ruby
        
facet "authors" do
  terms [:author_id, :name]
end

Jordan Sitkin about 12 years ago

Check out the diff between ryan's and karmi's code (linked above in the comments) for some ideas on this. Look at the :author index in article.rb
https://github.com/karmi/railscasts-episodes/compare/master

Bragaadeesh about 12 years ago

Thank you a lot Ryan !

Anthony Collini almost 12 years ago

I'm having trouble figuring out how to search substrings, or partial words from the search query. Any thoughts to this? Or perhaps you could make an elastic search part 3? :)

Marc almost 12 years ago

I noticed that you don't sanitize user input. I've also noticed that eleasticsearch is not very fond of several characters, like / [ { } : / \ " or '. There are probably others too.

Karim Helal over 11 years ago

Thanks for yet another great set of videos. Quick question however. Let's say your article would have a 1-n association with a tree-like structure called 'article_subjects', and that you would also want to be able to search based on these article_subjects (and their parents). Any idea on how we could achieve this?

Greg Grillone over 11 years ago

I am having some performance issues with deeply nested models. I have a parent Topic model. A topic has many posts, posts has many comments, and a comment has an author.

mapping do
  indexes :author_name, index: 'not_analyzed'
end

def author_name
  posts.map do |post|
    post.comments.map do |comment|
      comment.author.name
    end
  end.join(',')
end

Any tips on optimizing the performance?

Greg Grillone over 11 years ago

One improvement I made was implementing an includes call and setting it to an instance variable so I can reuse the nested relationships without having to re-query the model every time:

def author_name
  posts_with_children.map do |post|
    post.comments.map do |comment|
      comment.author.name
    end
  end.join(',')
end

private
  def posts_with_children
    @children ||= posts(:include  => { :comments => { :include => :author } })
  end

This improved performance slightly, but not a significant amount. Right now I have an initial page load of 5 seconds, but that is only with 4k records. If anyone has any other advice on how to improve this further that would be great. Thanks.

JPN about 11 years ago

Tire has been retired. Read the explanation here:

https://github.com/karmi/retire

Looks like there is a replacement:

https://github.com/elasticsearch/elasticsearch-ruby

Would love a Railscast for this, especially since this Tire railscast is almost 2 years old.

Ashley Connor about 11 years ago

Tire might be retired but this Railscast has dated well. Everything works in Rails 3.2.14.

elasticsearch-ruby doesn't appear to be finished and isn't as well documented.

See Karel Minarik's commit here for some improvements on Ryan's code - https://github.com/karmi/tire/pull/131

MicFin about 10 years ago

If not elasticsearch-rails then how about searchkey? https://github.com/ankane/searchkick.

umesh over 8 years ago

Hi,
It is very usefull if i know, How to set highlight in elasticsearch?