Outstanding job, Ryan. I think this is your best Railscast yet! Great job showing simultaneously how simple it is to use Thinking Sphinx and how much power you can leverage by drilling down into it. I'll be watching this one again. :)
@MikeInAZ, thanks for pointing that out. I normally use auto-completion for rake tasks as you may have noticed in this episode. If anyone's interested in that, see my dotfiles:
I just converted a large application from Ferret to Sphinx last week, and ran into a whole bunch of problems. I wrote a long <a href="http://blog.lrdesign.com/2008/07/fixing-problems-with-sphinx-search/">blog post</a> about the trials and tribulations I went through to get it working, including the fixes.
Though we were using ultrasphinx instead of thinking_sphinx, some of the struggles will be relevant to everyone using sphinx, for example the fact that foxy fixtures generates IDs that are often too large for sphinx's indexing strategy.
Just one question, how would you manage a more complicated search form.
Let's say that the user can say if he wants to search only by title, author or comments.
Is there an easy way to filter the indexes to use ?
First of all... thanks for the amazing episode again! Thanks for you r help with all the episodes you are making.
I have one thing what i dont understand. How can you manage this on a webhost server you dont maintain yourself? You have to restart it with ssh every time somebody wrote an article or smt else?
@Michael, it's best to set up a "cron" job to trigger the rake index task. If you aren't able to do this then you may be able to get your Rails app to spawn a background process to redo the index at a regular interval. I'm not exactly sure how that would work though.
Well done, I really learnt a lot from this cast and so from others. You have made it very simple to me to understand it without reading a single line on thinking sphinx. Thank you, waiting more interesting topics from you
Thanks for the podcast Ryan, I've been looking into the various search technologies for a while. As far as I can tell, one advantage to Ferret is it supports a "fuzzy" string match for search parameters. Any suggestions on how to handle this with Sphinx?
@Jeff, sphinx has some "fuzzy" matching for phrases, where there need to be a certain number of words a given distance a part. See the "extended" match mode for this behavior.
But I don't think this is what you're looking for. If you want fuzzy searching on a given word, where similar words are also found, then I don't think Sphinx supports this.
You may want to look into Xapian. It has some powerful features regarding this. Including stemming which can find variations of a given word (run, runs, running, etc.). it also supports spelling correction to find terms which are similar to a given word.
Hi, Ryan!
I always liked all your episodes! I'm buildind a Rails programming team at my company and I'm showing some of your episodes a day. Congratulations for the great work! :D
Best regards,
Felipe Giotto.
Jeff: Sphinx has support for fuzzy matching, but it's not enabled by default.
If you're using Thinking Sphinx, you can enable this by adding allow_star: true for every environment in config/sphinx.yml (ie: in a similar format to database.yml).
If anyone does work out the best route for highlighting search terms, please post back. I intend on giving them a solid background colour rather than using the highlight visual effect.
Sean Hess, I'm a new to js and Rails, are you suggesting an application.js script that takes the params and highlights the DOM elements that match?
I have been having trouble getting the :field_weights parameter to behave. I can't find it mentioned in the thinking_sphinx API entries on the various search methods:
http://ts.freelancing-gods.com/rdoc/
The best I can find is this:
http://ts.freelancing-gods.com/rdoc/classes/ThinkingSphinx/Index/Builder.html
Where it says :field_weights is an argument to the set_property method.
This seems to imply that field weighting happens when the index is built, not when the search is performed. I'd rather be able to declare my relevancy preferences at search time (like the video shows), not at index time (like the API seems to say).
The match_mode => :boolean ignores all field weighting, but match_mode => :extended honors them. This is a bit puzzling, since Extended seems to be just like Boolean, but with even more query features. Extended lets you use boolean ORs and NOTs, without losing the relevancy rankings.
Here's where Sphinx says it does that:
http://sphinxsearch.com/doc.html#weighting
And here's a code sample:
--- In app/models/article.rb
define_index block
...
# Declare a default relevancy table, which can be overridden at search
set_property :field_weights => { :title => 3, :subtitle => 2 }
...
end
--- In app/controllers/article_controller.rb
...
# Grab a page-load of articles, using title-centric field weights.
@articles = Article.search query_string,
:include => [:author, :comments], :match_mode => :extended,
:field_weights => { :title => 10, :subtitle => 6 },
:page => params[:page], :per_page => 20
I was thinking about the best way to run a daemon to reindex TS at a certain interval. Would it make sense to use the daemon gem in an initializer to kick off this process?
Hello,
I am trying to use thinking_sphinx, I saw a screencast on rails, I found it very useful to understand this.
I am having an issue when running 'rake thinking_sphinx:index', I get this message:
rake aborted!
undefined method `define_index' for #<Class:0x56ac908>
I am using windows for this, I have put the files for sphinx under %root\db\sphinx
I have not setup a sphinx.yml file.
I am new to all three 'sphinx', 'thinking_sphinx' and 'ruby on rails' so I am not able to figure out the right way to setup sphinx with thinking_sphinx.
Could you please help me understand this? I believe there is some issue with the shpinx setup.
Ryan, thank you so much for this screen cast. It helped me a great deal! I'm very happy with thinking_sphinx in my current fresh app, however, I plugged it into an old project and it overwrote the destroy methods of most activerecord objects, so I couldn't delete anything. Instead it threw a missing method error, for 'quoted_table_name'
Here's how to enable wildcard (*) or star searching with Thinking Sphinx: http://www.artofmission.com/articles/2008/10/24/wildcard-searches-with-thinking-sphinx
Great screen cast! I have one problem I came across. I want to add the condition "published_at IS NOT NULL" but I can't find a way to do this as a hash and just adding the string doesn't work. Any ideas?
Chris: If you want *all* results for that model to filter out records where published_at IS NULL, add 'where "published_at IS NOT NULL"' to your define_index block.
If it's only sometimes, add published_at as an attribute, and then sphinx should store null dates as 0's, so you can filter using :without => {:published_at => 0}
i m changing my project search from ferret to thinking sphinx, as ferret gives all data against * , thinking sphinx is not working for *,and how OR condition is handle is thinking sphinx. kindly give me solution.
If you want to get all results, just call #search without any arguments. If you want fuzzy matching using *'s, check out Ryan Heneise's link in comment 38.
Thanks Pat,Is sphinx providing highlight feature? right now i m using rails helper function for highlight, whether sphinx provide any
functionality like this?.
Sphinx itself does support this, but it's not yet been implemented in Thinking Sphinx (although I've had someone supply a patch in the last week - I just need to find time to review and possibly merge it).
Thanks Pat, i have almost completed my project into sphinx, and i found sphinx much much easier than ferret. Like it provides default will paginate support and its delta feature is also very very good and useful, Plus provides a lot of default functionalities. Pat i really appreciate and congrates you for developing this wonderful plugin. I also recommend others to use this plugin.
Hi Pat, there is one more issue,
sphinx is creating problem for rails polymorphic association
like
define_index do
indexes user.source.institution.name
end
it is not working for multiple source types,like if source types are Enterprise and Resource.and above condition is only for Resource.
is there any if condition while creating indexes like indexes user.source.institution.name if user.source_type == 'Resource'
Chetan: you can't do OR conditions across fields and attributes in Sphinx, nor can you check if a field is blank that simply. If you'd like to discuss the problem further, please visit the google group:
Some gotcha's to watch out for are:
1. Rails files path with spaces in it, so "My Documents" doesn't work. I created a new directory under c:\ and it worked.
2. Your ruby\bin directory must be in your environment variable "path". Remember to open a new command prompt to take the new settings if your using it for rake tasks.
@andrej: You can use jQuery to highlight the search term in the results. Here is a nice example how to select the search term and wrap it into a styled span tag with jQuery:
I want to know if you guys tested this sphinx in multiple application instance, single database, with multiple sphinx? What's the best thing to do? I'm having problem with regards to the indexing and starting of sphinx.
*Configurations*
Create a configuration file in _config/sphinx.yml_ with the following content. Port number varies on what you like, just make sure you will have different range if you are planing to use multiple instance of _searchd_(sphinx deamon)
<pre>
development:
port: 13000
test:
port: 13001
staging:
port: 13002
production:
port: 13003
</pre>
*Indexing*
Before _indexing_ you need to create a configuration file with each environment, or the environment that you are using, to do so...
<pre>
rake ts:config
</pre>
This will generate the _config/<env>.spinx.conf_
Before creating new index, or even reindexing you must delete the following files:
*_NOTES_*
Just make sure that your _port_ numbers differ from one application to another if you plan to run _sphinx/searchd_ in a single machine with the same database or even different database. _sphinx/searchd_ won't start if you already have a daemon running of the same port. Check your _log/searchd.log_ to verify that sphinx is running smoothly.
Seconding Jeremy's aforementioned file-path "gotcha". No spaces in your project file path. That's a no-go for me and renders the entire thing totally useless.
I'm bound by multiple file organizational systems for different hosts, as well as with project management schema. I can't just put my projects into any old folder. This is an amazing flaw - I'm shocked this is never mentioned in any of the sphinx tutorials.
@RyanBates Ezra from Engine Yard cites issues with corrupted indices with ferret in production (about a year ago). This was enough for me to stay away from Ferret.
I've been using Sphinx, via Ultrasphinx, for a couple years now, but thought I'd give this a try.
I've had 0 success in getting it to generate the correct sql for associated models.
I created a new project with this model:
class Article < ActiveRecord::Base
define_index do
indexes subject, :sortable => true
indexes content
indexes author.first_name, :as => :author_fname, :sortable => true
has author_id, created_at, updated_at
end
end
the sql generated looks like this:
sql_query = SELECT `articles`.`id` * 1 + 0 AS `id` , CAST
(`articles`.`subject` AS CHAR) AS `subject`, CAST(`articles`.`content`
AS CHAR) AS `content`, CAST(`articles`.`first_name` AS CHAR) AS
`author_fname`, `articles`.`id` AS `sphinx_internal_id`, 3448190970 AS
`class_crc`, '3448190970' AS `subclass_crcs`, 0 AS `sphinx_deleted`,
IFNULL(`articles`.`subject`, '') AS `subject_sort`, IFNULL
(`articles`.`first_name`, '') AS `author_fname_sort`,
`articles`.`author_id` AS `author_id`, UNIX_TIMESTAMP
(`articles`.`created_at`) AS `created_at`, UNIX_TIMESTAMP
(`articles`.`updated_at`) AS `updated_at` FROM `articles` WHERE
`articles`.`id` >= $start AND `articles`.`id` <= $end GROUP BY
`articles`.`id` ORDER BY NULL
It is trying to use the first_name column from the article object not the author!
I can't see anything I am doing wrong (I am not doing anything!)... so to me it looks like either a bug or a serious lack of documentation.
Hello All,
I wanted to implement sphinx search.
I have following models.
Blog
# post model has column "channel_feature_id"
has_many :posts, :foreign_key => "channel_feature_id"
------
Post
# Indexes for thinking
sphinx define_index do
indexes title
indexes abstract
indexes body
#indexes comments.body, :as => :comment_content
# OPTIONAL: keeps search index update
# First, add a boolean column to your model,
# named 'delta', and have it default to true.
# Then uncomment following line.
#set_property :delta => true
end
Great cast Ryan! I have gone on to allow searching over multiple indexes. Also due to the app Im working on have written the sql_query myself, if you do this you have to re-index:
Post
# Indexes for thinking
sphinx define_index do
indexes title
indexes abstract
indexes body
#indexes comments.body, :as => :comment_content
# OPTIONAL: keeps search index update
# First, add a boolean column to your model,
# named 'delta', and have it default to true.
# Then uncomment following line.
#set_property :delta => true
end
Hey i was working on an application where i need to search on the content of the file.
So is there a way in rails to index the content of the files stored on the local filesystem.
After installing the thinking_sphinx plugin i proceeded to install riddle because that is required for the latter plugin and found that I could not install riddle because of the following error.
WARNING: RubyGems 1.2+ index not found for:
RubyGems will revert to legacy indexes degrading performance.
ERROR: While executing gem ... (NoMethodError)
undefined method `gems' for #<Array:0x11ad58fc>
I have tried updating gems
installing new gems
i reinstalled rubygems and ruby and my os (OSX) with no luck.
Don't know what to do, and now I'm back to rails 1.2.3
I'm trying to use the plugin and get the following:
Sphinx cannot be found on your system. You may need to configure the following
settings in your config/sphinx.yml file:
* bin_path
* searchd_binary_name
* indexer_binary_name
I have the same issue as klumsyBird and gordon. I've spent hours trying to figure it out but just can't get it to work. I'm on Linux Centos 5.x. Installed it on Leapord OSX and it works perfect, but not on my linux server :(
Hi, could anyone guide me how to solve this problem. I have two models (person & job) which i need to index, it works fine when i index only one but when both of the models are indexed (when raking index), i get the following errors:
using config file '/var/www/nimbus-developer/config/development.sphinx.conf'...
ERROR: section 'job_extra_core_0' (type='source') already exists in /var/www/nim
bus-developer/config/development.sphinx.conf line 60 col 1.
FATAL: failed to parse config file '/var/www/nimbus-developer/config/development
.sphinx.conf'
Hey there!
I just want to see the search method which is used in the tutorial video and how the search results are listed when its found.
Can anybody help me?
Is there any reason why Thinking Sphinx would not read the config/sphinx.yml file? I'm using Sphinx 0.9.9 and Thinking Sphinx 1.3.16 (because I'm stuck with a Rails 1.2.6 app). Thank you for any suggestions!
Outstanding job, Ryan. I think this is your best Railscast yet! Great job showing simultaneously how simple it is to use Thinking Sphinx and how much power you can leverage by drilling down into it. I'll be watching this one again. :)
Long time fan here. Thanks for the continued outstanding quality of your railscasts.
Ferret used to be the de-facto choice. Any reasons not to use Ferret? Why having chosen Sphinx over other solutions?
Is it possible to perform searches across several models?
Thanks for the great screen casts. Keep them coming!
Is it possible to do result highlighting with Sphinx to show the search terms?
Crazy timing! Just implemented Thinking Sphinx into a major project in place of Solr.
@Swami, I haven't tried Ferret, so I can't comment too much about that. I have heard it has some stability issues, but that's all I know.
As for searching across multiple models, I believe it's possible with "ThinkingSphinx::Search.search", but I haven't tried it.
@andrej, I don't think Sphinx gives an easy way to highlight the search terms. However you could use the "highlight" helper method Rails provides.
Thanks for the screencast Ryan, I just implemented TS last week.
Also, instead of thinking_sphinx:index command, you can just type ts:in
ts:start
ts:stop
@MikeInAZ, thanks for pointing that out. I normally use auto-completion for rake tasks as you may have noticed in this episode. If anyone's interested in that, see my dotfiles:
http://github.com/ryanb/dotfiles
Thanks for the 'cast, Ryan.
I just converted a large application from Ferret to Sphinx last week, and ran into a whole bunch of problems. I wrote a long <a href="http://blog.lrdesign.com/2008/07/fixing-problems-with-sphinx-search/">blog post</a> about the trials and tribulations I went through to get it working, including the fixes.
Though we were using ultrasphinx instead of thinking_sphinx, some of the struggles will be relevant to everyone using sphinx, for example the fact that foxy fixtures generates IDs that are often too large for sphinx's indexing strategy.
Whoops, sorry about that, didn't realize your comments don't allow links. (suggestion: a preview button)
Here's the bare URL I mentioned in my previous comment:
http://blog.lrdesign.com/2008/07/fixing-problems-with-sphinx-search/
Outstanding screencast !
Just one question, how would you manage a more complicated search form.
Let's say that the user can say if he wants to search only by title, author or comments.
Is there an easy way to filter the indexes to use ?
@Swami: I just read this post (http://freelancing-gods.com/posts/thinking_sphinx_reborn)
Multi-model searching is (obviously) supported:
ThinkingSphinx::Search.search "help"
Haven't tried it though. Hope it helps!
@Evan, preview button is coming very soon. :)
@jblanche, you can use the :conditions hash to search by specific fields. I think that will do what you need.
How about a episode showing how to use you dotfiles...looks nice on this episode :P
Hey, just a suggestion, what about put the last comment in the top of the page. So, we can read from the more recent to the oldest. =P
My English listening is really weak :-), can you please write somewhere what provides instant indexing?
@Rafael, thanks for the suggestion. Most blogs do the earlier comments first, so I think it's best to follow that convention. But I'll consider it.
@Xavier, are you referring to delta indexing? See the bottom of the Thinking Sphinx Usage page for more info on that.
http://ts.freelancing-gods.com/usage.html
@Ryan, I'd agree with Rafel's idea.
Maybe the comment form could also be moved up (and made collapsable to save space?). This might encourage users to post (even) more comments :)
First of all... thanks for the amazing episode again! Thanks for you r help with all the episodes you are making.
I have one thing what i dont understand. How can you manage this on a webhost server you dont maintain yourself? You have to restart it with ssh every time somebody wrote an article or smt else?
@Michael, it's best to set up a "cron" job to trigger the rake index task. If you aren't able to do this then you may be able to get your Rails app to spawn a background process to redo the index at a regular interval. I'm not exactly sure how that would work though.
Well done, I really learnt a lot from this cast and so from others. You have made it very simple to me to understand it without reading a single line on thinking sphinx. Thank you, waiting more interesting topics from you
Thanks for the podcast Ryan, I've been looking into the various search technologies for a while. As far as I can tell, one advantage to Ferret is it supports a "fuzzy" string match for search parameters. Any suggestions on how to handle this with Sphinx?
@Jeff, sphinx has some "fuzzy" matching for phrases, where there need to be a certain number of words a given distance a part. See the "extended" match mode for this behavior.
But I don't think this is what you're looking for. If you want fuzzy searching on a given word, where similar words are also found, then I don't think Sphinx supports this.
You may want to look into Xapian. It has some powerful features regarding this. Including stemming which can find variations of a given word (run, runs, running, etc.). it also supports spelling correction to find terms which are similar to a given word.
Is there anybody lucky with installing sphinx on a Windows computer?
if i use the install link here above it says that vendor/plugin/sphinx is removed again. And after that it wont start sphinx
Hi, Ryan!
I always liked all your episodes! I'm buildind a Rails programming team at my company and I'm showing some of your episodes a day. Congratulations for the great work! :D
Best regards,
Felipe Giotto.
hi Ryan....is there any possibility that you can zip and publish this project on your site (and all future projects)...
Jeff: Sphinx has support for fuzzy matching, but it's not enabled by default.
If you're using Thinking Sphinx, you can enable this by adding allow_star: true for every environment in config/sphinx.yml (ie: in a similar format to database.yml).
@rob you can find the full code by clicking the "Full Source Code" link at the bottom of the code samples.
@andrej: Don't you guys use Javascript for anything? :) Thanks for the great episode Ryan!
If anyone does work out the best route for highlighting search terms, please post back. I intend on giving them a solid background colour rather than using the highlight visual effect.
Sean Hess, I'm a new to js and Rails, are you suggesting an application.js script that takes the params and highlights the DOM elements that match?
Indexing is taking ages on my MBP...
p.s. thanks again, Ryan.
Is there an error in the video?
I have been having trouble getting the :field_weights parameter to behave. I can't find it mentioned in the thinking_sphinx API entries on the various search methods:
http://ts.freelancing-gods.com/rdoc/
The best I can find is this:
http://ts.freelancing-gods.com/rdoc/classes/ThinkingSphinx/Index/Builder.html
Where it says :field_weights is an argument to the set_property method.
This seems to imply that field weighting happens when the index is built, not when the search is performed. I'd rather be able to declare my relevancy preferences at search time (like the video shows), not at index time (like the API seems to say).
Am I missing something here?
Oops. Forgot to attach my email address to that message.
Great work, Ryan. Your generous contributions of work have saved me a great deal of aggravation ever since I discovered them.
Okay, I think I found the problem.
The match_mode => :boolean ignores all field weighting, but match_mode => :extended honors them. This is a bit puzzling, since Extended seems to be just like Boolean, but with even more query features. Extended lets you use boolean ORs and NOTs, without losing the relevancy rankings.
Here's where Sphinx says it does that:
http://sphinxsearch.com/doc.html#weighting
And here's a code sample:
--- In app/models/article.rb
define_index block
...
# Declare a default relevancy table, which can be overridden at search
set_property :field_weights => { :title => 3, :subtitle => 2 }
...
end
--- In app/controllers/article_controller.rb
...
# Grab a page-load of articles, using title-centric field weights.
@articles = Article.search query_string,
:include => [:author, :comments], :match_mode => :extended,
:field_weights => { :title => 10, :subtitle => 6 },
:page => params[:page], :per_page => 20
I was thinking about the best way to run a daemon to reindex TS at a certain interval. Would it make sense to use the daemon gem in an initializer to kick off this process?
Hello,
I am trying to use thinking_sphinx, I saw a screencast on rails, I found it very useful to understand this.
I am having an issue when running 'rake thinking_sphinx:index', I get this message:
rake aborted!
undefined method `define_index' for #<Class:0x56ac908>
I am using windows for this, I have put the files for sphinx under %root\db\sphinx
I have not setup a sphinx.yml file.
I am new to all three 'sphinx', 'thinking_sphinx' and 'ruby on rails' so I am not able to figure out the right way to setup sphinx with thinking_sphinx.
Could you please help me understand this? I believe there is some issue with the shpinx setup.
Thank you for your help
-Meka
Ryan, thank you so much for this screen cast. It helped me a great deal! I'm very happy with thinking_sphinx in my current fresh app, however, I plugged it into an old project and it overwrote the destroy methods of most activerecord objects, so I couldn't delete anything. Instead it threw a missing method error, for 'quoted_table_name'
Matt: what version of Rails are you using in your old project? I try to keep TS working for 1.2.6, but every now and then it breaks.
Might be worth continuing this discussion on the google group:
http://groups.google.com/group/thinking-sphinx
Here's how to enable wildcard (*) or star searching with Thinking Sphinx: http://www.artofmission.com/articles/2008/10/24/wildcard-searches-with-thinking-sphinx
Great screen cast! I have one problem I came across. I want to add the condition "published_at IS NOT NULL" but I can't find a way to do this as a hash and just adding the string doesn't work. Any ideas?
Chris: If you want *all* results for that model to filter out records where published_at IS NULL, add 'where "published_at IS NOT NULL"' to your define_index block.
If it's only sometimes, add published_at as an attribute, and then sphinx should store null dates as 0's, so you can filter using :without => {:published_at => 0}
Thanks Pat! The second solution was what I was looking for.
Hi Ryan,
Is there a way to get the score or relevance of the returned results?
Thanks !
Ryan: The result set you get back from the search call should respond to each_with_weighting:
@results.each_with_weighting do |result, weight|
# ...
end
i m changing my project search from ferret to thinking sphinx, as ferret gives all data against * , thinking sphinx is not working for *,and how OR condition is handle is thinking sphinx. kindly give me solution.
Hi Faisal
If you want to get all results, just call #search without any arguments. If you want fuzzy matching using *'s, check out Ryan Heneise's link in comment 38.
Thanks Pat,Is sphinx providing highlight feature? right now i m using rails helper function for highlight, whether sphinx provide any
functionality like this?.
Hi Faisal
Sphinx itself does support this, but it's not yet been implemented in Thinking Sphinx (although I've had someone supply a patch in the last week - I just need to find time to review and possibly merge it).
Thanks Pat, i have almost completed my project into sphinx, and i found sphinx much much easier than ferret. Like it provides default will paginate support and its delta feature is also very very good and useful, Plus provides a lot of default functionalities. Pat i really appreciate and congrates you for developing this wonderful plugin. I also recommend others to use this plugin.
Hi Pat, there is one more issue,
sphinx is creating problem for rails polymorphic association
like
define_index do
indexes user.source.institution.name
end
it is not working for multiple source types,like if source types are Enterprise and Resource.and above condition is only for Resource.
is there any if condition while creating indexes like indexes user.source.institution.name if user.source_type == 'Resource'
So how can i handle this.
Hi Faisal
It should work, if there's existing records for both Enterprise and Resource instances tied to your user model.
TS needs existing records to figure out what to join on.
Might be best to continue this discussion on the google group:
http://groups.google.com/group/thinking-sphinx
How we can use OR condition in Sphinx ?
Eg. (zip!='' AND zip is NOT NULL) OR (is_nationwide=1)
Chetan: you can't do OR conditions across fields and attributes in Sphinx, nor can you check if a field is blank that simply. If you'd like to discuss the problem further, please visit the google group:
http://groups.google.com/group/thinking-sphinx
For those trying to install sphinx on windows, I found this article to be very helpful.
http://www.expressionlab.com/2008/11/2/thinking-sphinx-on-windows
Some gotcha's to watch out for are:
1. Rails files path with spaces in it, so "My Documents" doesn't work. I created a new directory under c:\ and it worked.
2. Your ruby\bin directory must be in your environment variable "path". Remember to open a new command prompt to take the new settings if your using it for rake tasks.
Hope it helps other users.
Is there away I can check if sphinx is running, so I can decide of to call the Model.search call in code?
Thanks.
@andrej: You can use jQuery to highlight the search term in the results. Here is a nice example how to select the search term and wrap it into a styled span tag with jQuery:
http://www.ryancoughlin.com/2008/12/20/jquery-search-and-highlight/
excuse me, I don't know is this right place for my query.
I tried to install thinking-sphinx but i got empty folder
F:\RubyProjects\SphinxTest>ruby script/plugin install git://github.com/freelancing-god/thinking-sphinx.git
Then I download thinking-sphinx from http://github.com/freelancing-god/thinking-sphinx/tree/master
and install it as a gem , as following
C:\Think-Sphinx>gem build thinking-sphinx.gemspec
Successfully built RubyGem
Name: thinking-sphinx
Version: 1.1.6
File: thinking-sphinx-1.1.6.gem
C:\Think-Sphinx>gem install thinking-sphinx-1.1.6.gem
Successfully installed thinking-sphinx-1.1.6
1 gem installed
Installing ri documentation for thinking-sphinx-1.1.6...
Installing RDoc documentation for thinking-sphinx-1.1.6...
Now when I run rake task. I got following error.
F:\RubyProjects\SphinxTest>rake thinking-sphinx:index
(in F:/RubyProjects/SphinxTest)
rake aborted!
Don't know how to build task 'thinking-sphinx:index'
(See full trace by running task with --trace)
Why this is not working?
I want to know if you guys tested this sphinx in multiple application instance, single database, with multiple sphinx? What's the best thing to do? I'm having problem with regards to the indexing and starting of sphinx.
Thanks
Oh I figured it out now.
*SPHINK*
by *_Julius Francisco_*
*INSTALL* (_Leopard with Macports_)
Install sphinx via macports
<pre>
sudo port install sphinx
</pre>
Install thinkingsphinx
<pre>
script/plugin install git://github.com/freelancing-god/thinking-sphinx.git
</pre>
*Configurations*
Create a configuration file in _config/sphinx.yml_ with the following content. Port number varies on what you like, just make sure you will have different range if you are planing to use multiple instance of _searchd_(sphinx deamon)
<pre>
development:
port: 13000
test:
port: 13001
staging:
port: 13002
production:
port: 13003
</pre>
*Indexing*
Before _indexing_ you need to create a configuration file with each environment, or the environment that you are using, to do so...
<pre>
rake ts:config
</pre>
This will generate the _config/<env>.spinx.conf_
Before creating new index, or even reindexing you must delete the following files:
_indexed files_
<pre>
rm -rf _db/sphinx/<env>/*_
</pre>
_pid and logs_
<pre>
rm -rf _log/searchd.*_
</pre>
Now, time to do index:
_indexing and re-index_
<pre>
rake ts:index
</pre>
Starting sphinx:
<pre>
rake ts:start
</pre>
Stopping sphinx:
<pre>
rm -rf _log/searchd.<env>.pid_
rake ts:stop
</pre>
*_NOTES_*
Just make sure that your _port_ numbers differ from one application to another if you plan to run _sphinx/searchd_ in a single machine with the same database or even different database. _sphinx/searchd_ won't start if you already have a daemon running of the same port. Check your _log/searchd.log_ to verify that sphinx is running smoothly.
Seconding Jeremy's aforementioned file-path "gotcha". No spaces in your project file path. That's a no-go for me and renders the entire thing totally useless.
I'm bound by multiple file organizational systems for different hosts, as well as with project management schema. I can't just put my projects into any old folder. This is an amazing flaw - I'm shocked this is never mentioned in any of the sphinx tutorials.
My English listening is really weak :-)
Hi, Ryan! Great post. thanks!
Do u think there is a way to combine this one with Advanced Search? Whats the hint? :)
Especially those minimum_price & maximum_price in advanced search screencast.. or thinking sphinx only supports ranges?
thanks!
@RyanBates Ezra from Engine Yard cites issues with corrupted indices with ferret in production (about a year ago). This was enough for me to stay away from Ferret.
http://www.ruby-forum.com/topic/137629#612225
thanks great introduction www.momo.ie
I've been using Sphinx, via Ultrasphinx, for a couple years now, but thought I'd give this a try.
I've had 0 success in getting it to generate the correct sql for associated models.
I created a new project with this model:
class Article < ActiveRecord::Base
define_index do
indexes subject, :sortable => true
indexes content
indexes author.first_name, :as => :author_fname, :sortable => true
has author_id, created_at, updated_at
end
end
the sql generated looks like this:
sql_query = SELECT `articles`.`id` * 1 + 0 AS `id` , CAST
(`articles`.`subject` AS CHAR) AS `subject`, CAST(`articles`.`content`
AS CHAR) AS `content`, CAST(`articles`.`first_name` AS CHAR) AS
`author_fname`, `articles`.`id` AS `sphinx_internal_id`, 3448190970 AS
`class_crc`, '3448190970' AS `subclass_crcs`, 0 AS `sphinx_deleted`,
IFNULL(`articles`.`subject`, '') AS `subject_sort`, IFNULL
(`articles`.`first_name`, '') AS `author_fname_sort`,
`articles`.`author_id` AS `author_id`, UNIX_TIMESTAMP
(`articles`.`created_at`) AS `created_at`, UNIX_TIMESTAMP
(`articles`.`updated_at`) AS `updated_at` FROM `articles` WHERE
`articles`.`id` >= $start AND `articles`.`id` <= $end GROUP BY
`articles`.`id` ORDER BY NULL
It is trying to use the first_name column from the article object not the author!
I can't see anything I am doing wrong (I am not doing anything!)... so to me it looks like either a bug or a serious lack of documentation.
<script src='http://pastie.org/524112.js'></script>
Hello All,
I wanted to implement sphinx search.
I have following models.
Blog
# post model has column "channel_feature_id"
has_many :posts, :foreign_key => "channel_feature_id"
------
Post
# Indexes for thinking
sphinx define_index do
indexes title
indexes abstract
indexes body
#indexes comments.body, :as => :comment_content
# OPTIONAL: keeps search index update
# First, add a boolean column to your model,
# named 'delta', and have it default to true.
# Then uncomment following line.
#set_property :delta => true
end
belongs_to :blog
has_many :comments
# GET published posts
named_scope :published, :conditions => [ 'published = ?', true ]
------
Comment
belongs_to :post
------
In my search action i have
blog.posts.published.search ( params[:search] )
i am unable to figure out error.
Missing Attribute for Foreign
Key channel_feature_id
here is full trace.......
vendor/plugins/thinking-sphinx/lib/thinking_sphinx/active_record/has_many_a ssociation.rb:18:in
`search'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/named_ scope.rb:158:in
`send'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/named_ scope.rb:158:in
`method_missing'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/base.r b:1857:in
`with_scope'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/associ ations/association_proxy.rb:164:in
`send'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/associ ations/association_proxy.rb:164:in
`with_scope'
(__DELEGATION__):2:in `__send__'
(__DELEGATION__):2:in `with_scope'
app/controllers/blogs_controller.rb:36:in `search'
Thanks,
Sandip
Thanks for the screencast Ryan, I just implemented TS last week.
Thanks Ryan. Do you know how to implement excerpts with thinking sphinx?
Great cast Ryan! I have gone on to allow searching over multiple indexes. Also due to the app Im working on have written the sql_query myself, if you do this you have to re-index:
rake thinking_sphinx:index INDEX_ONLY=true
Not to over write your custom sql!
Post
# Indexes for thinking
sphinx define_index do
indexes title
indexes abstract
indexes body
#indexes comments.body, :as => :comment_content
# OPTIONAL: keeps search index update
# First, add a boolean column to your model,
# named 'delta', and have it default to true.
# Then uncomment following line.
#set_property :delta => true
end
belongs_to :blog
has_many :comments
# GET published posts
named_scope :published, :conditions => [ 'published = ?', true ]
------
Comment
belongs_to :post
------
In my search action i have
blog.posts.published.search ( params[:search] )
i am unable to figure out error.
Missing Attribute for Foreign
Key channel_feature_id
here is full trace.......
vendor/plugins/thinking-sphinx/lib/thinking_sphinx/active_record/has_many_a ssociation.rb:18:in
`search'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/named_ scope.rb:158:in
`send'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/named_ scope.rb:158:in
`method_missing'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/base.r b:1857:in
`with_scope'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/associ ations/association_proxy.rb:164:in
`send'
/home/sandip/.gem/ruby/1.8/gems/activerecord-2.1.2/lib/active_record/associ ations/association_proxy.rb:164:in
`with_scope'
(__DELEGATION__):2:in `__send__'
(__DELEGATION__):2:in `with_scope'
app/controllers/blogs_controller.rb:36:in `search'
@Tair you can do a min and max price search by using a range:
:with => {:price => 10..100}
Thanks for the screencast.....
Hey i was working on an application where i need to search on the content of the file.
So is there a way in rails to index the content of the files stored on the local filesystem.
After installing the thinking_sphinx plugin i proceeded to install riddle because that is required for the latter plugin and found that I could not install riddle because of the following error.
WARNING: RubyGems 1.2+ index not found for:
RubyGems will revert to legacy indexes degrading performance.
ERROR: While executing gem ... (NoMethodError)
undefined method `gems' for #<Array:0x11ad58fc>
I have tried updating gems
installing new gems
i reinstalled rubygems and ruby and my os (OSX) with no luck.
Don't know what to do, and now I'm back to rails 1.2.3
I'm trying to use the plugin and get the following:
Sphinx cannot be found on your system. You may need to configure the following
settings in your config/sphinx.yml file:
* bin_path
* searchd_binary_name
* indexer_binary_name
to: klumsyBird
http://github.com/freelancing-god/thinking-sphinx/blob/6ecbc568ff8851c36c81c1784bf7f866d4077ca0/lib/thinking_sphinx/auto_version.rb
Update your sphinx version to 0.9.8 or 0.9.9
Hi,
In command prompt i executed:
c:\myproject>ruby script/plugin install git://github.com/freelancing-god/thinking-sphinx.git
It generates a blank "thinking-sphinx" folder inside the "vendor\plugins" folder.
what i should do....
i m also facing the same problem with "riddle" as per klumsyBird.
Can anyone please shine some light into this?
I have the same issue as klumsyBird and gordon. I've spent hours trying to figure it out but just can't get it to work. I'm on Linux Centos 5.x. Installed it on Leapord OSX and it works perfect, but not on my linux server :(
Is it possible to do result highlighting with Sphinx to show the search terms?
How would i show the search box on my view and what would the code be????
Hi Javon
There is a method called "Excerpts" that was supposed to be implemented in the latest version of thinking sphinx
Check that, it may help you
I have the same "riddle" issue as klumsyBird and gordon and Theokie. Sphinx compiled on an Arch Linux machine.
Also installed riddle gem, with no success. Any other Sphinx configuration needed (maybe in /etc/sphinx/)? Any ideas ?
Hi, could anyone guide me how to solve this problem. I have two models (person & job) which i need to index, it works fine when i index only one but when both of the models are indexed (when raking index), i get the following errors:
using config file '/var/www/nimbus-developer/config/development.sphinx.conf'...
ERROR: section 'job_extra_core_0' (type='source') already exists in /var/www/nim
bus-developer/config/development.sphinx.conf line 60 col 1.
FATAL: failed to parse config file '/var/www/nimbus-developer/config/development
.sphinx.conf'
Your help is much appreciated!
Hey there!
I just want to see the search method which is used in the tutorial video and how the search results are listed when its found.
Can anybody help me?
Thanks for the article.
I am getting the following error when I a ts:index/reindex/config
undefined method 'descendants' for ActiveRecord::Base:Class
I am running OS X 10.5.7,
thinking-sphinx(2.0.0.rc1 5fe4a03), rails 3.0.0.beta4.
Has anyone else got this error? Is there an issue with TS gem working with the rails3.0 version on OS X?
Thanks, Sam
Is there any reason why Thinking Sphinx would not read the config/sphinx.yml file? I'm using Sphinx 0.9.9 and Thinking Sphinx 1.3.16 (because I'm stuck with a Rails 1.2.6 app). Thank you for any suggestions!
A good mongoid cast would fit in nicely...of course would need xmlpipe
Of course you could do one with Sunspot....hopefully using mongoid
sphinx works well for english, how can I configure it for arabic language. kindly reply.
Thanks in advance.
Got bug when using #120 Thinking Sphinx and models what include concerns like in #398-service-objects
card.rb
concerns/userable.rb
Hi!
This is most likely because of your version of Thinking Sphinx. Upgrading to 2.0.14 should fix the problem.
2.0.14 - January 2nd 2013
* Fix model loading for Rails 3.2.9 when subdirectories exist (Kenn Ejima).
https://github.com/pat/thinking-sphinx/blob/master/HISTORY