ThinkingSphinx exits, enters ActsAsSolrReloaded
I used to work with ThinkingSphinx until the day I needed to index documents with dynamic attributes. As Sphinx indexes data from the result of an SQL query, the goal didn’t seem possible.
I decided then to take another look at Solr. Solr, differently from Sphinx, is an HTTP server and indexes data from posted XML documents. Each document can have a different structure, so it fits perfectly with the model of dynamic attributes.
Thiago Jackwin, aka RailsFreaks, created a plugin that integrates Rails with Solr called acts_as_solr. The plugin is very good, but Thiago disappeared from the map some time ago, he lost the domain, left GitHub and doesn’t answer emails any more. As a result of this, different forks and forks of forks have been created and the Git tree became a mess.
Annoyed with the situation of the project, I decided to fork the fork I liked the most and created a new repository called acts_as_solr_reloaded, with new features. This way, I hope the project gets easier to be found and that it gives more trust. I’m also compromising myself to keep the repository up to date and to pull contributions.
As today, the new features acts_as_solr_reloaded comes with are:
- support for dynamic attributes
- geo-localization or geo-spatial search
- integration with acts-as-taggable-on
- highlighting
- relevance ranking
To support geo-localization in Solr, it needed to be updated to the version 1.4 .
To make easier the experience of working with dynamic attributes and geo-localization, a few generators that setup the database were added to the project. You can use them like this:
script/generate dynamic_attributes_migration script/generate local_migration
You can after define your model this way:
class Document < ActiveRecord::Base acts_as_solr :dynamic_attributes => true, :spatial => true, :taggable => true end
Note that with :taggable => true you dont need to define your model as acts_as_taggable_on :tags, it’s done automatically.
To better demonstrate the new features in acts_as_solr_reloaded, I recorded a small video of five minutes showing the functionalities in action, hope you like it:
New features in ActsAsSolrReloaded from Diego Carrion on Vimeo.
Note that in the video I used acts_as_taggable_on :tags and :taggable => true, at the time of the recording this both declarations were necessary, not anymore.
If you appreciate this work, please consider to recommend me at Working With Rails.
Have you looked at sunspot? http://github.com/outoftime/sunspot_rails
Hi Nadeem, I discovered it after I implemented the mentioned features in acts_as_solr_reloaded. I don’t know if it supports the same functionalities that acts_as_solr_reloaded has, but seems a cool option to be analyzed.
I’m guessing that there must have been other reasons you moved from sphinx to solr - sphinx can index dynamic data using the xmlpipe data source.
Have you taken a look at Sunspot and Sunspot Rails?
Oops, what Nadeem Bitar said.
Hi Jeremy, I didn’t know about the xmlpipe until now. I found this:
http://www.sphinxsearch.com/docs/current.html#xmlpipe2
When I had the problem, I tried to find something working with ThinkingSphinx but didn’t find anything. I just tried to discover if ThinkingSphinx supports xmlpipe2 but didn’t find any reference from the official repository, just this commit from a fork:
http://github.com/ebeigarts/thinking-sphinx/commit/8ccd28995ba23b1ee506f226e9a6002f3d86bbee
The principal reason I change Sphinx for Solr was the dynamic attributes, but another features I like are the live updating and that I can use it with any database, as they don’t communicate.
I know music is personal taste, but the music in your video actively prevented me from watching the video..
Besides, good work!
Thanks for the feedback Fabio. In future videos I will not record the background music and just add something softly later
You mention that it should be able to work with “any rails model”, but then it seems like it requires ActiveRecord. So it won’t really work with any rails model… only models derived from ActiveRecord. I’m trying to use with MongoDB and don’t use AR. I’ve already got sphinx working with Mongo thru xmlpipe, which works well. But I’d love to try out yours in comparison if it can work without requiring AR.
Hi Matt, I meant to say that it works with ActiveRecord on any database, sorry for don’t being explicit.
Btw, are you using any plugin for Sphinx? Also, which adapter are you using for MongoDB?
Thanks for your feedback, it was very good and I will try to adapt acts_as_solr_reloaded to work with Mongo.
[...] ThinkingSphinx exits, enters ActsAsSolrReloaded | Diego Carrion [...]
[...] ThinkingSphinx exits, enters ActsAsSolrReloaded [...]
Hi Diego. I am playing with acts_as_solr_reloaded and I am having trouble getting back facet counts. Not sure if this this is a solr config issue or what. I am using jetty-solr with its out-of-the box configuration.
I have a model that has this:
acts_as_solr :fields => [:body], :facets => [:source_type, :project_id]
I do a find like this:
find_by_solr(’tester’, {
:highlight => {:fields => ‘body’},
:facets=>{:zeros=>true,:fields=>[:source_type, :project_id]}
})
and not facet counts come back (though I know there is matching data:
facet_fields”=>{”source_type_facet”=>{}, “project_id_facet”=>{}}
Even hitting solr directly yeilds no facet counts:
http://localhost:8982/solr/select/?q=tester%0D%0A&version=2.2&start=0&rows=10&indent=on&facet=true&facet.field=project_id_facet
Thoughts?
Also - searching for ‘*:*’ fails. I would typically use this if I wanted to do a faceted search without any text query.
The error showing up in solr shows that _t is being appended:
May 4, 2010 12:11:36 PM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: org.apache.lucene.queryParser.ParseException: Cannot parse ‘(*_t:*) AND (t
ype_s:Post)’: ‘*’ or ‘?’ not allowed as first character in WildcardQuery
Another question
Is can act_as_solr_reloaded be used with will_paginate?
Hi Homer, acts_as_solr_reloaded is now compatible with will_paginate, I just added the info to the readme:
http://github.com/dcrec1/acts_as_solr_reloaded
Hi, Diego
how used acts_as_solr_reloaded with rails 3 and mongoID?
thank you
Hi Fabrio, acts_as_solr_reloaded is not yet compatible with MongoID, but I will try to implemented it. If you wanna help, this module may help you as a base:
http://github.com/dcrec1/acts_as_solr_reloaded/blob/master/lib/acts_as_solr/mongo_mapper.rb
[...]below you’ll find the link to some sites that we think you should visit[...]…
hi, i’m currently using acts_as_solr_reloaded but i’m having trouble indexing rich documents, receiving tika exceptions. i wonder if its able to support?
Hi, Diego
how do to work in rails 3?
i saw that acts_as_taggable_on work different in rails 3, the acts_as_solr_reloaded change too?
I would like to thank you for the efforts you have put in writing this web site. I’m hoping the same high-grade website post from you in the upcoming as well. In fact your creative writing skills has encouraged me to get my own website now. Actually the blogging is spreading its wings rapidly. Your write up is a good example of it.
Great clarification. I love to make out the print Martha