ThinkingSphinx exits, enters ActsAsSolrReloaded

I used to work with ThinkingSphinx until the day I needed to index documents with dynamic attributes. As Sphinx indexes data from the result of an SQL query, the goal didn’t seem possible.

I decided then to take another look at Solr. Solr, differently from Sphinx, is an HTTP server and indexes data from posted XML documents. Each document can have a different structure, so it fits perfectly with the model of dynamic attributes.

Thiago Jackwin, aka RailsFreaks, created a plugin that integrates Rails with Solr called acts_as_solr. The plugin is very good, but Thiago disappeared from the map some time ago, he lost the domain, left GitHub and doesn’t answer emails any more. As a result of this, different forks and forks of forks have been created and the Git tree became a mess.

Annoyed with the situation of the project, I decided to fork the fork I liked the most and created a new repository called acts_as_solr_reloaded, with new features. This way, I hope the project gets easier to be found and that it gives more trust. I’m also compromising myself to keep the repository up to date and to pull contributions.

As today, the new features acts_as_solr_reloaded comes with are:

  • support for dynamic attributes
  • geo-localization or geo-spatial search
  • integration with acts-as-taggable-on
  • highlighting
  • relevance ranking

To support geo-localization in Solr, it needed to be updated to the version 1.4 .

To make easier the experience of working with dynamic attributes and geo-localization, a few generators that setup the database were added to the project. You can use them like this:

script/generate dynamic_attributes_migration
script/generate local_migration

You can after define your model this way:

class Document < ActiveRecord::Base
  acts_as_solr :dynamic_attributes => true,
               :spatial => true,
               :taggable => true
end

Note that with :taggable => true you dont need to define your model as acts_as_taggable_on :tags, it’s done automatically.

To better demonstrate the new features in acts_as_solr_reloaded, I recorded a small video of five minutes showing the functionalities in action, hope you like it:

New features in ActsAsSolrReloaded from Diego Carrion on Vimeo.

Note that in the video I used acts_as_taggable_on :tags and :taggable => true, at the time of the recording this both declarations were necessary, not anymore.

If you appreciate this work, please consider to recommend me at Working With Rails.

26 thoughts on “ThinkingSphinx exits, enters ActsAsSolrReloaded

  1. Hi Nadeem, I discovered it after I implemented the mentioned features in acts_as_solr_reloaded. I don’t know if it supports the same functionalities that acts_as_solr_reloaded has, but seems a cool option to be analyzed.

  2. Hi Jeremy, I didn’t know about the xmlpipe until now. I found this:

    http://www.sphinxsearch.com/docs/current.html#xmlpipe2

    When I had the problem, I tried to find something working with ThinkingSphinx but didn’t find anything. I just tried to discover if ThinkingSphinx supports xmlpipe2 but didn’t find any reference from the official repository, just this commit from a fork:

    http://github.com/ebeigarts/thinking-sphinx/commit/8ccd28995ba23b1ee506f226e9a6002f3d86bbee

    The principal reason I change Sphinx for Solr was the dynamic attributes, but another features I like are the live updating and that I can use it with any database, as they don’t communicate.

  3. I know music is personal taste, but the music in your video actively prevented me from watching the video..
    Besides, good work!

  4. You mention that it should be able to work with “any rails model”, but then it seems like it requires ActiveRecord. So it won’t really work with any rails model… only models derived from ActiveRecord. I’m trying to use with MongoDB and don’t use AR. I’ve already got sphinx working with Mongo thru xmlpipe, which works well. But I’d love to try out yours in comparison if it can work without requiring AR.

  5. Hi Matt, I meant to say that it works with ActiveRecord on any database, sorry for don’t being explicit.

    Btw, are you using any plugin for Sphinx? Also, which adapter are you using for MongoDB?

    Thanks for your feedback, it was very good and I will try to adapt acts_as_solr_reloaded to work with Mongo.

  6. Hi Diego. I am playing with acts_as_solr_reloaded and I am having trouble getting back facet counts. Not sure if this this is a solr config issue or what. I am using jetty-solr with its out-of-the box configuration.

    I have a model that has this:
    acts_as_solr :fields => [:body], :facets => [:source_type, :project_id]

    I do a find like this:
    find_by_solr(‘tester’, {
    :highlight => {:fields => ‘body’},
    :facets=>{:zeros=>true,:fields=>[:source_type, :project_id]}
    })

    and not facet counts come back (though I know there is matching data:
    facet_fields”=>{“source_type_facet”=>{}, “project_id_facet”=>{}}

    Even hitting solr directly yeilds no facet counts:

    http://localhost:8982/solr/select/?q=tester&version=2.2&start=0&rows=10&indent=on&facet=true&facet.field=project_id_facet

    Thoughts?

  7. Also – searching for ‘*:*’ fails. I would typically use this if I wanted to do a faceted search without any text query.

    The error showing up in solr shows that _t is being appended:

    May 4, 2010 12:11:36 PM org.apache.solr.common.SolrException log
    SEVERE: org.apache.solr.common.SolrException: org.apache.lucene.queryParser.ParseException: Cannot parse ‘(*_t:*) AND (t
    ype_s:Post)’: ‘*’ or ‘?’ not allowed as first character in WildcardQuery

  8. hi, i’m currently using acts_as_solr_reloaded but i’m having trouble indexing rich documents, receiving tika exceptions. i wonder if its able to support?

  9. Hi, Diego

    how do to work in rails 3?

    i saw that acts_as_taggable_on work different in rails 3, the acts_as_solr_reloaded change too?

  10. I would like to thank you for the efforts you have put in writing this web site. I’m hoping the same high-grade website post from you in the upcoming as well. In fact your creative writing skills has encouraged me to get my own website now. Actually the blogging is spreading its wings rapidly. Your write up is a good example of it.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="">