Scripting in Sphinx config

Did you know that Sphinx config enables you to use shebang syntax? Here is an example of how useful it can be in some cases: imagine you have 3 tables with identical structure that you would like to index into 3 indexes one per table. You can just make a php script which would do this and use it as a Sphinx config instead of describing each source/index separately:

#!/usr/bin/php
<?php

$source = "source [TABLE]_src {
    type      = mysql
    sql_host    = localhost
    sql_user    = user
    sql_pass    = password
    sql_db      = db
    sql_port    = 3306

    sql_query = SELECT [TABLE].id, title, body FROM [TABLE] WHERE [TABLE].id >=\$start and [TABLE].ID <=\$end
    sql_query_range = SELECT MIN(id), MAX(id) FROM [TABLE]
    sql_range_step = 1000
}
";

$index = "index [TABLE]_idx {
    source      = [TABLE]_src
    path      = /path/to/indexes/idx_[TABLE]
}
";

$tables = array('Cats', 'Dogs', 'Mouses');

foreach ($tables as $table) {
	echo str_replace('[TABLE]', $table, $source)."\n";
        echo str_replace('[TABLE]', $table, $index)."\n";
}
?>

searchd {
    listen          = localhost:9306:mysql41
    pid_file        = /path/to/pid_file.pid
}

Now you can use this file just like a normal sphinx config:

snikolaev@a5530:~$ searchd -c sphinx.conf
Sphinx 1.11-id64-dev (r2500)
Copyright (c) 2001-2010, Andrew Aksyonoff
Copyright (c) 2008-2010, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file 'sphinx.conf'...

3 Comments

Yaroslav VorozhkoNovember 17th, 2010 at 10:25 am

This seems is quite usefull to manage configuration of all indexes through ten lines of script.

Sergey NikolaevNovember 17th, 2010 at 10:49 am

Absolutely. There maybe even more complex cases than I described in the post, something like dynamic indexes creation and usage, this can be used on some shared hosting or in some complex web application.

Sergey NikolaevNovember 17th, 2010 at 11:07 am

Related bug http://sphinxsearch.com/bugs/view.php?id=596 (newer versions of indexer don’t work with scripted configs)

Leave a comment

Your comment

Notify me of followup comments via e-mail. You can also subscribe without commenting.