Searching: an Update

by pieterh on 05 Jul 2009 09:34

Sunday morning, and we (myself and my team) owe the Wikidot community an apology. On 24 June we migrated to a new search engine, and on 25 June Brunhilda reported that "My search box doesn't give any results, whatever you put inside…"

We found the problem rapidly but were faced with a reindexing process that took weeks to run on the whole of Wikidot. We started reindexing sites that were affected but this was still going to take many days. So now we've rewritten the reindexer. Before, it indexed each site in turn, which meant going to the database over and over, to update each piece of the index. Now it indexes the whole of Wikidot in memory, and only writes out the index at the end. This morning we tested it again and it runs in 2.5 hours. That's without indexing tags, but we'll add those back later when the rest is working and stable.

A week without search functionality is far too long, and I'm really sorry for the stress this has caused to affected site admins. Congrats to gerdami and RobElliott for proposing workarounds and to everyone who helped make it less painful.

Comments: 11

Add a New Comment