New Searching Engine
by pieterh
on 02 Jul 2009 09:17
Searching is very important, as Google proved. Wikidot's search has sometimes been… well, imperfect. So I'm happy to tell you about the new version, which has been rolled out and is working nicely though not yet perfectly.
First, we have the local per-site search on each of your Wikidot projects. Second, we have a "global search" at http://www.wikidot.com/search:all that will search the whole of Wikidot, including your private projects, if you are logged in.
The new search engine, based on the open source Lucene engine, is faster and more complete than the old version. It still has some teething issues, such as not properly searching Cyrillic or Chinese. Let us know if you find any other problems with it.
Comments: 7
page revision: 3, last edited: 07 Jul 2009 19:16
It's not working properly yet (and my sites have no Cyrillic or Chinese characters).
Eduardo R. Ribeiro
Macro-Jê linguistics, &c.
An update on this… we fixed a problem with the way the indexer was handling some Unicode character sets, and the reindexing is running on affected sites. That's taking 2-3 days. Indexing the whole of Wikidot was a 2-month job, a low-priority background process. Since that is now done, we only need to reindex changes (and for errors like the Unicode one), which is a lot faster.
@Kawina, we're going to check why your search does not work properly. Can you (a) explain what you get when you try a site search and (b) try the global search at http://www.wikidot.com/search:all, and (c) tell us what languages your sites are in? I'd check myself but your profile is blocked.
Portfolio
Thanks, Pieter, for looking into that. As for your questions:
(a) if my search terms contain any accents or other diacritics (cedilla ç, for instance), no results are found (even though there should be hundreds of them); even for search terms without any special symbols, the search results are incomplete (try, for instance, http://www.etnolinguistica.org/search:site/q/aryon; that used to yield almost a hundred results, as this is one of the main authors in our field, but now the search comes up with only a few results, including — oddly enough — pages in which the term does NOT occur); that problem occurs with all my sites.
(b) the global search seems to produce the expected results, except for one of my sites (http://tr.im/brasil);
(c) my sites' main language is Portuguese, but there'll be info in other languages as well (all of which use the Latin alphabet, sometimes with a few especial symbols not found in Portuguese, such as ẽ and ĩ).
The main problem is that search is currently unreliable, producing partial or null results. We've spent a lot of energy building a content-rich site (which is currently the main source of information in our field); it would be a pitty if our visitors ended up with the wrong impression).
Maybe it would be better, until this is resolved, if search was deactivated with an explanation to the users about the current status; is that possible?
By the way, sorry that my profile is blocked; I was unaware of that, and I'll make sure it's accessible now.
Thanks again,
Eduardo
Eduardo R. Ribeiro
Macro-Jê linguistics, &c.
Eduardo,
Thanks for the details, this is helpful. I don't think we want to turn off searching on all sites. You may want to add a notice for your users, until this is fixed.
I'm not 100% certain that problems with accents and diacritics will be fixed now but it does fit. We'll need to retest when the reindexing is finished, and I need to check with the devs.
The only reason I wanted to check your profile was to see which sites were affected and whether it was a language issue (which you confirm it is).
Cheers
-Pieter
Portfolio
Thanks, Pieter. I wasn't suggesting turning off search for all of Wikidot, just for my sites. Since that doesn't seem possible, I will post an advisory to the sites' visitors about the search problems. I hope that can be fixed soon.
Best,
Eduardo
Eduardo R. Ribeiro
Macro-Jê linguistics, &c.
Dear support,
when I create a new page with Chinese title using the side nav and top nav bar, it can only link to the same page of Chinese. Probably due to a search problem. There is no problem for other linked pages create within the pages. Wikidot is by far I found (I look other big sites also )the best wiki for me and China is a great potential market. It will be great to make Chinese wiki with wikidot.
check this site of mine created at wikidot: [http://vedas.wikidot.com/], view the Chinese top bar and side bar link and also the link within the page. How can I correct if I want the links for top and side navigation to show different Chinese pages. There are no problems if I set the links to be in English.
Regards,
Yong
You can write the links in pinyin or in English, but show chinese characters if you want?
For example, instead of writing this:
You should try writing this (this is an external link being used for an internal page):
(notice that there is only 1 square bracket on the ends now, not 3)
The page will be called introduction, but the button in the top or side navigation will say 簡介!! =)
Also, another suggestion: It looks like you have your menu set up so that the buttons are aligned vertically? If this is not what you want, and you want them to be aligned horizontally, you should try this…
Instead of writing this:
Write this (no extra lines):
Shane Smith ★ Website: Wikidot Development & Programming. ★ STE Wikidot Editor: Windows. Mac. Linux. ★ Twitter. Google+.