This week we pushed out some database improvements. There have been one or two glitches, but response time has improved, with maximum loads now around 60%, and average loads around 45%.
First, the glitches: some sites appear to have problems with their forums. Specifically, orphaned posts that no longer have a thread. We think this happened over time, due to some bug in the thread deletion code. This is not a serious problem (and we will fix the affected sites and the bug), but the new database code was treating this as fatal, and refusing to show the affected sites. If your site was affected, I'm sorry.
Second, during this work, database replication got broken, and on Monday afternoon was causing many "500" errors. This is now working properly again and Wikidot.com's master database is happily replicated to three slaves, two of them outside our main data center.
We have made a nicer 500 error page that should make your users less stressed if the Wikidot cluster can't handle a request. But the good news is that there have been few or no 500 errors over the last days.
Total system loads are now maxing at 60%, during the very peak hours, and averaging 45%. Before the database optimization, peak loads were approaching 90%. At 100%, we get risk of excessive internal queues, and errors (the infamous Snowleopard Syndrome).
We're continuing the work on static HTML caching of sites for anonymous visitors, which should reduce maximum and average loads further (I expect by about half but that needs to be seen).
Have a great weekend!