Additional level of caching added to Wikidot

by Gabrys on 16 Jun 2010 09:54

Varnish is our new forcefield

After extensive testing of various caching techniques, we have just enabled additional level of pages caching for anonymous users. We are using Varnish to do the hard job, keeping Lighttpd serving Wikidot pages for logged-in users.

Pages are cached for 3 minutes, and subsequent requests of the same URL to our backend service (based on PHP FastCGI processes) are queued in order to improve overall stability and performance.

Caching is applied only is the following conditions are met:

  • it's a wiki page (not uploaded file, login screen, AJAX requests etc)
  • user does not have session cookie set (this basically means users that are not logged in)
  • page is accessed with http, not https

This all assures logged in users always get the current version of page, while anonymous traffic may get page outdated by about 3 minutes. Disabling caching of https traffic means there should be no problem in creating account and logging in.

If you encounter any problems with Wikidot (especially when browsing as anonymous user) and you think it might be due the additional caching of content, please let us know at moc.todikiw|troppus#moc.todikiw|troppus.

The significant improvement is that Wikidot can handle traffic spikes much better now. In the past we have seen many sites getting enormous traffic from social sites (like Twitter or Digg) over short periods of time. From now on such traffic spikes will no longer impair performance and availability of other Wikidot sites. Thanks to Varnish, our new shield :-D

Comments: 11

Add a New Comment