More planned upgrades, yes. Moving to static HTML caching for anonymous users, and more front end servers to handle the page rendering load. We do these slowly and carefully so don't hold your breath :-)
More planned upgrades, yes. Moving to static HTML caching for anonymous users, and more front end servers to handle the page rendering load. We do these slowly and carefully so don't hold your breath :-)
There is caching now, but it still requires some application overhead. That is, fetching a cached page means hitting the application server edge. With static HTML caching, the application servers do not even see the request, which gets served by a machine running just a fast web server and serving HTML files. It's 10-100x faster and eliminates 60% or more of the load on the app servers.
We tried this when we had the Snow Leopard incident in August and have been slowly making it work. It would be for anonymous users only; the HTML pages then get updated in the background as the real pages change.
I've noticed a big improvement. Amazing job!
Just wondering how it's possible to change servers without downtime.
I'm assuming:
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
I think you've probably been misled slightly by the term server. I'm sure Pieter will correct me if I've got this a bit wrong, but I don't think the processors were upgraded with this exercise, only the storage devices. In which case, the internet presence of wikidot.com would not have been affected, so no IP address issues resulting here.
As I recall, RAID disk arrays have built-in data replication so that any disk can be hot-swapped in the event of a failure without losing access to the data that was on the disk in question. Also, the DBMS is most likely mirroring the whole database continually to a separate array anyway. So if one mirror is unavailable for some reason then the DBMS can still operate on the other one. Having features such as these built into the design of your architecture then enables you to make such hardware changes seamlessly, on the fly. It's pretty clever stuff and it all happens, like you say, as if by magic.
Sue
So as Michal explained: this is a back-end issue, it does not affect the front-end servers (which serve the Wikidot.com domain), and although there are IP address changes, they are internal to the cluster.
The real magic is to create multiple parallel databases and switch in real time from old to new, without losing replication, or current connections. Michal says it was "simple" but that is deceptive. This is the kind of operation that takes real skill and experience. Then, of course, it's simple.
One more reason why we're all happy that Wikidot.com exists, it takes over all the stress of managing our data. For me as a user, when I see this kind of smooth upgrade, I'm more confident than ever that everything else (such as replication) is also done properly and that in the event of a disaster (meteor hitting the data center, or whatever), I won't lose my sites.
In fact we moved the whole database (i.e. database servers) to new boxes, and for transition to be successful l we had to replicate the databases live, and once they were synced, we simply pointed application to the new servers. The front-end (PHP application) servers remained as they were.
The thing is when you deals with tons of gigs of data, and the service (Wikidot.com) needs to be available without any outages, simple copying of data is not the way. In fact it would take us down for about 5 hours. Live syncing and instantaneous switchover removed this problem.
We will now dispose the old database servers - they were good a year ago, but not sufficient any more as Wikidot grows, and new (cool) hardware is available at good prices.
Michał Frąckowiak @ Wikidot Inc.
Visit my blog at michalf.me
Maybe this info should be inserted in a Wikidot article at Wikipedia, with this thread as reference… I would do that, but guys, I don't understand a single word Pieter wrote here, except that Wikidot is now faster, so maybe it would be better that someone else writes it….
If slaughterhouses had glass walls, everyone would be vegan. - Paul McCartney