Hi Wikidot folks,
we would like to inform you that next week,
Monday to Wednesday (3—5 September)
we will be performing a major re-configuration of our servers behind Wikidot.com. All our servers and services will be migrated to a different location and a different provider, namely to Amazon Web Services in Virginia, which is about 1,000 miles North-East from our current location, Dallas TX.
No worries. Although it is a huge operation we have been planning it ahead in much detail. We expect no interruption to Wikidot.com services and no sites hosted at Wikidot should be affected. We cannot rule out unforeseen issues with an operation so complex, but we will be there to fix them ASAP.
One thing will however change and it will require your attention: static IP addresses for custom domains. If any of your Wikidot sites use "A" records on your DNS servers, you will need to update these records. But do not worry — we will email admins of all affected sites with detailed instructions and we will provide a reasonable transition period.
In the long term this change will help us easier maintain our resources, and should leave us more time to do the cool stuff we all want at Wikidot. Not to mention we expect better reliability and performance from the new setup so that your sites would be served better.
For these interested: we are moving to a configuration with approximately 25-30 CPU cores, 125-130 GB memory and 1.5 TB of fast disk space on 16-20 provisioned servers, all managed with the awesome Chef software. Plus, there are 6 external services we heavy rely on. We will also utilize all sorts of goodies like automatic resource provisioning, multi-level redundancy, load-balancing, monitoring, auto-scaling and so on. My colleagues tell me it is a state-of-art and bulletproof configuration that will perfectly suit Wikidot, so let us see!
PS. All critical data is already being replicated live to the new location, so the process has started and is going very well. We will keep you informed about the progress.
Image by PhilipC. No, we are not moving any hardware either by train or any other means. We always rent servers, this way we stay flexible :-)
UPDATE (Tue, 8:59 UTC): We have just moved. Uploaded files were unavailable for a short while and we are still bringing some functionality back. Activities are still not up-to-date and site statistics (the Pro feature). But other than that, things are running pretty smoothly.
UPDATE (Tue, 10:21 UTC): We are still working on improving performance and making all translations work for your sites.
UPDATE (Tue, 12.00 UTC): It looks like all major problems has been resolved. The whole operation went smoothly. I hope most of you did not even notice the transition :-)
UPDATE (Tue, 15:36 UTC): The transition went pretty smoothly. The last remaining piece, the traffic statc for Pro accounts, will be brought on-line tomorrow. Sorry about it, but I highly appreciate your patience!
UPDATE (Fri, 14:29 UTC): It has been a touch week, but we have solved almost all issues. The transition resulted in a few unexpected problems, but thanks to you they have been all traced back and solved. Wikidot is considerably faster now and (as we believe) more stable. Thanks for your patience!
I wish you all good luck for this transer and no problems!
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?
Yaayy! I'm wondering how this new configuration compares with the old servers' specs. Can we expect much quicker page generation and loads?
Kenneth Tsang (@jxeeno)
In the sense of "computing power" this is more or less 1-to-1. But what we aim for is flexibility and ease of management.
When I was about to rent the first server for Wikidot I had to wait 2 weeks until it was provisioned. A year later, the next one came in 2 days. Right now with SoftLayer in Dallas we need to wait 4 hours. On top of this comes our work to configure it, which even now to add any new server we need at least 6-8 hours. Plus we pay up-front for the whole month and can cancel any server only by the end of the billing cycle.
With Amazon+Chef we can deploy new servers within one minute and get it automatically configure within the next minute. We pay only for the time we have it (rounded to full hours).
This is great if:
- We have large spikes in traffic: new web servers will be available within minutes, even without our intervention
- Even in 24h periods, we have less traffic at night, more traffic during the day. We can control number of servers and pay only for what we need instead of keeping all the servers all the time
- When a server breaks down - we will ditch it in the ground and start a new one. This is much easier and faster than filling a ticket "Please replace one of our RAM modules because we are getting errors" and waiting for hours.
- If we suddenly decide our search servers need to be upgraded — setting up a new server with better specs is just minutes away.
So what you can expect is that there will be resources on our side for whatever crazy you might want to do with your sites. If you have a million visitors coming suddenly from nowhere — we will have enough servers to handle them. If you trigger recompilation of million pages — we will spin workers that will do the job.
Also you might wonder why we need up to 20 servers. This is critical. Right now we have 5 powerful machines, loaded with multiple CPUs, tons of RAM, SAS and SSD drives. But each of them play multiple roles and run multiple services. So if there is a hardware problem or one of the services starves resources, several other services are impacted. With AWS we can easily separate roles so that one failure does not affect other components.
Moreover, we are distributing critical components into different data centers (Amazon calls them Availability Zones). So e.g. we keep one database server in zone "a", and another with the replicated data in zone "b". We keep half of the web servers in zone "c" and another half in "d". Even if any given zone is down, we loose only some of our servers and the rest should still work and serve Wikidot.com without interruption. We do it for all critical components, spanning servers over 4 different zones.
I cannot guarantee you will see a significant difference immediately, but I am sure in the long term you will. We still will need some time to tune-up the new setup, but we will start it with enough resources to cope with the stress Wikidot generates.
After the transition is complete we will probably publish a whitepaper about it.
Michał Frąckowiak @ Wikidot Inc.
Visit my blog at michalf.me
Okay, sounds good :)
Since you'll be using different Availability Zone, does that mean that not all instances will be located in Virginia?
Kenneth Tsang (@jxeeno)
Thanks for the detailed update Michal, hope the changeover goes smoothly for you guys.
- Ye Olde
Ye Olde - Creator and Chief Admin of www.music-industrapedia.com (Global Music Industry Directory & Encyclopedia) hosted on Wikidot.
Rob Elliott - Strathpeffer, Scotland - Wikidot first line support & community admin team.
Great! Wonderfull collection of modern backoffice.
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?
ROTFL :-)
Michał Frąckowiak @ Wikidot Inc.
Visit my blog at michalf.me
This sounds like a good move. Hope it goes well. I can see I'll be busy next week changing IP addresses :)
Rob Elliott - Strathpeffer, Scotland - Wikidot first line support & community admin team.
For us who are hopelessly lost in cyber space and never will be found: what EXACTLY do I have to do? :(
Will there be a new Wikidot IP address that I have to put in my GoDaddy control pannel? Do I have to change anything in my Wikidot sites admin panel?
Thanks! :)
PS: Rob, you really gave me a good laugh! We should have a "like" button, like in FB for such things… :P
If slaughterhouses had glass walls, everyone would be vegan. - Paul McCartney
Brunhilda, right now you do not need to do anything. If you need to touch any piece of cyber-stuff we will email you and make sure you do it right and on time. Right now I am just giving a sneak peek what to expect next week.
Michał Frąckowiak @ Wikidot Inc.
Visit my blog at michalf.me
If slaughterhouses had glass walls, everyone would be vegan. - Paul McCartney
Hooray! How exciting!
Hopefully this might mean faster loading of web pages *crosses fingers*
Yes that's what I'd hope for as well.
Rob Elliott - Strathpeffer, Scotland - Wikidot first line support & community admin team.
Is it just me, or are page loads significantly quicker now ~!?
Kenneth Tsang (@jxeeno)
It should load much quicker. I think it's not the placebo effect, but we also feel that Wikidot is fast like hell now :)
Haha, that's nice to know!
Kenneth Tsang (@jxeeno)
It's reeeeally fast now! ;-)
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
Sounds awesome, and looks like it was the logical next step for Wikidot.
Good luck with the move, and may it all go smoothly :)
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
at the moment there are answer times of 15 secknds and more - my feeling - the waiting on themes.wdfiles.com is the reason…
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?
Some pages are loading very quickly indeed but others seem to hang for a long time and, like helmuti reported, that does seem to be on the wdfiles server. But after the first time of hanging it seems to speed up so it could just be that the routing table is getting updated for each site the first time it's loaded then it's fine after that.
Rob Elliott - Strathpeffer, Scotland - Wikidot first line support & community admin team.
We had indeed slight performance issue due to a bug in software we use. It should work fine now. But if you have URLs that fail to load, please report them. We will take a look and see things are OK.
Michał Frąckowiak @ Wikidot Inc.
Visit my blog at michalf.me
I'm seeing the same thing periodically. Some pages hang on wdfiles for a very long time. Pages that were hanging earlier today are now popping right open (and that's from a different PC). I have found that stopping the page load and then refreshing seems to help in most cases.
Overall, things seem to be really fast now. Pages (with ListPages modules displaying form data) that would sometimes take over 20 seconds to load before the migration are now loading in less than 3 seconds!
Community Admin
I am seeing "Waiting for blog.wdfiles.com…" (same on all tenants).. I can't get a HTTP response (manually) from blog.wdfiles.com. The socket accepts, but does not respond at all to HTTP or garbage requests.
Do we have a DNS rollout issue - it resolves to 23.21.85.188 here.