Some of you noticed Wikidot was down yestarday for more than two hour. This is not something that happens often — previous serious outage we had was in November 2014.
The root cause of this outage was Amazon S3 failure. Wikidot, as well as millions of other websites all over the internet, rely on S3 for hosting files. S3 is exactly where we keep the files you upload in your wikis, but it's also where we keep JavaScript and CSS files required for displaying our web pages in your browsers.
Till now S3 has never failed before to this extent. It sure was not a small failure and it took down several other Amazon services. It also affected all services and websites that use Amazon Web Services. It's not just Wikidot alone: Trello, Travis CI, GitHub and GitLab, Quora, Medium, Signal, Slack, Imgur, Twitch.tv, Razer, Apple's iCloud and several other websites could not function properly (or were not reachable at all). A significant percentage of websites all over the world relies on S3 and only now we learned what happens when it's down.
The issue was so severe that even Amazon could not update their status board to let us know about the problems. It was probably hosted on S3 as well… It looks like engineers simply assumed that S3 would be available no matter what.
Wikidot infrastructure design relies on a certain assumption about S3 as well. We simply assumed everything can break, but not S3 itself. Even our backup site (in case databases and servers fail) is hosted on S3.
I guess today several admins and developers (especially from services affected by the S3 outage) try to find a way to loosen their dependencies on S3 and protect their services against similar events. We are going to look at this too — the data you keep with us is our top priority.
Thanks for your understanding and I am sorry for any trouble our outage might have caused.
Michal and the Wikidot Team
Thank you for the prompt and detailed response here on the blog, and the regular updates on Twitter.
Communication is key and I believe you handled the problem quite well despite being unable to physically do much to rectify it apart from waiting on Amazon.
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
something is wrong
This is awesome thanks for respond and check out this one
Hi Michal,
thanks for Information!
What a gift from Amazon for this time:
Congratulation to your birthday!
Regards
Helmut
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?
Thanks for the detailed info, and job well done. Hopefully these sort of events raise awareness and reduce the likelihood of recurrence!
Perdón por molestar, únicamente me cree una cuenta aquí para compartir mis opiniones con los demás pero antes quiero informarme bien sobre algunos de los "scp" que tenéis expuestos, pero en algunos me pide autorización de nivel 4 y quería saber como puedo contactar con ese personal o con personal superior para preguntar mis dudas y ellos me respondan con sinceridad. Respetare absolutamente todo lo que me digan y no les reprochare nada esté o no de acuerdo. Si me pudieras ayudar me harías un gran favor. Un saludo y muchas gracias.
Un poco de ayuda: http://handbook.wikidot.com/es:start
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?
I keep on Amazon s3 only backups. I 've never had any problems. For such purposes is ok, but for important things, maybe not necessarily.
A distance of 3000 kilometers, Dalian stem Shing homes through a network of real estate of Mr. Cai found us Sisson . While browsing the official website, Mr. Cai seemed to be at the Xisen site, with real pictures of the workshop, equipment, and products appearing in front of him. In the end, Mr. Cai chose the Sisen DN100 steam flow meter without any suspense .
Why choose Xisen steam flow meter ?
Xisen is the only full-caliber, full-scale manufacturer in South China that has passed the "License for Manufacturing Measuring Instruments", equipped with top-notch verification equipment . Every steam flow meter purchased in Xisen must be verified before leaving the factory, and only qualified products can be delivered to customers. The quality is excellent, you can buy with confidence.
Sisen Declaration:
You only smell my perfume, but you don't see my sweat. You have your rules, I have my choice. You deny my present, I decide my future. You laugh at me for nothing, I pity you always waiting. You can despise our youth, and we will prove whose age this is. Dreams are destined to travel alone, and doubts and ridicules are indispensable on the road , but so what, even if the body is wounded, you must live beautifully. —— I [ Sisson . China ], I speak for myself!
Is that right !
Hey Wikidot, it feels like you have become disconnected from your users. Many pages appear not updated or abandoned. No one has any idea if you are working on new features, or have disappeared unless something brings down the farm. I also hear that spam is an issue.
Will any of this be addressed? Can you let us know what is going on, what is being planned, and what the future will be? The feedback page has items 'in progress' from 2010. Thats SEVEN years ago.
yeah we feels that too¡¡¡ :/
We are still waiting on answers and waiting on information. Why have you not added a layer of security to joining that will stop most of the bots? Why don't you update? Why do you ignore us? Half of your new users every time I refresh the whats hot page are spam users making spam wikis. How long has this been going on for? Are the majority of the wikis hosted on wikidot just spam? How much space are you wasting hosting these sites instead of communicating with us? No business would find this acceptable. Have you given up on Wikidot?
SEVEN YEARS for the wishlist updates? Do you even have staff anymore? Is that the issue here?
Omnia mutantur, nihil interit.
Hi Naomi,
From the bot-fighting point of view I can add some input, as a community administrator.
Wikidot has over the years tried multiple different techniques to block bot users, and thousands are getting automatically blocked and/or deleted by the system when they match certain activity patterns.
Further, thousands more are being deleted manually be community administrators, which trains the automatic tool further and catches the less standard bots (of which there are plenty).
It's hard to explain just how massive the attacks have been in recent times, but you probably have somewhat of an idea from your email notifications if you're watching some of the main sites. About 1-2 weeks ago there was a significant one on the community site.
Everything else: I cannot comment on this, that's for Wikidot to answer.
~ Leiger - Wikidot Community Admin - Volunteer
Wikidot: Official Documentation | Wikidot Discord server | NEW: Wikiroo, backup tool (in development)
Goodness I was salty. Well, I still stand by it for the most part.
I have no gripes with the community admins and mods because you have all been doing the best you can and I can only shower you in kudos for it. I do still think a pre-account creation action would be effective rather than trying to do damage control after these accounts are made. But I imagine that is out of everyone's hands here because its not dealing with users but with account/signup nuts and bolts.
I am deeply worried over Wikidot's silence. I have been waiting on purchasing a pro account for some months now in fear that the plug will suddenly be pulled or some sort of incident.
Hmm.
Omnia mutantur, nihil interit.
I couldn't agree more, Naomi.
I have been spreading some salt of my own over the last few years - sometimes a subtle jab and other times trying to rub it into an open wound, but it continues to have no effect. I have given up on any new features being added or bugs being fixed.
It's extremely disappointing and makes zero sense to me from a business point of view. I don't know what Wikidot is worth, but I bet it would be worth a whole lot more if they invested in some development resources and started engaging the community. I'd love to see Wikidot end up in the hands of a group who cares and doesn't ignore their user base.
I'll continue to hang out here and help squash whatever spam I can and offer help once in a while, but my enthusiasm is long gone.
Community Admin
Whats so buck wild to me is that there is NO way to contact anyone that has proper control of the site, at least I didn't come up with anything in my digging. I went so far as to try and use the 'contact us' feature on projects the Company owner has been working on (no reply of course).
On the one hand, I wonder if the pushing is causing more silence. On the other, what is the fear here? Is it the hope that Wikidot fades out after having been the poster child held up as credentials to speak at seminars and conferences? Is Wikidot no longer meant to succeed/is being sabotaged via inaction in hopes it goes away?
If they want it to go away, give it to the users who care about it, because it IS a great system. A great system that hosts mostly bot content these days.
Omnia mutantur, nihil interit.
In the first time Wikidot had great problems to communicate with their users - after they have installed a dedicated one for communications alone it was a better time and a massive go forwards with implementations an innovations. Sqwark has left wikidot and it is siloence now since this moment.
Peter - he was the CEO for some years and pushed a lot forward in internal technic - is dead.
I do not know the philosophy behind the workload of Wikidot now , the team, the organisation - nobody is answering now on private messages or on mails.
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?
This is ridiculous. Half of the comments you get here are bots. Aren't you embarrassed?
Omnia mutantur, nihil interit.
Yes wikihow does not remains down everyday but if it remains down then it makes us boring.
I was completely unaware of this outage, thank you for sharing this information. To be honest, I am done with the scams from bots.
That is a ton of bots around here, huh? Well, good luck on getting rid of them because it's nearly impossible. Nearly impossible, not impossible.
If you get lots of spam (bots or human) you can always message these users/accounts on http://spambotdeathwall.wikidot.com/
Service is my success. My webtips:www.blender.org (Open source), Wikidot-Handbook.
Sie können fragen und mitwirken in der deutschsprachigen » User-Gemeinschaft für WikidotNutzer oder
im deutschen » Wikidot Handbuch ?