Database problems earlier today

by Gabrys on 05 Aug 2010 09:25

We would like to apologize for problems with Wikidot sites that all our users experienced earlier today. The problem persisted from about 06:34 to 07:24 UTC. It is now solved. There was no data loss.

Problem started at 06:34 when database engine (PostgreSQL) tried to acquire another segment of shared memory (SHM) from Linux kernel1. Kernel was not able to allocate it because it reached the limit set for this kind of memory. This resulted in database stopping serving data and other processes not being able to request SHM. This caused the server not reachable in usual ways (like SSH shells), so we had to reboot it remotely.

A few moments later (when the machine booted up) we discovered what the problem was, raised the limits for SHM memory, started database and checked its integrity. When we verified no damage was done to users data at 07:24 we enabled the access to the database and Wikidot started to work again.

Comments: 8

Add a New Comment