Server Errors Update

by pieterh on 15 Aug 2009 07:38

Over the last days we've been seeing too-frequent "500 Internal Server Errors" and other random JavaScript failures. These now seem to be mostly resolved, though not entirely.

Technical explanation: it looks like ip_conntrack was watching some local (127.0.0.1 <-> 127.0.0.1) connections and it was dropping packets. We changed /etc/apf/fwd.rules and raised /proc/sys/net/ipv4/netfilter/ip_conntrack_max to 150000.

Yes, this is voodoo to me too. What we know is that:

  • These problems are due to increasing traffic. Popularity has its price.
  • It's not about the usual problems that hit web applications: database, or bandwidth. Our servers and networks are running nicely, with lots of spare capacity.
  • Things are better but not fully fixed - I saw an error this morning.

Please let us know whether you also get problems. We're treating this as top priority.

Comments: 9

Add a New Comment