> I'd be happy to help if you're having database problems - feel free to email me.
> Can we have a real technical explanation of what the problem was?
Wish I could give one, but I'm not really sure.. The load was very high (low 20s, compared with ~0.2 usually), the database server was taking up a lot of CPU, and there were lots of sleeping MySQL sessions.
The queries which were timing out were the forum queries, although nothing has changed for them recently.
I put the server into maintenance mode, keeping the MySQL DB from running anything complex, and it recovered very quickly. I then checked the vitals and it turned out the query cache was disabled, I thought it'd be a good idea to enable it since forum queries are the same for all users. Analyzed the forum queries too, comparing it to the dev server; the SQL EXPLAIN analyzer didn't point to any obvious difference between the query running here and the dev server.
Also turned on increased logging for the likely event that the problem came back the moment I opened the server back up. (Also checked the server logs for telltale signs of a DoS attack, but nothing obvious showed up)
Opened it all back up and no issues since. Not very encouraging really, but as I said before I don't have the ability to troubleshoot if the problem isn't repeatable.
Any MySQL gurus are welcome to give their 2c.
)
Justin Beiber gay or straight
)
)
)
)
)