I just had two back to back. 735, 736 Eastern

Our code is about as optimized as it can reasonably be: If you want to see fewer internal server errors here is the best dedicated hosting I've been recommended:
http://www.leaseweb.com/en/dedicated-servers/dual-core-servers

These are the options:
- Refresh a couple of times
- Make a habit of coming at less busy times (server load is based on the US timezone)
- Come up with some way of sustainably getting 50 euros/month for webdiplomacy.net
- Post each 500 error you have here and pray to baal for fewer errors

Have you looked at changing your registry of webdiplomacy.net to GoDaddy? They offer free web sites to people who register the domain through them. OF course, I haven't looked at the bandwidth restrictions, if any, or what kind of capabilities the server would have. That might not be a working solution if they throttle the bandwidth, charge more for over use, or just don't provide the hardware needed to handle the load.

kestas, what's in the apache error log for the 500s? What's in the log is much more informative than what is usually output to the user-agent.

500s can happen for a lot of reasons, the quality of the code usually doesn't have much to do with it (absent some glaring error).

Free hosting is usually worth what you pay for it. Although, when I look at godaddy's hosting that's about $5/month, it doesn't look too shabby.

How many hits do you get in a day/week/month on average?

grep, I don't believe webdip is using free hosting. It is just less expensive than the 50 euros/month that would be required for a better host.

The questions still stands as to how many hits and how much bandwidth does WebDip use. Maybe we can find a way to get us on a better set of servers, whether it be at GoDaddy or somewhere else.

I had a few very bad experiences with GoDaddy, and I moved to Dreamhost. Then, I realized that Servint are just better (although less nice) and moved there...

I've been with GoDaddy for years. Of course, I don't have any real domain hosting going on there, but I do use them for some experimental things on occasion.

Well, kestas, I personally pray to Baal (pbuh) 11 times a day, and by his divine grace I never get Error 500's.

So I highly recommend the last option. It's well worth the ~$5k I spend weekly on purchasing livestock.

I also happen to know via a divine vision that if we switch to GoDaddy Baal will either:

A) Strike us dead with lightning
B) Cause us to be smitten in by a fireball of spontaneous combustion
C) Some combination of the above

But hey it's a free country, so yeah.

"it's a free country"

Keep in mind kestas is Australian and lives in Australia. Does Australia still count as a free country with all the computer censoring and computer game restrictions and such? (I don't know the details, just heard a thing or two.)

by it's a free country i meant it to mean I.T. as in international territory.

if you're curious what i'm talking about, read the book of baal

http://bit.ly/f5Ifyz

You're not going to solve the internal server errors without dedicated hosting (or some kind of divine intervention).

Errors are caused because scripts take slightly too long to execute, so the processes which run them stack up until they exceed our memory quota and the processes get terminated.
The reason the scripts can take too long sometimes can occur for several reasons, but they are all out of our control on shared hosting.

To explain by analogy why this thread is frustrating imagine this was a start-up delivery business, still using a car to make deliveries. Business picks up and the car can't carry enough stuff, people notice this and suggest
- "I have a mazda, try getting one of those"
- "Maybe you need to try a blue car"
- "My car was really cheap, and has air conditioning!"
- "Let's make a list of every time the delivery is late"
when actually we now simply need a truck, and any car will have the same problem.

Australia a free country?

Hm. Let me think about that. Yeah, they don't have any oil. We'll let them stay free.

@kestas - I understand all of that. It was one of the mods suggested we see if we could identify a pattern, so I started the thread with that goal in mind. My apologies if it offended you in some way.

wait why didnt you address my suggestions about baal.

come on man can't a body get any respect in this club.














lol jk i love you kestas

@Alderian draugnar was suggesting godaddy's free hosting.

@kestas shared hosting isn't directly to blame, I would venture. Do you have any control over the apache configuration, such as for tweaking the size of the listen queue, whether keep-alive is on, and so on?

For static elements, are you using a far-future expires so that user-agents can cache whatever they can?

What grep asked. :-)

Seriously, I'm a .Net developer and IIS manager. Give me an IIS server and I can configure the Web Garden to take better advantage of the processor cores. But the apache stuff eludes my knowledge.

Yikes. To reload the home page involves 36 HTTP requests! That's where any problem lies.

This page describes a Firefox add-on that will help you measure what's going on with the site:

http://developer.yahoo.com/yslow/

Ignore the discussion of a CDN, although you could probably use Amazon S3 for serving static elements at an incredibly cheap rate!

http://aws.amazon.com/s3/

@grep; I wouldn't tie your flag to that mast :-P it has nothing to do with static files

Here is exactly how these errors occur, in detail:
- People are requesting pages, the server is running webDip scripts to respond to the page requests, everything is going smoothly.
- Another user of the same hardware, or many users in general, begin to use more server resources (typically database resources, i.e. intensive queries).
- The amount of time needed to respond to a page request increases, so it perhaps takes a second instead of 0.1 seconds, but it's still no big deal and things are going smoothly.
- Then the average amount of time to respond to a page request rises above a critical point; it rises above the average amount of time between page requests. (e.g. it takes 1.2 seconds to respond and a request is made every 1.1 seconds)
- Things still seem to be going smoothly, but requests are coming in faster than the webDip processes can respond to them.
- The total number of webDip response processes slowly adds up, as more requests are made than are responded to.
- Each webDip response process requires memory, and slowly the total amount of memory webDip is using increases.
- Eventually a point is reached at which our account has enough webDip response processes in action that our account is using more memory than it is allowed.
- The server detects this and shuts down as many webDip response processes as are required to bring our memory use back within our limits.
- Apache is informed that the response process has been terminated before giving a response, so it responds with a 500 internal server error code.

If only slightly more requests are coming than can be responded to 500 errors might come but very infrequently, if someone is really hogging the database you'll get nothing but 500 errors.

Note that while all this is going on you'll be seeing static 500 error pages full of images and things at lightening speed, because static requests use no memory and take no time at all (if they're not cached anyway).

So although I appreciate the suggestions I'm not speaking out of ignorance; there is no mystery about where the errors come from or why, or about what is needed to take care of them