Death by Popularity – The Web Hosting Conundrum
There is a bit of a conundrum with the Internet. The more popular a site is, the more likely it is to fall over. This is especially true when you get a sudden spike of popularity.
I can remember on September 11th, hitting the news sites to be met with a spectacular array of error numbers but mainly telling me “server too busy”, at the very point the world really needed the Internet, it killed it in the process. The only way I could find out anything on the Internet was too seek out a less popular news site and try there. I eventually got some news from the then fairly new, Ananova. Truth was, the only real way to get any news that day was to phone someone who could see the infinite power and bandwidth of a TV.
( This sort of thing happened again recently after the sad death of Steve Irwin. )
Things haven’t changed much over the intervening years. A number of times I have seen the BBC website choose to feature a new site or service, only to temporarily kill it in doing so. You can imagine the chat in the offices of the site:
“Hurrah, we’re featured on the BBC website…”
“Only, no one can see what our site does…coz its now dead.”
The basic rule is, if a high performance/high hit rate site features a site of less power and bandwidth, there is a good chance it will kill it with the flood of hits. The site will only gain the visitors that remember to go back and try later, which is far less than they would hope for from such exposure.
I saw this again this week, when my favourite photo site MorgueFile suddenly got a massive peak in traffic because of an article on Digg (well done to Michael and the gang for keeping it just about alive).
This is clearly something that the next generation of hosting has to address. There is no point in having a service that is almost guaranteed to fail at the point of maximum demand. Neither does it make sense, economic or otherwise, for sites to plan for the peaks and have a server farm sitting and waiting on the off chance. The ability to turn up the power and bandwidth virtually instantaneously (on-demand or automatically) will be a killer feature of hosting. This has already started to emerge, particularly in a lot of the hosting that BT is starting to offer (although with an ironic quirk of fate, when I went looking for a link to see what hosting they are providing now, their site was down!).
If you are unable to read post its most likely because it has just become massively popular 🙂