Death by Popularity - The Web Hosting Conundrum

There is a bit of a conundrum with the Internet. The more popular a site is, the more likely it is to fall over. This is especially true when you get a sudden spike of popularity.
I can remember on September 11th, hitting the news sites to be met with a spectacular array of error numbers but mainly telling me "server too busy", at the very point the world really needed the Internet, it killed it in the process. The only way I could find out anything on the Internet was too seek out a less popular news site and try there. I eventually got some news from the then fairly new, Ananova. Truth was, the only real way to get any news that day was to phone someone who could see the infinite power and bandwidth of a TV.
( This sort of thing happened again recently after the sad death of Steve Irwin. )
Things haven't changed much over the intervening years. A number of times I have seen the BBC website choose to feature a new site or service, only to temporarily kill it in doing so. You can imagine the chat in the offices of the site:
"Hurrah, we're featured on the BBC website..."
"Only, no one can see what our site does...coz its now dead."

The basic rule is, if a high performance/high hit rate site features a site of less power and bandwidth, there is a good chance it will kill it with the flood of hits. The site will only gain the visitors that remember to go back and try later, which is far less than they would hope for from such exposure.

I saw this again this week, when my favourite photo site MorgueFile suddenly got a massive peak in traffic because of an article on Digg (well done to Michael and the gang for keeping it just about alive).

This is clearly something that the next generation of hosting has to address. There is no point in having a service that is almost guaranteed to fail at the point of maximum demand. Neither does it make sense, economic or otherwise, for sites to plan for the peaks and have a server farm sitting and waiting on the off chance. The ability to turn up the power and bandwidth virtually instantaneously (on-demand or automatically) will be a killer feature of hosting. This has already started to emerge, particularly in a lot of the hosting that BT is starting to offer (although with an ironic quirk of fate, when I went looking for a link to see what hosting they are providing now, their site was down!).

If you are unable to read post its most likely because it has just become massively popular :-)

When new technology gets old

Technology moves very fast these days (Cybil Fawlty, specialised subject 'the bleeding obvious').
Not so many years ago, well, 26 years actually, I had a ZX81 and it was the pinnacle of home computing. And, believe me, it was fab. Thing is, it has no particular legacy now, only a museum like curiosity that was once kept for a difference engine or Colossus.
The technologies that I want to take about are those that leave a more permanent echo. Two candidates are mobile phones and laser eye surgery. No one knows what 20, 30 years of mobile phone use can do to you. Similarly with 20 year old laser'd eyes. "It'll be fine" they say. Supposedly the same people that used to sell asbestos to builders. I'm not a scaremonger, I love technology, advocate it and use it widely (although you can stick your laser eye surgery where the sun don't shine). I'm merely using these examples to illustrate the fact that we tend to get so caught up in the new and the now that we rarely look at the long term impact.
One of the potentially most interesting forms of this is the internet itself. We are creating content now that could still be around in 20, 30, 50, 100 years time. My grandchildren might read this very blog post and think "what an old twat" and click on the link to a now neurally implanted version of Wikipedia to find out just exactly how little power a ZX81 had. When my daughter is older, her teenage friends might track me down on the Internet and tease her at school because "her Dad is such a dull square and there's no way to you were on a billboard in Las Vegas".
The thing is, new technology today isn't all disposable future museum pieces, it has the ability (and perhaps role) to persist, to form part of a personal, cultural historical record.
We can already go to the WayBack Machine to see how gloriously crap the original BBC website was. Who knows where this post, along with the rest of the internet content, will end up in the future. Will there only ever be one transitory internet, with sites living and dying with us? Or will we devise a mechanism to maintain an on-going record (the WayBack Machine being a laudable but limited early attempt) so that this will post and many millions more like it will remain beyond my/our lifetime as a record of this time?

Website of the day:
Well, how apropos (was it deliberate?), have a look at this. People are already seeing that the passage of time has a place in applications of today.

Track of the day:
I was taken with Panther Dash by The Go Team! tonight. You can never have too many harmonicas.
( all together now grandkids of the future, "Granddad that is soooo 2004...")

A billboard!

Been a bit lax posting news of photo usage lately. I was inspired to post some more by news I just received of one of my photos being used on a billboard in Las Vegas. Pleased with this one.



many thanks to Julie Hurd for the photo


Here are a few more while I'm on the subject:

• A painting competition
• A few blog uses, 1, 2, 3
  
• One of my photos painted
• In a lovely deviation
• One of my photos is in here somewhere
  

Also, one of my photos is being used by Craig Zobel to promote his film Great World of Sound and the Sundance Film Festival, which is pretty cool. Best of luck to Craig at the festival.

Snap!

My reader(sic) may notice that I have added Snap to my site and blog (if you have no idea what this is, hover over an external link and you'll see). I suppose there are many arguments with regard to its usefulness but for now the novelty of it is tipping towards the 'quite cool' end of the scale.
I have to say it is a really good implementation, a doddle to integrate and it seems to render the page previews very quickly. Obviously, with my penchant for the funky client-side implementation, I'm always going to be a fan.
Does make me think that it could change the surfing dynamic a little if it takes off, people could perhaps see a lot less spurious hits with low wait time.
I first saw it being used on JP Rangaswami's excellent blog Confused of Calcutta which I would encourage you to read.

Frank Who?

My love of all things space is known to most who know me, so I suppose it is a little surprising that I have yet to comment on such matters herein. Time to put that right.
I am hugely interested in the US Manned missons on the 60's, Mercury through to Apollo not only as an engineering marvel but also for the human endeavour. Obviously, the moon landings themselves take most of the public interest, in my opinion, by far the most impressive of all missions was Apollo 8.
Let's put this into a little context. There had been a fire during a test, later to be called Apollo 1 which had set the plan back (and most say to good effect) a long way. There had been a few unmanned tests and one manned orbital mission Apollo 7 which was not even the full configuration. With things running late and the end of the decade target at risk, time needed to be made up. It was time for one of the best management decisions imaginable. George Low proposed that Apollo 8 go to the moon. Only the 2nd ever flight. The lunar module was late and not going to be ready on schedule so they could make up overall time in the plan this way.
I love this because it is exactly the right kind of management decision, a perfect balance of risk and opportunity. Looked at now with the Shuttle disasters behind us, it could look a little foolhardy but the mission modification itself was still a well thought out proposal. If you are ever seeking inspiration for how to take a technology program forward, look at Apollo 8 and you will wonder what your dilemma is all about. This is real decision making. There was only one true risk, that the engine wouldn't fire and get them out of lunar orbit. But this was a risk inherent in the Apollo design, so didn't have be any more or less risky with Apollo 8. The simplicity of the engine (virtually no moving parts at all) had made this possibility as risk free as it possibly could.
Frank Borman commanded Apollo 8. Its not a name that many will have heard of but he stands out for me as the number 1 astronaut of that time. Not only because of his flight in Gemini or Apollo but mainly for the work he did after the fire, in the investigation and beyond.
Apollo 8 was a resounding success with some notable firsts, especially the first earthrise picture. The Christmas day broadcast from the moon is also strangely chilling, even for me who isn't at all religious.

It wouldn't do it justice to talk about Apollo 8 here but I would encourage you to read the superb Wikipedia page and, of course, Andrew Chaikin's excellent book A Man On The Moon.

The story of Apollo 8 is not told enough and it is one that anyone in management should read and understand, it shows what can happen when you take your own personal fear of failure out of the equation and make choices based on good solid thinking.

Website of the Day:
Suppose it has to be this!

Track of the Day:
"Meet Za Monsta" by PJ Harvey was particularly splendid tonight.

The Silence of the Editor

Quiet round here. I'm editing the Beatle Man. Its going OK. In the re-read I made myself laugh. I take that as a good sign. I'm at Chapter 11 and I've come to one of the first and biggest excisions. Dropping a character and a small sub-plot. Its strangely painful but, I believe, as necessary as the original inclusion was pointless.
I should complete the first complete pass by the end of the week. Then its time to get writing again, I have at least 10 new scenes to write to make the revised plot make sense. I'm looking forward to that bit, following the narrative exposition mantra so closely did create some fun stuff to do. Then I'll be doing the flow work with the storyboard (which I'm very glad I took the time to do).
I seem have to become grammar-blind though (not that I was ever 20-20). During the editing process I am strangely compelled to add commas, dunno why, it just seems to be the done thing. I need to resist because in quite a few situations I've been punctuating out my arse, time to get the Strunk and White out again.

Website of the Day:
This is lovely. Whatever it is.

Track of the Day:
Aaaah, Clair de Lune just came on (sounds great on the new Sennheisers). Its such a great piece of music. And since I'm in the mode of Pacino film references...

The uselessness of modern man

Hey wow, look at me, typing into this computer, blogging if you don't know. I've got qualifications you know, I know so much stuff, I am the modern man, OK, not in that metrosexual way, but I am a man of technology, I know stuff, I can do stuff, I am going for the record for the most link to Wikipedia from any blog ever. Beat that!
And yet, I am intrinsically useless.
Have you ever wandered what use to society you would be if we were bombed/heated/plagued/bored/asteroided(?) back to the stone age?
So far, the list I have compiled is pretty small, I have:

• Eat me as food
• Use me as ballast (for an, as yet, unknown purpose)
• Use me as fuel for a fire (although someone else will have to light it as I've never managed that skill before, my Dad had a lighter)
  

Of course, I'm going to claim this isn't my fault, the sequential nature of society being as it is, I can quite easily leach off the shoulders of giants. Why not? Everyone else has, even Isaac Newton ( although he did stick pins in his eyes and try to make gold so he's not necessarily to be trusted).



"Oi, Mildred, Barry says they've lost the ability to make bullets and those feathery dart thingies"


I would, however, like to be more useful to that future perilous society, but sadly, they ain't going to have much use for us 21st century technophiles. Its going to be a simple questionnaire (presumably scribbled on a rock using, well, another rock, a sharp one).

1. Can you catch stuff to eat? (y/n)
2. Can you cook stuff once you have caught it? (y/n)
3. Can you make anything useful? (y/n) (Note: putting up shelves doesn't count, we don't have walls)

Its not going to well is it? "Three n's? Throw yourself on that fire, there's a good chap."

Personally, I blame the education system. What use is an honours degree in Electical Engineering in a post-apocalyptic wilderness when you don't have the slightest clue about how to make it? You can see it now...

"Don't worry folks, degree education, just get me a magnet and some wire and we'll have some leccy before you can say Michael Faraday "

And you can see their poor, forlorn faces as the walk off into the distance to kill what remains of the wildlife.

"Twat".

Yes, I don't know how to make a magnet and short of saying "That rock is green, that might have copper in it", the chances of me getting any wire are a little remote.

Its not all doom and gloom. There is obviously much to celebrate in this new, stark, world, here's my top five:

1. Tracy Emin would be more useless than me as she isn't even worth eating, she'd just be killed for sport to relieve the boredom.
2. Ditto for Pete Docherty.
3. The Americans have already killed all the buffalo, boy will they regret that.
4. We have an opportunity to create a new world without politicans
5. Ray Mears would be the hottest man on the planet and would have a harem of over 2000 women. I think there is something pleasing about that. He put the effort in, he deserves it.

Note, neither Emin or Docherty are getting a link, pointless as they are.

This scenario does also provide one other wonderful irony, those people in the current 3rd world know how to look after themselves better than we do so there would be a swap, 3rd world would become 1st world and vice versa (no one knows where the 2nd world is, but they'll be fine, they know how to handle potatoes).

So, what to do? Should I buy a Swiss Army Knife? Go native for a while? No. I have a plan.

People love sarcasm and rubbing other people's noses in it, so I'm going to stockpile T-Shirts with slogans like "What use is your iPod now fat boy?" and swap them for food with people who know what they are doing. And therein lies the answer to it all, if you understand the nature of humans, you will always have a way out, always have an answer.

Maybe I not quite that useless after all...

( yes, I left that deliberately hanging in that way to make it easy for you to lambast me in any follow-up comments, as I said, I know what you lot are like)

Convergence, Information, Learning and Semantics

Hope the subject line didn't put you off, couldn't think of anything suitably pithy to summarise what I wanted to write about.

Lets get the background out of the way. We have the Internet crammed with great information repositories (and even more rubbish ones), we have high-speed, high availability broadband/wi-fi, we have convergent mobile devices, we have advanced search capabilities, we have more access to more immediate information and knowledge than we ever thought we would have 10 years ago.

But still it all leaves me a little frustrated because getting what you need, finding out what you need to know still requires an amount of skill or luck.



Searching, the old-school way

By way of example, I did some work recently relating to a patent application (not my own I hasten to add). I was doing research, looking at the market, for opportunities, prior art, that kind of thing. I found quite a lot and hopefully it proved useful to those receiving the information (I think it did ). But the basic problem is that I was able to find things that they (and it seems the patent office) couldn't.

So, here is my contention, this wonderful information age we live in will never actually be real until such time as the skill involved in finding things out is obsolete.

A few years ago, I wrote an article for a training magazine that discussed the nature of information in this way and used, as its popularist hook, the idea of using technology to cheat at Who Wants to Be a Millionaire. If you remember, the people who tried to cheat used coughing and all manner of stupidity to win £1 Million pounds. I discovered that if you typed the text of the question unedited into Google, for almost all, the answer was somewhere in the summary text in the top hits. This was pretty impressive. If you could harness speech recognition, search technology, a screen reader and some bluetooth you weren't far from cheating your way to the £1M. But as a day to day solution it was, at best, flakey.

This lead me to think of two things, semantics and the nature of information itself.

I have no doubt, at some point in the future, that the nature and value of information will change. It will be intrinsically simple to find out any fact, that the value in remembering anything will diminish greatly. This has massive implications for education. What we will have to teach children and assess them on will have to change entirely. Remembering that the Battle of Hastings was in 1066 will have no more importance than the time of the next bus (although I'm sure many would argue that that is as its should and may already be).

Even now, with my laptop permanently on at my side, there is no nagging question I can't get the answer to, and when devices are up to it, that will be true 24X7 wherever I am (although I do OK now with searching on my PDA). And as the way of living becomes prevalent and USABLE to everyone, the nature of knowledge will surely change.

(BTW, as an aside, a really good addition to the armchair knowledge hunter's armoury is the Wikipedia gadget for the Google desktop sidebar, it works really well and none of that messy web page browsing required.)

The one remaining hurdle to be accomplished in all of this is semantics. Information retrieval by text based searching, however powerful, is limited. And crucially, it requires the skills of the user to select the correct search terms. The next generation must surely be based on a semantic engine but although the ideas of the Semantic Web have been around for a while now, it has still to make its way out into the world and there is the Metacrap lobby that doubt if it ever will.

The problem with any Internet search is that you go looking for a definitive answer and you often end up with a 'well, it depends which site/person you believe' feeling. One thing that the addition of semantics will provide is to be able to measure the weight of an opinion i.e. the relative numbers of answer a) versus those saying answer b). I was given an example of this very problem today with the question "Who was the 5th Beatle?", it seems to depend on who you ask. But if you could measure the numbers of each answer? If I can return to the beginning, this would be akin to an Internet version of 'Ask the audience?' in Who Wants to Be A Millionaire. Currently, you have to do that measuring yourself, no one does, its tedious. Understanding the context and content of a set of semantic search results would make this possible.

I firmly believe a semantic knowledge representation on the web will happen, but perhaps not in the form currently proposed, and if there is anyone out there looking enviously at Larry and Sergey they should perhaps start thinking about a semantic view of the web and knowledge storage and retrieval as the next thing to try and take over the world.

The benefits would be huge. A single device. You asked it a question, verbally. It accesses the web with a semantic search and talks back the most likely answer, anytime, anywhere.

Obviously, the steps to achieving this are non-trivial but the raw material is there, we already have a gloriously populated Internet. But we have to solve two key problems, how to create a usable semantic index from that which already exists and, crucially, to know you can trust what you find.

The element of trust is one that is also still to be addressed. There is an implicit trust based on the users on interpretation of the site they are on. We read the BBC we believe it, we read Wikipedia, we believe it (even though it is user written) but if we are on 'Big Al's List of Stuff', we may seek corroboration. But there is no formal/technical mechanism for trust, maybe there doesn't need to be, but if we are to use this new form of knowledge retrieval as an agent for change in education, then it might be a good idea to know we are being told the correct information. The simple fact is that I could easily build a 'Capitals of the World' website that says the Capital of Peru is LiddellTown and, with good SEO, I could make it the de facto answer for people searching all over the world.

Which might be a good laugh, if nothing else.

So, there we have it, it seems that progress only creates a deeper thirst for more progress. I imagine it has always been so. Next week, I'll be banging on about the need to develop and market a replicator and a holodeck.

No, not the Brandy

There are those among my reader(sic) that believe my love of Albert Camus comes from a teenager's desire for pretentious chic or, at best, some sort of angsty, pseudo-philosophical hankering. ( And anyway, if I had wanted to be pretentious for the sake of it, I was reading Ionesco too in those days, and you only read that in French. Sadly, this is no longer an option.)
There has to be an element of that in that somewhere. Not least because I had read pretty much everything he had ever written by the age of 17 or so and there is no doubt I didn't understand much of the more philosophical stuff (The Rebel, Myth of Sisyphus etc). But it did all sound cool when you read it.
Perhaps I will have a go at such philosophy again one day but for now I am re-reading L'Etranger (The Outsider, or The Stranger if you prefer) and La Peste (The Plague). I'm currently half-way through The Outsider (my Penguin edition still in very good condition and bearing the sticker inside that testifies that I bought it after winning an Engineering prize at school, which is altogether a very distant memory).
I decided to re-read them as I plan to embark on writing Terra Exitus sometime this year and I wanted to read something very bare to see, florid as I am, if I could have a slight style adjustment.

Without doubt, Camus is the most stripped down prose you could read. This could be related to the translation but, get ready for the sigh Stuart, having read it in French I think its just like that. The text doesn't really flow, its quite choppy and what comes across most is the sheer simplicity of it. And therein lies the tale, there is no way I could even think about trying to write in anything like that style, it would drive me mad. So, I'm not going to try, don't get me wrong though, Terra Exitus will be far from cheery.

Still, it has been great to go back to over 20 years since I first read it. And the important thing is, that for all its simplicity and starkness, I still love it. I'm very much looking forward to La Peste now which, for all its chill outlook on the 'human condition' is by far the best book I have ever read. I would encourage everyone to read it.

Track of the Day:
Deadbeat Club by the B. 52's, for all their sillyness, this really is a moment of splendour.

Website of the Day:
Ball in a Cup Vid - I admire the dedication it took to make this and because of the 'Box Game' we used to play at University

Stumble - You Know You Want To

As someone would have said if he were still alive, the Internet is big. Very big. (he may have even said it while he was alive, he was like that). I would add that, if you're not careful, it can be pointlessly big. Lets face it, the way most of us use the Internet is a bit like the Monopoly view of London, round and round the same set of places, rarely drifting off the beaten path.
It might be something like:

• BBC/CNN/geographically significant source for news to see what's going on in the world
  
• The Onion/The Register/you know the type of thing for offbeat laughs
• Amazon/Play/your favourite retail therapy location to see what you fancy wasting money on
• IMDB/Wikipedia/info site to answers questions and then pretend you knew all along
  
• Your favourite forum(s) to share some wisdom with like-minded people
• Places you end up as a result of ad-hoc searches, bookmark and then forget about
• Sites sent to you for the 14th time by email that you eventually decide to click (generally yet another video of someone falling off/on/through something)
• Oh hang on, I've not been to the BBC site for a while
  
• You know what I mean, I'll not labour the point

Essentially, its a loop, round and round. I'd hazard a guess that we all work on a basic subset of a maximum of a dozen sites. On our Monopoly board, we get to see even less of London.

And this presents a problem. There is an awful lot of good stuff out there that we don't get to see (Obviously, there is an order of magnitude more rubbish stuff too).


This changed for me the day I discovered the StumbleUpon toolbar. (Note: this isn't a new thing, I'm not claiming a discovery, this is more on an homage type thing). But I felt inspired to write as today I was nursing my sick daughter, who had fallen asleep on my left arm, leaving me with only the ability to click on the mouse button on the laptop (typing was tricky). This is, just one of the many occasions, where Stumbling comes into its own.

The Internet remains, for the most part, a pull medium. You have to go look. There is no schedule, index or order of service. This is often good, it gives immense freedom with a 24/7, on-demand vibe. Trouble is, when you don't really know what to demand, you can easily get lost in the Internet Loop (which is only made worse by this multi-tab Firefox thing).

It reminds me of an old Jerry Sadowitz joke (probably the only clean one) about a guy who goes into a bookshop and asks for a book on making Persian rugs, he is told they don't have any so he replies "OK, what else do you have?".

In short, this is what StumbleUpon does. You can walk into the Internet and say "OK, what else do you have?". And, the joy of it is, its very simple.

Install the toolbar, tell it what you like, click 'Stumble' and away you go. Find a page you like, click "I like it", if its pants, click "Not for me". Over time, you'll get more and more of the kind of things you are after with the added benefit that everything you liked is stored on your page which can then act like a mobile favourites page.

Its hugely liberating, massively useful and is as near as you can currently get to making the Internet a push medium. You get to places that would otherwise have passed you by. No more will you look at your computer and think, "all that Internet out there and I've no idea where to go".

Just imagine what life would have been like if you hadn't seen something like this, a random Stumble during the writing of this post.

As an aside, and as previously trailed, Stumble is great for parents of young children. When the TV is monopolised by the toddler generation, StumbleUpon creates a 'channel' away from the TV that can stop the adult mind turning to mush after the 15th consecutive viewing of High School Musical.

Its already a big thing and I predict it'll get bigger. Get Stumbling, you won't look back.