Hope the subject line didn’t put you off, couldn’t think of anything suitably pithy to summarise what I wanted to write about.

Lets get the background out of the way. We have the Internet crammed with great information repositories (and even more rubbish ones), we have high-speed, high availability broadband/wi-fi, we have convergent mobile devices, we have advanced search capabilities, we have more access to more immediate information and knowledge than we ever thought we would have 10 years ago.

But still it all leaves me a little frustrated because getting what you need, finding out what you need to know still requires an amount of skill or luck.

Searching, the old-school way

By way of example, I did some work recently relating to a patent application (not my own I hasten to add). I was doing research, looking at the market, for opportunities, prior art, that kind of thing. I found quite a lot and hopefully it proved useful to those receiving the information (I think it did ). But the basic problem is that I was able to find things that they (and it seems the patent office) couldn’t.

So, here is my contention, this wonderful information age we live in will never actually be real until such time as the skill involved in finding things out is obsolete.

A few years ago, I wrote an article for a training magazine that discussed the nature of information in this way and used, as its popularist hook, the idea of using technology to cheat at Who Wants to Be a Millionaire. If you remember, the people who tried to cheat used coughing and all manner of stupidity to win £1 Million pounds. I discovered that if you typed the text of the question unedited into Google, for almost all, the answer was somewhere in the summary text in the top hits. This was pretty impressive. If you could harness speech recognition, search technology, a screen reader and some bluetooth you weren’t far from cheating your way to the £1M. But as a day to day solution it was, at best, flakey.

This lead me to think of two things, semantics and the nature of information itself.

I have no doubt, at some point in the future, that the nature and value of information will change. It will be intrinsically simple to find out any fact, that the value in remembering anything will diminish greatly. This has massive implications for education. What we will have to teach children and assess them on will have to change entirely. Remembering that the Battle of Hastings was in 1066 will have no more importance than the time of the next bus (although I’m sure many would argue that that is as its should and may already be).

Even now, with my laptop permanently on at my side, there is no nagging question I can’t get the answer to, and when devices are up to it, that will be true 24X7 wherever I am (although I do OK now with searching on my PDA). And as the way of living becomes prevalent and USABLE to everyone, the nature of knowledge will surely change.

(BTW, as an aside, a really good addition to the armchair knowledge hunter’s armoury is the Wikipedia gadget for the Google desktop sidebar, it works really well and none of that messy web page browsing required.)

The one remaining hurdle to be accomplished in all of this is semantics. Information retrieval by text based searching, however powerful, is limited. And crucially, it requires the skills of the user to select the correct search terms. The next generation must surely be based on a semantic engine but although the ideas of the Semantic Web have been around for a while now, it has still to make its way out into the world and there is the Metacrap lobby that doubt if it ever will.

The problem with any Internet search is that you go looking for a definitive answer and you often end up with a ‘well, it depends which site/person you believe’ feeling. One thing that the addition of semantics will provide is to be able to measure the weight of an opinion i.e. the relative numbers of answer a) versus those saying answer b). I was given an example of this very problem today with the question “Who was the 5th Beatle?”, it seems to depend on who you ask. But if you could measure the numbers of each answer? If I can return to the beginning, this would be akin to an Internet version of ‘Ask the audience?’ in Who Wants to Be A Millionaire. Currently, you have to do that measuring yourself, no one does, its tedious. Understanding the context and content of a set of semantic search results would make this possible.

I firmly believe a semantic knowledge representation on the web will happen, but perhaps not in the form currently proposed, and if there is anyone out there looking enviously at Larry and Sergey they should perhaps start thinking about a semantic view of the web and knowledge storage and retrieval as the next thing to try and take over the world.

The benefits would be huge. A single device. You asked it a question, verbally. It accesses the web with a semantic search and talks back the most likely answer, anytime, anywhere.

Obviously, the steps to achieving this are non-trivial but the raw material is there, we already have a gloriously populated Internet. But we have to solve two key problems, how to create a usable semantic index from that which already exists and, crucially, to know you can trust what you find.

The element of trust is one that is also still to be addressed. There is an implicit trust based on the users on interpretation of the site they are on. We read the BBC we believe it, we read Wikipedia, we believe it (even though it is user written) but if we are on ‘Big Al’s List of Stuff’, we may seek corroboration. But there is no formal/technical mechanism for trust, maybe there doesn’t need to be, but if we are to use this new form of knowledge retrieval as an agent for change in education, then it might be a good idea to know we are being told the correct information. The simple fact is that I could easily build a ‘Capitals of the World’ website that says the Capital of Peru is LiddellTown and, with good SEO, I could make it the de facto answer for people searching all over the world.

Which might be a good laugh, if nothing else.

So, there we have it, it seems that progress only creates a deeper thirst for more progress. I imagine it has always been so. Next week, I’ll be banging on about the need to develop and market a replicator and a holodeck.