Prior to 2006, I had sort of heard of the Semantic Web. To be honest, I didn‚Äôt know much ‚Äì it was just another buzzword. I‚Äôve been hearing about Microformats for years, and cool but useless initiatives like XFN. However to me it was simply just another web thing being thrown around.
Then in August 2006, I came across Adrian Holovaty‚Äôs article where he argues journalism needs to move from a story-centric world to a data-centric world. And that‚Äôs when it dawned on me: the Semantic web is some serious business.
I have since done a lot of reading, listening, and thinking. I don‚Äôt profess to be a Semantic Web expert ‚Äì but I know more than the average person as I have (painfully) put myself through videos and audios of academic types who confuse the crap out of me. I‚Äôve also read through a myriad of academic papers from the W3C, which are like the times when you read a novel and keep re-reading the same page and still can‚Äôt remember what you just read.
Hell ‚Äì I still don‚Äôt get things. But I get the vision, so that‚Äôs what I am going to share with you now. Hopefully, my understanding will benefit the clueless and the skeptical alike, because it‚Äôs a powerful vision which is entirely possible
1) The current web is great for humans; useless for machines
When you search for ambiguous terms, at best, search engines can algorithmically predict some sort of answer that partially answers your query. Sometimes not. But the complexity of language, is not something engineers can engineer to deal with. After all, without ambiguity of natural languages, the existence of poetry is impossible.
What did you think when you read that? As in: ‚ÄúI‚Äôve had it ‚Äì fine!‚Äù which is like another way of saying ok or agreeing with something. Perhaps you thought about that parking ticket I just got ‚Äì illegal parking gets you fined. Maybe you thought I am applauding myself by saying that was one fine piece of wordcraftship I just wrote, or said in another context, like a fine wine.
Language is ambiguous, and depending on the context with other words, we can determine what the meaning of the word is. Search start-up company Powerset, which is hoping to kill Google and rule the world, is employing exactly this technique to improve search: intelligent processing of words depending on context. So by me putting in ‚Äúit‚Äôs a fine‚Äù, it understands the context that it‚Äôs a parking ticket, because you wouldn‚Äôt say ‚Äúit‚Äôs a‚Äù in front of ‚Äòfine‚Äô when you use it to agree with something (the ‚Äòok‚Äô meaning above).
But let‚Äôs use another example: ‚ÄúHilton Paris‚Äù in Google ‚Äì the worlds most ‚Äòadvanced‚Äô search engine. Obviously, as a human reading that sentence, you understand because of the context of those words I would like to find information about the Hilton in Paris. Well maybe.
Let‚Äôs see what Google comes up with: Of the ten search results (as of when I wrote this blog posting), one was a news item on the celebrity; six were on the celebrity describing her in some shape or form, and three results were on the actual Hotel. Google, at 30/70 ‚Äì is a little unsure.
Why is Paris Hilton, that blonde haired thingy of a celebrity, coming up in the search results?
Technologies like Powerset apparently produce a better result because it understands the order of the words and context of the search query. But the problem with these searches, isn‚Äôt the interpretation of what the searcher wants ‚Äì but also the ability to understand the actual search results. Powerset can only interpret so much of the gazilions of words out there. There is the whole problem of the source data, no just the query. Don‚Äôt get what I mean? Keep reading. But for now, learn this lesson
Computers have no idea about the data they are reading. In fact, Google pumping out those search results is based on people linking. Google is a machine, and reads 1s and 0s ‚Äì machine language. It doesn‚Äôt get human language
2) The Semantic web is about making what human‚Äôs read, machine readable
Tim Berner‚Äôs Lee, the guy that invented the World Wide Web and the visionary behind the Semantic Web, prefers to call it the ‚Äòdata web‚Äô. The current web is a web of documents ‚Äì by adding this extra data to content ‚Äì machines will be able to understand it. Metadata, is data about data.
A practical outcome of having a semantic web, is that Google would know that when it pulls up a web page regardless of the context of the words ‚Äì it will understand what the content is. Think of every word on the web, being linked to a master dictionary.
The benefit of the semantic web is not for humans ‚Äì at least immediately. The Semantic Web is actually pretty boring with what it does ‚Äì what is exciting, is what it will enable. Keep reading.
3) The Semantic web is for machines to interpret, not people
A lot of the skeptics of the semantic web, usually don‚Äôt see the value of it. Who cares about adding all this extra meta data? I mean heck ‚Äì Google still was able to get the website I needed ‚Äì the Hilton in Paris. Sure, the other 60% of the results on that page were irrelevant, but I‚Äôm happy.
I once came across a Google employee and he asked ‚Äúwhat‚Äôs the point of a semantic web; don‚Äôt we already enough metadata?‚Äù To some extent, he‚Äôs right ‚Äì there are some websites out there that have metadata. But the point of the semantic web is so that machines once they read the information, can start thinking like how a human would and connecting it to other information. There needs to be across the board metadata.
For example, my friend Michael was recently looking to buy a car. A painful process, because there are so many variables. So many different models, different makes, different dealers, different packages. We have websites, with cars for sale neatly categorised into profile pages saying what model it is, what colour it is, and how much. (Which may I add, are hosted on multiple car sites with different types of profiles). A human painfully reads through these profiles, and computes as fast as a human can. But a machine can‚Äôt read these profiles.
Instead of wasting his (and my) weekends driving around Sydney to find his car, a machine could find it for him. So, Mike would enter his profile in ‚Äì what he requires in a car, what his credit limit is, what his prior history with cars are ‚Äì everything that would affect his judgement of a car. And then, the computer can query every online website with cars to match the criteria. Because the computer can interpret these websites across the board, it can evaluate and it can go back to Michael and say ‚Äúthis is the car for you, at this dealer ‚Äì click yes to buy‚Äù.
The semantic web is about giving computers the information to be able to interpret data, so that it can do what they do really well ‚Äì compute.
4) A worldwide database
What essentially Berner‚Äôs Lee envisions, is turning the entire world wide web into a database that can be queried. Currently, the web looks like Microsoft Word ‚Äì one swab of text. However, if that swab of text was neatly categorised in an Excel spreadsheet, you could manipulate that data and do what you please ‚Äì create reports, reorder them, filter, and do whatever until your heart is content.
At university, I was forced to do an Information Systems subject which was essentially about the theory of databases. Damn painful. I learned only two things from that course. The first thing was that my lecturer, tutor, and classmates spoke less intelligible English than a caterpillar. But the second thing was that I learned what information is and how it differs from data. I am now going to share with you that lesson, and save you three months of your life.
You see, data is meaningless. For example, 23 degrees is data. On its own, it‚Äôs useless. Another piece of data in Sydney. Again, – useless. I mean, you can think all sorts of things when you think of Sydney, but it doesn‚Äôt have any meaning.
Now put together 23 degrees and Sydney, and you have just created information. Information is about creating relationships between data. By creating a relationship, an association, between these two different pieces of data ‚Äì you can determine it‚Äôs going to be a warm day in Sydney. And that is what information is: Relationship building; connecting the dots; linking the islands of data together to generate something meaningful.
The semantic web is about allowing computers to be able to query the sum of human knowledge like one big database to generate information
You are probably now starting to freak out and think ‚ÄúTerminator‚Äù images with computers suddenly erupting form under your computer desk, and smashing you against the wall as a battle between humans and computers begins. But I don‚Äôt see it like that.
I think about the thousands of hours humans spend trying to compute things. I think of the cancer research, whereby all this experimentation occurring in labs, is trying to connect new pieces of data with old data to create new information. I think about computers being about to query the entire taxation legislation to make sure I don‚Äôt pay any tax, because it knows how it all fits together (having studied tax, I can assure you ‚Äì it takes a lifetime to only understand a portion of tax law). In short, I understand the vision of the Semantic web as a way of linking things together, to enable computers to compute ‚Äì so that I can sit on my hammock drinking my beer, as I can delegate the duties of my life to the machines.
All the semantic web is trying to do, is making sure everything is structured in a consistent manner, with a consistent dictionary behind the content, so that a machine can draw connections. As Berner‚Äôs Lee said on one of the videos I saw: ‚Äúit‚Äôs all about creating links‚Äù.
The process to a Semantic Web is boring. But once we have those links, we can then start talking about those hammocks. And that’s when the power of the internet – the global network – will really take off.