Search and the Semantic Web

Recently I have been slowly working my way through Lee Odden’s recommended reading list. I just finished John Battelle’s The Search and can see why Mr. Battelle has been so successful. He is a captivating writer and a pretty good audio book narrator not to mention a brilliant marketer and entrepreneur. Released in 2006, The Search provides a concise history of search and the rise of the internet billionaires behind Google and various other search related companies. It is amazing to think how far the web and internet search has come in the last decade and even more amazing to realize that most leading edge search companies agree that search is in it’s infancy and is likely no more than 5% of the true potential.

One of the more interesting innovations in search in the last few years as noted by Battelle is IBM’s WebFountain. WebFountain is a wobbly step towards the search holy grail that is clarity into the database of intentions. Unfortunately, according to IBM, although WebFountain is incredibly powerful and far more complex than the slurp and burp Google index of today, it lacks scalability. The WebFountain index generates 10 megabytes of data for every 1 megabyte it reads in. There are, it seems, many internet billionaires yet to be crowned in the kingdom of search.

Of course the database of intentions is tightly coupled to the fabled semantic web. The father of the internet, Tim Berners Lee recently did an interview at HP and spoke on his thoughts about the future of web science and the semantic web. One of the large problems Sir Lee addressed in the interview was how the wiki model is breaking down. He discusses how the open world editable concept behind wikis is slowly being deflated as they are forced to filter and limit control due to misuse and abuse and how the world needs to work together to eliminate this issue by making collective decisions that contribute to a successful meritocrasy. He also addressed the issue surrounding access and control of private systems such as MySpace. He believes that companies and sites that attempt to wall off their data and prevent open use and access will risk being abandoned in favor of open social networks that will allow people to access, share and control their own data. In the post interview Q&A Berners Lee touched on the white hot topic of centrally managed and openly accessible social networking data. As the web evolves it will be more about people interacting with people he explains. We need to move away from the document level focus and instead shine our light in the direction of people’s actions and the relationships between real things.

The solution to all these problems, Sir Lee posits, lies in the adoption of RDF, a directed, labeled graph data format for representing information in the Web. RDF in conjunction with various other semantic web technologies like
GRDDL (Gleaning Resource Descriptions from Dialects of Languages) and a query language SPARQL. The likelihood of successful adoption of these technologies seems high given the group supporting the adoption and the fact that GRDDL is basically extended XML and SPARQL is syntactically similar to SQL.

It is an exciting time to be an SEO. In reality it is an exciting time to be an internet user. As we look back on 2007 in future years we will likely scoff at the rudimentary tools used by the cro-magnon man searcher of yesteryear.