Frequent thinker, occasional writer, constant smart-arse

Tag: techmeme

On the future of search

Robert Scoble has put together a video presentation on how Techmeme, Facebook and Mahalo will kill Google in four years time. His basic premise is that SEO’s who game Google’s algorithm are as bad as spam (and there are some pissed SEO experts waking up today!). People like the ideas he introduces about social filtering, but on the whole – people are a bit more skeptical on his world domination theory.

There are a few good posts like Muhammad‘s on why the combo won’t prevail, but on the whole, I think everyone is missing the real issue: the whole concept of relevant results.

Relevance is personal

When I search, I am looking for answers. Scoble uses the example of searching for HDTV and makes note of the top manufacturers as something he would expect at the top of the results. For him – that’s probably what he wants to see – but for me, I want to be reading about the technology behind it. What I am trying to illustrate here is that relevance is personal.

The argument for social filtering, is that it makes it more relevant. For example, by having a bunch of my friends associated with me on my Facebook account, an inference engine can determine that if my friend called A is also friends with person B, who is friends with person C – than something I like must also be something that person C likes. When it comes to search results, that sort of social/collaborative filtering doesn’t work because relevance is complicated. The only value a social network can provide is if the content is spam or not – a yes or no type of answer – which is assuming if someone in my network has come across this content. Just because my social network can (potentially) help filter out spam, doesn’t make the search results higher quality. It just means less spam results. There is plenty of content that may be on-topic but may as well be classed as spam.

Google’s algorithm essentially works on the popularity of links, which is how it determines relevance. People can game this algorithm, because someone can make a website popular to manipulate rankings through linking from fake sites and other optimisations. But Google’s pagerank algorithm is assuming that relevant results are, at their core, purely about popularity. The innovation the Google guys brought to the world of search is something to be applauded for, but the extreme lack of innovation in this area since just shows how hard it is to come up with new ways of making something relevant. Popularity is a smart way of determining relevance (because most people would like it) – but since that can be gamed, it no longer is.

The semantic web

I still don’t quite understand why people don’t realise the potential for the semantic web, something I go on about over and over again (maybe not on this blog – maybe it’s time I did). But if it is something that is going to change search, it will be that – because the semantic web will structure data – moving away from the document approach that webpages represent and more towards the data approach that resembles a database table. It may not be able to make results more relevant to your personal interests, but it will better understand the sources of data that make up the search results, and can match it up to whatever constructs you present it.

Like Google’s page rank, the semantic web will require human’s to structure data, which a machine will then make inferences – similar to how Pagerank makes inferences based on what links people make. However Scoble’s claim that humans can overtake a machine is silly – yes humans have a much higher intellect and are better at filtering, but they in no way can match the speed and power of a machine. Once the semantic web gets into full gear a few years from now, humans will have trained the machine to think – and it can then do the filtering for us.

Human intelligence will be crucial for the future of search – but not in the way Mahalo does it which is like manually categorising pieces of paper into a file cabinet – which is not sustainable. A bit like how when the painters of the Sydney harbour bridge finish painting it, they have to start all over again because the other side is already starting to rust again. Once we can train a machine that for example, a dog is an animal, that has four legs and makes a sound like “woof” – the machine can then act on our behalf, like a trained animal, and go fetch what we want; how those paper documents are stored will now be irrelevant and the machine can do the sorting for us.

The Google killer of the future will be the people that can convert the knowledge on the world wide web into information readeable by computers, to create this (weak) form of artificial intelligence. Now that’s where it gets interesting.

Tangler

This is the second post in a series – wizards of oz – which is to highlight the innovation we have down under, and how the business community needs to wake up and realise the opportunities. I review Tangler, a Sydney-based start-up that has recently released their application to the world as a public beta.

Tangler is a web-service that enables discussions over a network. Think of discussions with the immediacy of Instant Messaging (it’s easy), but with the persistency of a forum (messages are permanently stored). Discussions are arranged into communities of interest (groups), which are further broken down into topic areas. Click here to see a video overview.

Value

1) It’s a network application. Although it’s got a great design, and looks like a funky website, the real power of this web service is what it’s working towards: discussions over a network. Imagine a little widget with the topic “What do you think of Elias Bizannes?” placed on my (external) personal blog, my internal work blog, my myspace/facebook/social networking page, as well as it’s own dedicated forum on the Tangler site. A centralised discussion, in a decentralised manner. That’s big.

2) It’s community has great DNA. Communities are not easy things to build – my own experience on a getting-bigger-by-the-day internal project has shown that it is a complex science, touching everything from understand motivational theory to encouraging the right kind of behaviours (policing without policing). My usage on the site has shown to me that the active community building currently occuring, is on the right track. Anyone can hire a code monkey, wack on some flashy front-end, and say they have a great product. But not anyone can build a strong community – even Google struggles on this (the acquisition of YouTube happened largely because of community, because the YouTube community beat Google’s own service). Tangler’s community is already turning into a powerful asset – the DNA is there – now it just needs exposure, and the law of cumulative advantage will kick in.

3) The founder and staff are responsive to its community. I posted a question on the feedback forum, to prove this point: I got a response in an hour, on a Saturday. The staff at Tangler are super responsive – which in part, is due to the real-time discussion ability of the software – but also because of their commitment. As I state above – the value of Tangler is the community of users it builds – this type of responsiveness is crucial to keep its users satisfied to come back, because it makes them feel valued. Additionally, the community is driving the evolution of the application, and that’s the most powerful way to create something (adapting to where there is a need by the people that use it)

4) It’s a platform. What makes Tangler powerful, is that it encourages discussions around niche content areas. Make that niche content, being created for free. Low cost to produce + highly targeted content = an advertisers dream. Link it with a distributed network across the entire Internet (see 1 above), and you’ve got something special.

Conclusion

Social networks, which is what Tangler is, are characterised by:
1) the existence of a repository of user-generated content and
2) the need of members to communicate.

Tangler’s user-generated content and communications web make them an interesting fit for both media conglomerates and telecommunication companies (but for different reasons). I see a Tangler acquisition as a no-brainer for the big Telco’s. Integrating a social network like Tangler into Telstra, builds on the synergy between the communication needs of social network users and the communications expertise and service infrastructure of the communication companies. Unlike voice calls that are a commodity now, the Telco’s need to take advantage of their network infrastructure and accommodate for text-based discussions, which can be monetised for as long as the content exists (with advertising).

The challenge for Tangler however – as with any other Internet property – is that the scale of the audience of social networks determines the nature of the relationship with a communications company. Micro-sized social networks are not interesting to communication companies. Massive social networks are, but history has shown they would rather be partners than be acquired. To be attractive to the big end of town, Tangler needs to show to have a scale large enough to grow as a business but not too large to dictate the terms of the business.

My observations conclude me to think that they will be a hit once they open up their application to external developers, which will relieve the development bottleneck faced by their resource and time constrained team. However they shouldn’t rush this, as I still think their performance issues are not completely ironed out yet. An open API would be taken up by its enthusiastic community who are technologically orientated. Not too mention the strong relationships the CEO and CMO have forged with the local web entrepreneurial and development community in Australia.

My boss is currently doing a secondment as acting Finance Director at Sensis, Telstra’s media arm. Maybe I need to organise a catch-up with him, before these guys get snatched up by some US conglomerate!

My media consumption

I’ve only recently started blogging again, and I need to get into the habit. Even though I have a gazillion things I want to write about, I literally don’t have the time! So here’s a quick post, to keep me – ahem – regular. It’s actually something I think about a lot, given my interest in the internet started from my interest/background in media. Meme was started by Jeremiah, and I saw it on a posting by Marty.

Continue reading