Frequent thinker, occasional writer, constant smart-arse

Tag: search (Page 2 of 3)

Study finds people would not pay for privacy options

A 2007 study by researchers at Carnegie Mellon and the University of California at Berkeley found that most subjects were unwilling to spend even a quarter [25 cents] to keep someone from selling sensitive information about them — such as their weight or number of sex partners. “People prefer money over data, always,” says Alessandro Acquisti, assistant professor of information technology and public policy at CMU. Source: Wired

Is privacy really that big a deal? In short – yes – but paying for paid privacy options is not something I am sure of. Rather, I would expect the forces to swell up like a hurricane that will require political action to enforce privacy as a legal right. As a business, you shouldn’t think of privacy as a revenue stream, but rather, a branding tool to build trust. It’s users ignorance of the truth, not acceptance of the consequences, that has people giving away their data. Don’t abuse that trust if given the opportunity.

On the future of search

Robert Scoble has put together a video presentation on how Techmeme, Facebook and Mahalo will kill Google in four years time. His basic premise is that SEO’s who game Google’s algorithm are as bad as spam (and there are some pissed SEO experts waking up today!). People like the ideas he introduces about social filtering, but on the whole – people are a bit more skeptical on his world domination theory.

There are a few good posts like Muhammad‘s on why the combo won’t prevail, but on the whole, I think everyone is missing the real issue: the whole concept of relevant results.

Relevance is personal

When I search, I am looking for answers. Scoble uses the example of searching for HDTV and makes note of the top manufacturers as something he would expect at the top of the results. For him – that’s probably what he wants to see – but for me, I want to be reading about the technology behind it. What I am trying to illustrate here is that relevance is personal.

The argument for social filtering, is that it makes it more relevant. For example, by having a bunch of my friends associated with me on my Facebook account, an inference engine can determine that if my friend called A is also friends with person B, who is friends with person C – than something I like must also be something that person C likes. When it comes to search results, that sort of social/collaborative filtering doesn’t work because relevance is complicated. The only value a social network can provide is if the content is spam or not – a yes or no type of answer – which is assuming if someone in my network has come across this content. Just because my social network can (potentially) help filter out spam, doesn’t make the search results higher quality. It just means less spam results. There is plenty of content that may be on-topic but may as well be classed as spam.

Google’s algorithm essentially works on the popularity of links, which is how it determines relevance. People can game this algorithm, because someone can make a website popular to manipulate rankings through linking from fake sites and other optimisations. But Google’s pagerank algorithm is assuming that relevant results are, at their core, purely about popularity. The innovation the Google guys brought to the world of search is something to be applauded for, but the extreme lack of innovation in this area since just shows how hard it is to come up with new ways of making something relevant. Popularity is a smart way of determining relevance (because most people would like it) – but since that can be gamed, it no longer is.

The semantic web

I still don’t quite understand why people don’t realise the potential for the semantic web, something I go on about over and over again (maybe not on this blog – maybe it’s time I did). But if it is something that is going to change search, it will be that – because the semantic web will structure data – moving away from the document approach that webpages represent and more towards the data approach that resembles a database table. It may not be able to make results more relevant to your personal interests, but it will better understand the sources of data that make up the search results, and can match it up to whatever constructs you present it.

Like Google’s page rank, the semantic web will require human’s to structure data, which a machine will then make inferences – similar to how Pagerank makes inferences based on what links people make. However Scoble’s claim that humans can overtake a machine is silly – yes humans have a much higher intellect and are better at filtering, but they in no way can match the speed and power of a machine. Once the semantic web gets into full gear a few years from now, humans will have trained the machine to think – and it can then do the filtering for us.

Human intelligence will be crucial for the future of search – but not in the way Mahalo does it which is like manually categorising pieces of paper into a file cabinet – which is not sustainable. A bit like how when the painters of the Sydney harbour bridge finish painting it, they have to start all over again because the other side is already starting to rust again. Once we can train a machine that for example, a dog is an animal, that has four legs and makes a sound like “woof” – the machine can then act on our behalf, like a trained animal, and go fetch what we want; how those paper documents are stored will now be irrelevant and the machine can do the sorting for us.

The Google killer of the future will be the people that can convert the knowledge on the world wide web into information readeable by computers, to create this (weak) form of artificial intelligence. Now that’s where it gets interesting.

Google: the ultimate ontology

A big issue with the semantic web is ontologies – the use of consistent definitions to concepts. For those that don’t understand what I’m talking about – essentially, the next evolution of the web is about making content readable by not just humans but also machines. However for a machine to understand something it reads, it needs consistent definitions. Human’s for example, are intelligent – they understand that the word “friend” is also related to the word “acquaintance”, but a computer would treat them to mean two different things. Or do they?

Just casually looking at some of my web analytics, I noticed some people landed on my site by doing a google search for how many acquaintances do people have, which took them to a popular posting of mine about how many friends people have on facebook. I’ve had a lot of visitors because of this posting, and its been an interesting case study for me on how search engines work. However today was something different from other times: I found the word acquaintance weird. I know I didn’t use that word in my posting – and when I went to the Google cache I realised something interesting: because someone linked to me using that word, the search engine replaced the word ‘friend’ with ‘acquaintances’.

acquaintances

Google’s linking mechanism is one powerful ontology generator.

You need to be persistently adaptable

Tim Bull has recently written an interesting discussion point on when is the right time to innovate. In a post titled “Steam engine time“, he asks:

If innovation is a process of the right idea, in the right place and at the right time, how do we judge what the right time is and measure what is going on around us to hit the right spot?

Some would say luck has something to do with it, although I believe that is the perception from an outsiders point of view. In my eyes, a core set of attributes are required for innovation.

Consider this quote from Calvin Coolidge, 30th president of USA:

Nothing in the world can take the place of Persistence. Talent will not; nothing is more common than unsuccessful men with talent. Genius will not; unrewarded genius is almost a proverb. Education will not; the world is full of educated derelicts. Persistence and determination alone are omnipotent. The slogan ‘Press On’ has solved and always will solve the problems of the human race.

I think Tim is wrong to ask when is the right time, because innovators understand their environment, adapt to it – and then push until they get there. Persistence and adaptability, in my eyes, are two crucial aspects needed in a person or even a country or company, for it to successfully move forward. However whilst persistence is key – you need determination to push forward despite the barriers you are going to encounter – adaptability is the real secret to successfully innovating.

A case-study: multiculturalism in a flat world
Although I was born and bred in Australia, I have been brought up under a very strong Greek influence. With an Australian-born father, and a fresh-off-the-boat Greek mother – I have lived a life straddled in two cultures. Going to an Anglo-Saxon school, yet at the same time doing Greek classes at 9am Saturday (but leaving early for my schools footy games) – I grew to resent Australia’s multiculturalism policy. Without going into too much detail because this will turn it into a political discussion and detract from the point I wish to make – I disliked the fact that Greeks in Australia refused to integrate into the local culture. The Australian government’s stance of officially supporting Multiculturalism, which does things like pay for that Saturday morning tuition, was to me a stupid policy.

Fast forward to 2005, when I visited the Balkans as part of my nine months traveling around Europe. Serbia’s story is one of the saddest stories in Europe. Walking around the city of Belgrade, interacting with its inhabitants, and just generally experiencing Serbia – you realise you have come across a hidden gem in Europe. Yet once you look at the statistics and talk to some of the educated, you understand otherwise: a basket case situation that has little hope.

Serbia, like a lot of other countries I discovered in my travels, have a cultural problem: they can’t let go of the past. Millions of people have died over differing interpretations of history. The Republic of Macedonia’s identity is entirely staked on the fact they are situated on the lands of Alexander the Great. Identity to the nation states of Europe, is in history. And challenges to that history, and their identity, has led to some stupid wars affecting millions of innocent lives.

So guess what? I now think multiculturalism is the best thing my country could ever do, for the simple fact we can never have a fixed identity – what it meant to be Australian 50 years ago looks very different from what it looks like now. In Europe, identity is based on ethnicity with a fixed identity tied to history, language and a religion. In Australia, our identity isn’t allowed to be based on a certain ethnicity, and forces us to find common ground on what really matters like our way of life. If it wasn’t for the policy of Multiculturalism, we would be turning into one of these static nation states within Europe who become fixed as a certain point of time. The Greeks are still mourning over the Turks capturing the Great City of Constantinople from them in 1453 (which is why Tuesday is the unlucky day of the week for them). Yet for the countries like Australia, who don’t have much of a history – they are not locked – and consequently look forward, rather than back. Multiculturalism is a crucial ingredient to our success, because with all that diversity, it means we are constantly evolving our culture to the times without any one group fixing it. And with a globalised word, Australia’s ability to adapt to circumstances will be a key competitive advantage we have over countries.

If you don’t agree with me, have a read of Thomas Friedman’s The World is Flat – a book a entrepreneur/intrapreneur suggested I read. This guy who told me about the book was a German from Argentina, working for an Indian company to set up the company’s presence in Turkey! He told me that after he read that book, he quit his job and got himself into his current role. He faced the facts, and adapted his career.

Adaptability as success
You’re probably wondering what I am trying to get at, but to tie it back to my point about adaptability, successfully innovators need to constantly adapt to their environment. What happens with people once they get an idea, is that they spend all their time trying to fit it into a world that once existed, only for the world to be a entirely new place. Successful innovators need to constantly evolve their ideas, to the changing circumstances.

In October last year, I made a proposal at my firm to implement a new technology. For the months leading up to that point, people had to some extent talked down my idea and some even flat out rejected it. October however had me find the right person to hear my idea. And yet if I look at what I originally had thought, and what it is now – it is almost a completely different thing. Because when I pitched my idea, I was asked “why” it works and “how” is it different from anything else. It was that ‘why’ question that had me spend countless hours researching and understanding – adapting – my idea to the scenario being presented to me. I successfully made my business case, because I was given the opportunity to reframe my idea and adapt it to the circumstrances I was presented. Had I not adapted my original idea and vision, I wouldn’t be doing what I am doing now.

Of course, I could have summed up the above by mentioning Charles Darwin’s theory of evolution. Survival of the fittest, right? Adapt to the Green forest like that Green lizard that looks like a leaf, and you’ll find some food (rather than being the food yourself). Adaptability in life is a key critical success factor; and with innovation, it is the hidden factor that on the outside and in retrospect by others, gets attributed as luck.

Update 20/6/07: Catching up on some reading, I just came across a great posting by Marc Andreessen, an internet pioneer, who talks about the four types of luck and which nicely complements my thoughts above.

A casual chat with a media industry insider

Today I had the chance of picking the mind of Achilles from the International Herald Tribune, who last year was appointed Vice-President, circulation and development. Achilles is a family friend and I took the opportunity to talk to him about the world of media and the challenges being faced.

The IHT is one of the three daily financial newspapers of the world, along with the Wall Street Journal and the Financial Times. It is currently owned by the New York Times, and has a global circulation of 240,000 people. I had a great chat on a lot of different themes which could have me blog about for a week straight, but here are some of the facts I picked up from our discussion, which I will summarise below as future talking points:

  • On Murdoch’s acquisition of the Wall Street Journal: “very interested to see if he will remove the paid wall”.
  • The IHT experiemented with a paid wall for it’s opinion content, but they will be removing that later this year
  • He says the Bancroft family sold it because they are emotionally detached from the product. It was just an asset to them.
  • A lot of the content is simply reedited content from the NYT and internationalising it. For example, replacing sentences like “Kazakhstan in the size of New York state” doesn’t work well for an international reader who has no idea how big New York state is.
  • On the threat of citizen journalism with traditional media: “they are a competitive threat because we are competing for the same scarce resource: the attention of readers”
  • The problem with citizen journalism and bloggers is the validity of their information – behind a newspapers brand, is trust from readers of the large amounts of research and factchecking that occur. They have no credibility.
  • A blog may develop credibility with an audience greater than the New York Times. But this poses problems for advertising as advertisers might only advertise because of its niche audience. Blogs are spreading the advertising dollars, which is hurting everyone – it’s become decentralised and that has implications which are problematic.
  • The IHT’s circulation is spread thinly across the world. For example, it has 30,000 readers in France and six in Mauritius.
  • Their target market is largely the business traveller, which has its own unique benefits and problems. For example, a business traveler will read it for two days but when they get back home, they will revert to their normal daily newspaper. It’s not a very loyal reader.
  • Readership is a more important concept than circulation as it tells advertisers how big the actual audience of a publication is. For example, the average newspaper has 2.7 readers per copy. However due to the nature of the IHT’s readers, despite having high circulation, they have low readership.
  • IHT is in a unique position of relying on circulation revenue more than advertising. For example, a normal daily relies on circulation revenue as 20% of its total revenue; the IHT counts on it for 50%.
  • It’s hard to get advertising because a readership of university professors is less desirable than fund managers that might read the WSJ. Advertisers prefer to target key decision makers.
  • It doesn’t rely on classifieds as a revenue source – a key thing hurting the newspaper industry currently.
  • Although they place more reliance on circulation revenue, they still get some good advertising opportunities as a lot of readers are politicians and government decision makers.
  • They get a lot of advertising for fashion
  • Psychographic data is more important to advertisers than circulation and it shows what type of readership a publication has.

Half the problem has been solved with time spent

On Thursday, I attended the internal launch of the Australian Entertainment & Media Outlook for 2007-2011. It was an hour packed with interesting analysis, trends, and statistics across a dozen industry segments. You can leave a comment on my blog if you are interested in purchasing the report and I’ll see if I can arrange it for you.

One valuable thing briefly mentioned, was the irony of online advertising.
Continue reading

Thoughts on attention, advertising, and a metric to measure both: keep it simple

Advertising on the Internet is exploding. Assuming you accept my premise that the Internet will be the backbone of the world’s attention economy – then, I am sure you can see the urgency of developing an effective metric for measuring audiences that consume content online. Advertisers are expecting more accountability online and there is increasing demand for an independent third-party to verify results. But you can’t have accountability and there is no value in audits, if one place measures in apples and the other in bananas.

The Attention Economy is seriously lacking an effective measurement system

Ajax broke the pageview model of impressions, the one billion-dollar practice of click-fraud is the dirty big secret of pay-for-performance advertising, and the other major metric of using unique visitors (through cookies) is proving inaccurate.

It sounds crazy, doesn’t it? The Internet has the best potential for targeted advertising, and advertisers are moving onto it in stampedes – and yet, we still can’t work out how to measure audiences effectively. Measurement is broken on the Net.

(Although I am focusing on advertising, this can be applied in other contexts. An advertising metric is simply putting a monetary value on what is really an attention metric.)

Yet when we look at the traditional media, are we being a little harsh on this new media? Is the problem with the web’s measurement systems just that it is more accountable for its errors? After all – radio, television, and print determine their audience through inference which are based on sampling methods and not actually directly measuring an audience. Sampling is about making educated guesses – but a guess is still a guess.

Maybe another way of looking at it is that the old way of doing advertising is no longer effective. Although we can say pageviews are broken due to AJAX, the truth is it was always an ineffective measurement system, as it was based on the traditional media’s premise of how many viewers/subscribers theoretically and potentially could see that ad. As an example of why this is not how it should be: when people visit my blog via Google Images, they hang around for 30 seconds. People that search for business issues on the web that I write about, like stuff you are reading right now – spend 5+ minutes. If both are equal in terms of page views, but the later actually reads the pages and the former only scans the content for an image – why are we treating them equally? My blog is half about travel, and half about the business of the internet, which is why I have two very different audiences. Just because I get high page views from my travel content, doesn’t mean I can justify higher CPM’s for people that want to advertise on internet issues. Not all pageviews are the same – especially when I know the people giving me high pageviews, arn’t really consuming my content

Another issue is that advertisers are so caught up on who can create the most entertaining 30 second ad, that the creativity to get people entertained has ovetaken the reason why advertising happens in the first place: to make sales. The way you do that, is by communicating your product to the people that would want to buy it. If I placed advertising on this blog, from people who want to do web-business related stuff, they should only pay for the peope that read my blog postings for 5+ minutes on the Attention economy, not for the Google images searchers who are looking for porn (my top keywords, and how people find my blog, makes me laugh out loud sometimes!).

When we create a metric that measures attention, lets be sure of one thing: the old way is broken, and the new ways will continue to be broken if we simply copy and paste the old ways. New ways like click-through ads that appear on search results, and account for 40% of internet advertising is not how advertising should be measured. The reason is because it is putting the burden of an effective advertising campaign, on a publisher. Why should a publisher not get paid, with the opportunity cost of not using another ad that would have paid, because of the ineffectiveness of the advertisers campaign strategy at targeting?

When measuring audience attention, lets not overcomplicate it. It should be purely measuring if someone saw it. As an advertiser, I should be able to determine which people from which demograph can see it my ad – and yes, I will pay the premium for that targeting. If it turns into a sale, or if they enjoyed the content – is where your complex web analytic packages come in. But for a simple global measurement system, lets keep it simple.

Concluding thought

If I stood at the toll booths of the Sydney Harbour bridge naked, some people will honk at me and others won’t. If I can guarantee that they can see me naked, that’s all as a publisher I need to do. It’s the advertisers problem if people honk at me or not. (Not enough honks means as a model I should still get my wage. They just need to hire a better looking model next time!)

The attention economy needs a consistent base

Okay, enough naval gazing. The journalist in me (by experience), the accountant in me (by education), and the businessman in me (by occupation) is going to synthesise my understanding of the world and propose a new metric for the attention economy. I don’t know the answer yet, but I am going to use this blog to develop my thinking. I can’t promise a solution, however I am sure breaking the issue down into key requirements, assumptions, and needs of what this magical metric is – will add value somewhere for someone.

So let’s start with the most important assumption of all: what are we measuring? As Herbert Simon coined it, and smart guys like Umair, Scott and Chris have extended (at least for my conceptual understanding) – it is called the attention economy. It is important to note however, that the attention economy is an aspect of the Information Sector (see below). And as I described in a previous posting, the attention economy needs a metric for two reasons: monetisation and feedback.


What incorporates the attention economy?
Well, this is a bit like a related problem I had when I first came to grips with what new media was. A few years back, I did some active research trying to understand how a book, a television, a newspaper, and a search engine – could all somehow be classed as “media”. I found my question answered by Vin Crosbie’s manifesto (read this for a recent summary). Take note of what he considers is the key element of new media (the technology aspect).

I am going to propose one of my key assumptions of the future, which will answer this question. It might not happen for another 5, 10 or even 20 years – but I am convinced this is the future. The Internet will act as infrastructure.

I believe the unifying aspect, and the backbone of the attention economy, will be the Internet. All enterprise software, all consumer software, all (distributed) entertainment, all (distributed) communications and all information – will be delivered digitally over the Internet. I think the people at the US Census bureau?Ç? conceptually have already worked this out by defining the information sector of the economy, which classes the above mentioned and more into this one diverse category. The Internet is the enabler of the Information Age, just like how the production line was for the Industrial Age

I’m not saying we are going to live, sleep, and eat on computers in the future. However just think – anything that runs on electricity, can connect to the Internet. And look at the technologies being developed that enable the Internet to live beyond the computer screen like electronic paper and?Ç? dynamic interfaces. Even more powerfully, is that the Internet has brought entire industries to their knees – like the newspaper and music industries – because it is providing a more efficient way of delivering content. If it’s information, communication or entertainment related – then it probably works better in digital format, over the Internet. (Excluding of course the things like theme parks and the like, which are more about physical entertainment and not distributed entertainment like a television programme).

I think this is an important issue to be recognised, that the Internet will the the backbone of the attention economy. By being the core back-end, it means that no matter the output device – whether it is mobile phone, a computer, or a television – it will be providing a consistent delivery mechanism for digital information. For a measurement system to work, it needs to be consistent. The Internet infrastructure will be that consistency. If you can recognise that, then that is a big step forward to solving the issue.

Faraday Media – Particls

This series of blog posts – wizards of oz – is to highlight the innovation we have down under. So I begin with Faraday media, a Brisbane based start-up that launched their keynote product today,

Particls is an engine that learns what you are interested in, and alerts you when content on the internet becomes available – through a desktop ‘ticker’ or pop-up alerts.

Value
1) It’s targeted. Particls is an attention engine – it learns what you want to read, and then goes and finds relevant information. That’s a powerful tool, for those of us drowning in information overload, and who don’t have time to read.

2) It catches your attention. Particls is based on the concept of ‘alerts’ – information trickles across your screen seemlesly as you do your work, like a news ticker. For the things that matter, an alert will pop-up. The way you deal with information overload is not by shutting yourself out – it’s by adjusting the volume on things that you value more than other things.

3) The founders understand privacy. They started the APML standard – a workgroup I joined because it’s the best attempt I have seen yet that tackles the issue of privacy on the internet. For example, I can see what the Particls attention engine uses to determine my preferences – lists of people and subjects with “relevance scores”. And better yet – it’s stored on my hard-disk.

4) It’s simple. RSS is a huge innovation on the web, that only a minority of users on the internet understand. The problem with RSS (Real Simple Syndication), is that it’s not simple. Particles makes it dead simple to add RSS and track that content.

Conclusion

Why the hell doesn’t Fairfax acquire the start-up, rather than wasting time creating yet another publication (incidently in the same city) that we don’t have time to read. In my usage of the product, I have been introduced to content that I am interested in, that I never would have realised had existed on the web. In my trials, I have mainly used it to keep track of my research interests, and despite my skepticism about how ‘good’ the the attention engine is, it has absolutely blown me away.

And it’s not just in the consumer space – a colleague (who happens to hold a lot of influence in enterprise architecture of our 140,000 person firm) was blasting RSS one day on an internal blog – saying how we don’t yet have the technology to ‘filter’ information. I told him about Particls – he’s now in love. If a guy like him, who shapes IT strategy for a $20 billion consulting firm, can get that excited – that’s got to tell you something.

« Older posts Newer posts »