Frequent thinker, occasional writer, constant smart-arse

Tag: mashup

Data portability allows mashup for Australian bush fire crisis

Last night in Australia, one of the states developed a series of bush fires that have ravaged communities – survivors describe it as “raining fire” that came out of no where. As I write this, up to 76 people have been killed.

Victorian AU Fires 2009
The sky is said by Dave Hollis to look how it is in the movie ‘Independence Day’

An important lesson has come out out of this. First, the good stuff.

Googler Pamela Fox has created an invaluable tool to display the bush fires in real time. Using Google technologies like App engine and the Maps API (which she is the support engineer for), she’s been able to create a mashup that helps the public.

She can do so because the Victorian Fire department supports the open standard RSS. There are fires in my state of New South Wales as well, but like other Fire Department’s in Australia, there is no RSS feed to pull the data from (which is why you won’t see any data on the map from there) It appears states like NSW do support RSS for updates, but it would be more useful if there was some consistency – refer to discussion below about the standards.

For further information, you can read the Google blog post.

While the Fire Department’s RSS allows the portability of the data, it doesn’t have geocodes or a clear licence for use. That may not sound like a big deal, but the ability to contextualise a piece of information in this case matters a hell of a lot.

As a workaround, Pamela sent addresses through the Google geocoder to develop a database of addresses with latitude and longtitude.

GeoRSS and KML
In the geo standards world, two dominant standards exist that enable the portability of data. One is an extension to RSS (GeoRSS) that allows you to extend an RSS feed to show geodata. The other in Keyhole Markup Language, which was a standard developed by Google. GeoRSS is simply modifying RSS feeds to be more useful, while KML is more like how HTML is.

If the CFA and any other websites had supported them either of these standards, it would have made life a lot more easier. Pamela has access to Google resources to translate the information into a geocode and even she had trouble. (Geocoding the location data was the most time-consuming of the map-making process.)

The lessons
1) If you output data, output it in some standard structured format (like RSS, KML, etc).
2) If you want that data to be useful for visualisation, include both time and geographic (latitude/longitude information). Otherwise you’re hindering the public’s ability to use it.
3) Let the public use your data. The Google team spent some time to ensure they were not violating anything by using this data. Websites should be clearer about their rights of usage to enable mashers to work without fear
4) Extend the standards. It would have helped a lot of the CFA site extended their RSS with some custom elements (in their own namespace), for the structured data about the fires. Like for example <cfa:State>Get the hell out of here</cfa>.
5) Having all the Fire Department’s using the same standards would have make a world of difference – build the mashup using one method and it can be immediately useful for future uses.

Pamela tells me that this is the fifth natural disaster she’s dealt with. Every time there’s been an issue of where to get the data and how to syndicate it. Data portability matters most for natural disasters- people don’t have time to deal with scraping HTML (didn’t we learn this with Katrina?).

Let’s be prepared for the next time an unpredictable crisis like this occurs.

Three startups in 24 hours – lessons in the costs of innovation

I’m sitting here at Start-up weekend, a concept instigated by Bart Jellema and partner Kim Chen, as an experiment of bringing the Australian tech community together, part of the broader effort several of us are pursuing to build Silicon Beach in Australia.

I’ve dropped in on the tail-end, and it’s amazing to see what happened. Literally, in the space of 24 hours, three teams of five people have created three separate products. They are all well-thought out, with top-notch development, and clearly designed to actually pass off as a genuine start-up that’s been working on the idea for weeks if not months.

TrafficHawk.com.au is a website delivering up to the minute traffic alerts for over six million drivers in the state of New South Wales, the biggest state in Australia
LinkViz.com is a service that enables you to visually determine what’s hot on Twitter, a social media service.
uT.ag is a service that monetises links people share with other people.

All three of these products have blown me away, not just in the quality, but the innovation. For example, ut.ag is a url shortening application that competes against the many others out on the market (popular with the social media services) but adds an advertisement to the page people click to view. Such a stupidly simple idea that I can’t believe it hasn’t been created before…and it’s already profitable in the few hours of operation! LinkViz provides a stunningly visual representation of links, that it’s hard to believe this was created – from concept to product – in such a short time. Traffic Hawk is a basic but useful mashup that genuinely adds value to consumers.

The challenges
This is a clear demonstration that when you get a bunch of people together, anything can happen: pure innovation, in a timewarp of a few hours. Products, with real revenue potential. It’s scary to witness how the costs of business in the digital economy can be boiled down to simply a lack of sleep! But talking to the teams, I realised that the costs were not as low as they could have been. It also is an interesting insight into the costs to business on the internet, seen through this artificial prism of reality.

For example, Geoff McQueen (a successful developer/businessman, that was also one of the founders of Omnidrive) of the Traffic Hawk site was telling me about the difficulty they experienced. Whilst the actual service looks like a basic mashup on Google maps, the actual scraping of data from a government website (real time traffic incidents), was considerably painful. Without attention from the team in future, their site will break as the scraping script is dependent on the current configuration with the HTML pages displaying the data. The fact that this is a ‘cost’ for them to develop the idea, is a wasteful cost of business. The lost resources in developing the custom scraping script, and the future maintenance required, is an inefficient allocation of resources in the economy. And for what good reason? Traffic hawk is making the same data, more useful – it’s not hurting, only helping the government service.

This is a clear example of the benefits of open data, where in a DataPortability enabled world, the entrepreneurial segments of the economy that create this innovation, can focus their efforts in areas that add value. In fact, it is a clear demonstration of the value chain for information.

LinkViz on the other end have a different disadvantage. Whilst they have access to the Twitter API that enables them to pull data and create their own mashup with it (unlike the hawkers), the actual API is slow. So the price for a better service from them, is money to get faster access.

Talking with the ut.ag boys, the issue was more a matter of infrastructure. They could have had the same product launched with more features, 12 hours earlier (ie, 12 hours AFTER conception) if it wasn’t for the niggly infrastructure issues. This is more a function of the purposely disorganised nature of the camp, and so the guys lost a lot of time with setting up networks and repositories. I believe however, this should be a lesson about the impact of the broader ecosystem and governments influence on innovation.

For example, we need faster broadband…but also an omnipresent connection. A person should be able to flip open their laptop, and access the internet in an affordable matter that connects them to their tools, anywhere.

Concluding thoughts
The whole purpose of this camp, was not the destination but the process. People getting to know each other, learn new skills – or as Linda Gehard says: “It’s like exercise for entrepreneurs”. However what started as a quick and dirty post to give the guys some exposure, has had me realise the costs to innovation in the economy. When you take out the usual whines about investors and skills shortages, and put together some highly capable people, there are some specific things that still need to be done.

Update 8/9/08:Apologies for the typos and misspellings – my blog locked me out due to a corruption in the software and didn’t save my revised version that I thought I saved. And a huge thank you to Joan Lee, who has now become my official proof reader. Liako.Biz, it appears, is no longer a one man band!