Last night in Australia, one of the states developed a series of bush fires that have ravaged communities – survivors describe it as “raining fire” that came out of no where. As I write this, up to 76 people have been killed.
The sky is said by Dave Hollis to look how it is in the movie ‘Independence Day’
An important lesson has come out out of this. First, the good stuff.
Googler Pamela Fox has created an invaluable tool to display the bush fires in real time. Using Google technologies like App engine and the Maps API (which she is the support engineer for), she’s been able to create a mashup that helps the public.
She can do so because the Victorian Fire department supports the open standard RSS. There are fires in my state of New South Wales as well, but like other Fire Department’s in Australia, there is no RSS feed to pull the data from (which is why you won’t see any data on the map from there) It appears states like NSW do support RSS for updates, but it would be more useful if there was some consistency – refer to discussion below about the standards.
For further information, you can read the Google blog post.
While the Fire Department’s RSS allows the portability of the data, it doesn’t have geocodes or a clear licence for use. That may not sound like a big deal, but the ability to contextualise a piece of information in this case matters a hell of a lot.
As a workaround, Pamela sent addresses through the Google geocoder to develop a database of addresses with latitude and longtitude.
GeoRSS and KML
In the geo standards world, two dominant standards exist that enable the portability of data. One is an extension to RSS (GeoRSS) that allows you to extend an RSS feed to show geodata. The other in Keyhole Markup Language, which was a standard developed by Google. GeoRSS is simply modifying RSS feeds to be more useful, while KML is more like how HTML is.
If the CFA and any other websites had supported them either of these standards, it would have made life a lot more easier. Pamela has access to Google resources to translate the information into a geocode and even she had trouble. (Geocoding the location data was the most time-consuming of the map-making process.)
The lessons
1) If you output data, output it in some standard structured format (like RSS, KML, etc).
2) If you want that data to be useful for visualisation, include both time and geographic (latitude/longitude information). Otherwise you’re hindering the public’s ability to use it.
3) Let the public use your data. The Google team spent some time to ensure they were not violating anything by using this data. Websites should be clearer about their rights of usage to enable mashers to work without fear
4) Extend the standards. It would have helped a lot of the CFA site extended their RSS with some custom elements (in their own namespace), for the structured data about the fires. Like for example <cfa:State>Get the hell out of here</cfa>.
5) Having all the Fire Department’s using the same standards would have make a world of difference – build the mashup using one method and it can be immediately useful for future uses.
Pamela tells me that this is the fifth natural disaster she’s dealt with. Every time there’s been an issue of where to get the data and how to syndicate it. Data portability matters most for natural disasters- people don’t have time to deal with scraping HTML (didn’t we learn this with Katrina?).
Let’s be prepared for the next time an unpredictable crisis like this occurs.
Fascinating read! I've always been a big fan of interoperable data. I just saw that someone had used a mashup website to directly stream CFA RSS data to Twitter. Great for those on the go. Now we just need more permissive licensing for the data
Great article Elias about a difficult subject. Having roots in Southern California, I know first hand the devastation and horror of wild fires and also the value of fast and accurate information flow. Hopefully Data Portability will be able to help this process because in this case its success just may help save lives. In the mean time, I hope everyone down there stays safe!
Fascinating read! I’ve always been a big fan of interoperable data. I just saw that someone had used a mashup website to directly stream CFA RSS data to Twitter. Great for those on the go. Now we just need more permissive licensing for the data
Great article Elias about a difficult subject. Having roots in Southern California, I know first hand the devastation and horror of wild fires and also the value of fast and accurate information flow. Hopefully Data Portability will be able to help this process because in this case its success just may help save lives. In the mean time, I hope everyone down there stays safe!
It's enormous tragedy and, one would hope, that out of it some good will eventually come – like true and reliable inteoperable data that would be so helpful in situations like this.
The whole essence of the interoperability is that information will be available when needed. aus-emaps.com is built on this principle but the reality is that data suppliers are not reliabe and hence undermine the whole concept. It's not that it's "cutting edge stuff"… the standards for Web Map Service and Web Feature Service are in place since early 2000's (well before Google put forward KML standard). A few years ago Australian Government even spent $1.5 million promoting interoparability for emergency response and still not much happened. One can just hope that "common interest" will prevail.
When it works, you can see fire hotspots in Australia from satellite observations on this page: http://www.aus-emaps.com/fires.php
Elias good article and something the LA fire department have been very open about developing. They use Twitter, google maps, sms applications and Yahoo pipes to monitor and report in real time fire fighting operations. An incredible story and its great to see technology like this being used in such a way, very proud to be in this space. Its not all about marekting and social media!
"Short of motorized fire apparatuses, this technology is the best thing that's happened to our department in 122 years," Humphrey says. "It holds more potential to save lives than any other civic tool." – Brian Humphrey, Firefighter/Specialis, Public Service Officer, Los Angeles Fire Department
http://forwebsake.blogspot.com/2008/11/la-fire-de…
It's enormous tragedy and, one would hope, that out of it some good will eventually come – like true and reliable inteoperable data that would be so helpful in situations like this.
The whole essence of the interoperability is that information will be available when needed. aus-emaps.com is built on this principle but the reality is that data suppliers are not reliabe and hence undermine the whole concept. It's not that it's "cutting edge stuff"… the standards for Web Map Service and Web Feature Service are in place since early 2000's (well before Google put forward KML standard). A few years ago Australian Government even spent $1.5 million promoting interoparability for emergency response and still not much happened. One can just hope that "common interest" will prevail.
When it works, you can see fire hotspots in Australia from satellite observations on this page: http://www.aus-emaps.com/fires.php
Elias good article and something the LA fire department have been very open about developing. They use Twitter, google maps, sms applications and Yahoo pipes to monitor and report in real time fire fighting operations. An incredible story and its great to see technology like this being used in such a way, very proud to be in this space. Its not all about marekting and social media!
"Short of motorized fire apparatuses, this technology is the best thing that's happened to our department in 122 years," Humphrey says. "It holds more potential to save lives than any other civic tool." – Brian Humphrey, Firefighter/Specialis, Public Service Officer, Los Angeles Fire Department
http://forwebsake.blogspot.com/2008/11/la-fire-de…
Elias,
I am aware that NSW RFS have very recently developed software to create GeoRSS feeds which is great news and an example for many other government bodies who have such rich stores of geographic information tied up in proprietary formats.
The supply of data through GeoRSS/KML is particularly interesting given that it moves beyond the model where organisations supply data in the form of a static or dynamic map based upon images. By supplying data in these formats you open up a world of opportunity for others to consume and be creative.
Having standards in place does enable data to be more readily shared however traditionally organisations such as the fire service have not necessarily had the people to push information sharing and technology up the agenda. Its a continuing challenge.
Elias,
I am aware that NSW RFS have very recently developed software to create GeoRSS feeds which is great news and an example for many other government bodies who have such rich stores of geographic information tied up in proprietary formats.
The supply of data through GeoRSS/KML is particularly interesting given that it moves beyond the model where organisations supply data in the form of a static or dynamic map based upon images. By supplying data in these formats you open up a world of opportunity for others to consume and be creative.
Having standards in place does enable data to be more readily shared however traditionally organisations such as the fire service have not necessarily had the people to push information sharing and technology up the agenda. Its a continuing challenge.
I have been looking for location based data for bushfire hotspots. The data can have (longitude, laititude) and the we can developed a similar density map using google map and heatapi
http://themechanism.com/barkode/demos-tutorials/g…
I hope someone can point me in the write direction so we can develop this app.
I've been getting private e-mails about this, so responding to XP publicly for other peoples benefit. Pamela Fox who developed the map, has the data stored on Google App Engine: http://mapvisage.appspot.com/
They take the CFA RSS feed and turn it into a GeoRSS feed, available here: http://mapvisage.appspot.com/query/fire?agency=vi…
You can use that in the Maps API via GGeoXml or just load into maps by putting the URL in the search box.
Hi
Look at this:
http://xorprime.azzenti.com/densityMap/
I just got the data from: http://sentinel.ga.gov.au/acres/sentinel/gpx_data…
without putting any thinking on it.
Anyway, thanks for sharing that link.
I have been looking for location based data for bushfire hotspots. The data can have (longitude, laititude) and the we can developed a similar density map using google map and heatapi
http://themechanism.com/barkode/demos-tutorials/g…
I hope someone can point me in the write direction so we can develop this app.
I've been getting private e-mails about this, so responding to XP publicly for other peoples benefit. Pamela Fox who developed the map, has the data stored on Google App Engine: http://mapvisage.appspot.com/
They take the CFA RSS feed and turn it into a GeoRSS feed, available here: http://mapvisage.appspot.com/query/fire?agency=vi…
You can use that in the Maps API via GGeoXml or just load into maps by putting the URL in the search box.
Hi
Look at this:
http://xorprime.azzenti.com/densityMap/
I just got the data from: http://sentinel.ga.gov.au/acres/sentinel/gpx_data…
without putting any thinking on it.
Anyway, thanks for sharing that link.
We (at Pitney Bowes) have also extended FireLocator to offer the CFA and sentinel data.
The CFA integration uses a similar approach.
Check it out at http://firelocator.net
Regards
We (at Pitney Bowes) have also extended FireLocator to offer the CFA and sentinel data.
The CFA integration uses a similar approach.
Check it out at http://firelocator.net
Regards