In Memoriam
December 21, 2011 — October 12, 2016
Real World Index was an experiment in real-time data to measure what matters in our society. I still think it's important to bring social and environmental metrics into our day-to-day awareness. But it's not a job to which I can do justice in my spare time.
The decision to retire this project came after realising that Real World Index had plateaued. I haven't had time to add new metrics, and some of the ones I had started with were no longer working reliably. There are data quality issues. I'm not a statistician, and I never was able to synthesize the data into a clear, meaningful index.
I leave this thread here, in the hope that someone picks it up and wants to move forward with it in the future. If that someone is you, please get in touch.
— Sam Nabi
Screenshots
Data
Use this data at your own risk. It is certainly not suitable for academic use.
Data quality issues
- Waterloo Region Crime data not comparable year-over-year (change in reported data after August 20, 2013; not comparable)
- All UNIX timestamps do not correctly reflect UTC. They are offset 4 or 5 hours depending on whether daylight saving time applies in the Eastern timezone. (So when a timestamp displays 5:00 PM UTC, it's really representing 5:00 PM Eastern time.)
- The Air Quality Health Index (AQHI) replaced the Air Quality Index (AQI) on June 24, 2015. Numbers from before this time are not comparable to those after this time, even though they're in the same database.
Instances of data loss
- 20 Aug 2013 - 13 Oct 2013: Waterloo Crime data stopped scraping. WRPS redesigned their website; needed to update the scraper.
- 13-15 July 2013: Data loss for all charts. Unknown cause.
- 17-18 December 2012: An unexpected server failure and resulting emergency maintenance on the marrs server caused the site to go offline for about 36 hours
- 28 November - 18 December 2012: A change to the way data was scraped resulted in an interruption of data collection for the Electricity Sources chart
Air Quality
- Download
- samnabi_rwi_aqi.csv [6.9 mb]
- Columns
- Timestamp, Date, Average AQI, Barrie, Belleville, Brampton, Brantford, Burlington, Chatham, Cornwall, Dorset, Grand Bend, Guelph, Hamilton Downtown, Hamilton Mountain, Hamilton West, Kingston, Kitchener, London, Mississauga, Morrisburg, Newmarket, North Bay, Oakville, Oshawa, Ottawa Central, Ottawa Downtown, Parry Sound, Petawawa, Peterborough, Port Stanley, Sarnia, Sault Ste Marie, St Catharines, Sudbury, Thunder Bay, Tiverton, Toronto Downtown, Toronto East, Toronto North, Toronto West, Windsor Downtown, Windsor West
- Start date
- 2012-01-28 20:00:00
- End date
- 2016-06-29 09:00:00
- Source
- http://www.airqualityontario.com/aqhi/index.php
Waterloo Region Crime
- Download
- samnabi_rwi_crime_waterloo.csv [169 kb]
- Columns
- Motor Vehicle Collision Damage, Disturbance, Traffic, Theft Under 5k, Theft Over 5k, Property Damage, Motor Vehicle Collision Injury, Timestamp, Date, Total, Robbery, Assault, Breaking and Entering, Drugs, Threatening, Offensive Weapon, Motor Vehicle Theft, Prostitution, Fire, Other
- Start date
- 2012-02-17
- End date
- 2016-10-11
- Source
- http://www.wrps.on.ca/mediacentre/dailycallsummary.html
Electricity Demand
- Download
- samnabi_rwi_electricity_demand.csv [6.8 mb]
- Columns
- Timestamp, Date, Total Market Demand, Ontario Demand
- Start date
- 2002-05-01 01:00:00
- End date
- 2016-10-11 22:00:00
- Source
- http://www.ieso.ca/imoweb/market/mi5_includes/maag/maag_xml/maag.xml
Electricity Type
- Download
- samnabi_rwi_electricity_type.csv [3.1 mb]
- Columns
- Timestamp, Date, Total, Nuclear, Hydro, Natural Gas, Coal, Wind, Other
- Start date
- 2011-12-21 13:00:00
- End date
- 2016-10-11 22:00:00
- Source
- http://www.ieso.ca/
Lake Ontario Water Levels
- Download
- samnabi_rwi_lakeontario.csv [2.9 mb]
- Columns
- Timestamp, Date, Port Weller, Burlington, Toronto, Cobourg, Kingston
- Start date
- 2012-04-01 01:00:00
- End date
- 2016-09-16 12:00:00
- Source
- http://www.great-lakes.net/envt/water/levels/levels-cur/ontwlc.html
Ontario Highway Traffic Disruptions
- Download
- samnabi_rwi_traffic.csv [3.1 mb]
- Columns
- Timestamp, Date, Total, Niagara, Hamilton, Halton, Peel, Toronto, York, Durham, Simcoe, Bancroft, Ontario East, Ottawa
- Start date
- 2012-08-06 17:00:00
- End date
- 2016-10-12 00:00:00
- Source
- http://www.mto.gov.on.ca/english/traveller/trip/traffic_reports.shtml