The following discussion is an excerpt from our December 2016 submission to the World Bank Big Data Innovation Challenge. We trust it provides some more detail around the mission of ChennaiFloodManagement.org.

Feedback is our fuel! We'd love to hear your thoughts, please get in touch at david.kennewell@hydrata.com

Executive Summary

ChennaiFloodManagement.org provides an online collaboration platform for citizens, engineers, property developers and Government organisations to develop coordinated, sustainable solutions to the city’s flood challenges.

We recognise that sustainably developing a city to manage flood risk requires a transparent, data-driven and adaptive approach. We make big data easily available, provide cloud-based open source flood modelling tools, and a transparent reporting portal for communication of results.

This allows us to break down the contractual silos that can limit individual projects. It enables mobile, social, satellite and internet-of-things data sources to integrate into the “best available” data for transparent, resilient, sophisticated decision making.

 

What problem(s) are you helping to solve?

Flooding is a global challenge causing significant economic damage and loss of life each year. As urban populations increase, and our climate changes, the economic impacts of flooding are increasing significantly.

We selected Chennai as the city to prototype our collaborative flood management solutions because it has a significant flood management challenge that is typical of cities around the world. Flood impacts occur in Chennai because:

  • Buildings get built in the wrong place. Often city planners don’t have access to accurate information, or a lack of transparency has created conditions where corrupt approvals are possible.
  • Watersheds change. Climate changes. Engineers constantly need to adapt drainage systems and management measures to keep up, whilst balancing this with other city development pressures
  • Residents are sometimes unaware and unprepared. A lack of publicly available flood mapping, evacuation planning and response protocols means flood impacts can be exacerbated.
  • Insurance is unaffordable or unavailable. This limits the ability of business and populations to recover from an event.
  • Managing all this change is difficult in an environment where stakeholders (particularly planners, design engineers, construction companies, property developers and government) are limited to contractual arrangements with one other party.

We aim to address this by providing a transparent, defensible and data-rich environment for these calculations to be undertaken online, reducing approvals risk and cost (where appropriate) and saving developers money on expensive, often conflicting consulting engineers advice.

 

What is your proposed solution/prototype and approach? What research indicates that your solution is viable in addressing this problem?

ChennaiFloodManagement.org is a web application made up of three components. It currently runs on Amazon Web Services’ cloud infrastructure.

Component 1 - A content management system for geospatial, social and IoT data.

We use the open-source Geonode geospatial content management system to ingest, track and integrate our geospatial data.

This software has benefited from significant World Bank contributions over the years, and provides a well-developed and tested framework. This is where we store and serve data over the internet using Geoserver to allow ourselves and other users access to consistent, up-to-date datasets.

Component 2 - A calculation platform to turn this data into online, agile flood maps and flood warning systems. We use the open-source hydraulic model ANUGA to calculate flood maps from the terrain and rainfall inputs. The hydraulic model code is open source, and available online at http://github.com/Hydrata/ChennaiFloodModel. Contributions to improve the accuracy of the model can be made from anyone around the world, using GitHub’s collaboration tooling.

ANUGA is connected to Geonode through a custom developed web interface, allowing users to run the ANUGA hydraulic models on Hydrata’s Amazon computing cloud. Results are stored back in Geonode.

Component 3 - A publication platform to publish and update results.

We use the open-source Django CMS content management system to produce online, collaborative text reporting. We integrate the geospatial data inputs and flood result outputs into these reports using a custom-written plugin for Django-CMS, available as open source on Hydrata’s GitHub page here - https://github.com/Hydrata/geonode_cms_map. This platform allows us to coordinate all the work, data and results into a single Flood Management Plan, available here:

http://chennaifloodmanagement.org/en/reports/chennai-flood-management-plan/

 

Other relevant technical details

(Apologies for the jargon in this paragraph...) The platform is build using a DevOps approach, and new instances of the entire system architecture can be automatically created within an hour. We maintain our source code in GitHub, and create our servers using Vagrant. We utilise AWS tools to provide automated system data backup and restore capability.

Our incorporation of Django-CMS means we can create a template of our application to roll out to any city in the world, with site-specific data and content uploaded and edited by users in that city. This provides an inherently scalable platform where we can create custom sites for many cities, but still roll out software updates to all these sites in a consistent, maintainable, reproducible manner.

 

Viability

The long-term financial viability of our solution depends on providing value for those who have a commercial benefit from accurate flood data. Potential commercial funding sources include

  • governments purchasing catchment-management-as-a-service,
  • engineering consultancies purchasing an infrastructure design and environmental impact assessment service (for things like flood impact assessment of proposed road embankments or property developments)
  • claims assessment support for insurance companies
  • a marketplace to match contributing engineers/hydrologists/scientists/planners with infrastructure developers
  • and (an interesting one we weren’t expecting when we started), as a corruption management solution.

Experiments to understand and validate the market requirements for each of these business models are ongoing.

 

What stage of product development is your solution in?

ChennaiFloodManagement.org has been operating as a geospatial data store and reporting platform since June 2016. We have hosted reports and datasets from Non-Government Organisations, including

  • Citizen Consumer and Civic Actions Groups “Mapping floods to assess risks and vulnerabilities in the Chennai Metropolitan Area”,
  • Data from Arappor Iyakkam’s “Audit of Chennai Waterways – A Citizen’s Report”
  • Civil infrastructure data from data.gov.in

One theme consistently emerged from these data management exercises. To gain sustainable traction for the project, we needed a form of online flood mapping. Flood maps are, by far, the most requested feature.

Based on this feedback we worked to incorporate a hydraulic model into our application, with the aim of providing flood maps that can be calculated on-demand, and easily updated as new information, such as stormwater infrastructure surveys, new building footprints, landuse data or terrain data, becomes available. We published our first low-resolution outputs from this model on October 5, 2016.

We are currently in the process of collecting the additional terrain, rainfall and infrastructure data required to improve the accuracy and resolution of this flood map product.

Shortly, we will push two more features into a validation phase:

  • Citizen-powered rain gauge kits. These gauges will connect to our hydraulic model to improve our understanding of rainfall temporal variation.
  • A twitter listening service to help calibrate our flood maps with tweeted, georeferenced photos.

Development of ChennaiFloodManagement.org uses an agile project management approach, together with lean start-up techniques to validate our assumptions.

 

How does your solution or prototype utilize Big Data (it could be a combination of traditional and big data)? How does it address privacy concerns (i.e. how does your solution protect personally identifiable data or commercially sensitive data, if applicable)?

Our solution currently utilises

  • global SRTM digital terrain models,
  • crowdsourced civil infrastructure survey data from a mobile application,
  • land use planning data from Open Street Map,
  • georeferenced and digitised historic land use maps,
  • satellite observations of flooded areas from India’s National Remote Sensing Centre
  • geospatially tagged and timestamped twitter photographs of flood elevations and velocities
  • Global Surface summary of Day (GSOD) data from the National Climatic Data Centre, hosted on AWS. This data will be combined with our rain gauge data to disaggregate the daily rainfall to a more useful temporal resolution (say 5 minute or 30 minute intervals), and improve the calibration of radar rainfall records.

Our computing infrastructure and flood models are built and ready to ingest remote sensing products such as

  • satellite rainfall observations
  • radar rainfall observations
  • LIDAR terrain data
  • Cartosat-2 orthocorrected terrain data at 1m resolution

We are currently working to negotiate appropriate access to these valuable sources.

Our flood mapping solution uses the ANUGA finite-element solution to the shallow-water equations to produce flood mapping. The model outputs are at the same resolution as the terrain data we feed in. However, the model accuracy is very dependent on having high-resolution terrain data that recognises the ground features that can control flooding in an urban environment – roads, buildings, channels and gutters.

The ANUGA code can be run on multiple AWS computers, in parallel, to reduce run times for high-resolutions models from weeks to down to hours.

 

In what ways is your solution or prototype novel or unique?

ChennaiFloodManagement.org has been designed from the ground up to uniquely play a role that no existing organisation can fully cover. Existing flood management solutions rely on a professional services provider entering a contract with a client. This creates silos, both culturally and contractually, and results in a data environment where:

  • Government generally can’t store and publish citizen generated data or datasets developed by consulting engineers.
  • Engineering consultancies generally can’t publish any data except their own.
  • Property developers produce publicly important data to support land development projects, however are often not motivated to publish this.
  • Citizens generally don’t have the technical skills or resources to turn their crowdsourced data into effective decision making tools for use by government or property developers.

By acting as an independent third party, ChennaiFloodManagement.org can play a facilitator role and add value to these various data sources, whilst still requiring the end-user to be responsible for their use-case.

Because our solutions are provided through a web-application, we can scale up our methodologies in a manner that a traditional engineering consultancy could not. By keeping our input data, calculations and outputs online and in version control, our solutions can evolve over time into accurate, trusted assets for a city.

This contrasts with the traditional static PDF style reporting from a traditional consultancy service provider. These PDF documents are difficult to build upon, and expensive to update, particularly once the service provider’s contract is finished.

 

What are three key things that are critical to implementing and bringing your solution to scale?

 

Partnerships with government

It is vitally important that the information we manage is coordinated with, and beneficial to, government at all scales. We aim to supplement government efforts with results and data that would not typically be collected by government, and develop an effective synergistic approach.

We have significant interest from highly qualified volunteers around Chennai, India and the world, who would like to donate their time and expertise. Feedback from these volunteers is that we need to show on-ground results and action by government to make their contribution worthwhile.

 

Access to data with appropriate licencing

A key factor for this project’s success is access to quality terrain and rainfall data, largely held by Indian government agencies. Open data policies in India are becoming stronger, however it is currently a time-consuming challenge to access the data sets we to provide value. By coordinating with Government and demonstrating the value and benefits of this project, we anticipate these access issues will decrease.

 

Marketing and training

Citizen data contributions such as those through Twitter, and potential citizen data uses, such as using our flood maps to inform a property purchase decision, require us to understand exactly how citizens will interact with our project. We would like to run a series of workshops throughout Chennai. These workshops would allow us to evolve our product in an appropriate manner and ensure we are providing the right advice in the right format. We are currently seeking a funding source for these workshops.