Summary
Understanding environmental and climate change requires data fusion, format conversions, processing and visualization to gain insight into the data. Our open source scientific Python and JavaScript based tools makes it easy to manipulate geo-spatial and climate data, create and execute workflows, and produce visualizations over the web for scientific and decision making tools.
Description
The impact of climate change will resonate through a broad range of fields including public health, infrastructure, water resources, and many others. Long-term coordinated planning, funding, and action are required for climate change adaptation and mitigation. Unfortunately, widespread use of climate data (simulated and observed) in non-climate science communities is impeded by factors such as large data size, lack of adequate metadata, poor documentation, and lack of sufficient computational and visualization resources. Additionally, working with climate data in its native format is not ideal for all types of analyses and use cases often requiring technical skills (and software) unnecessary to work with other geospatial data formats.
We present open source tools developed as part of ClimatePipes and OpenClimateGIS to address many of these challenges by creating an open source platform that provides state-of-the-art user-friendly data access, processing, analysis, and visualization for climate and other relevant geospatial datasets making the climate and other geospatial data available to non-researchers, decision-makers, and other stakeholders.
The overarching goals are:
- Enable users to explore real-world questions related to environment and climate change.
- Provide tools for data access, geo-processing, analysis, and visualization.
- Facilitate collaboration by enabling users to share datasets, workflows, and visualization.
Some of the key technical features include
- Support for multiprocessing for large datasets using Python-celery distributed task queuing system
- Generic iterators allowing data to be streamed to arbitrary formats (relatively) easily (e.g. ESRI Shapefile, CSV, keyed ESRI Shapefile, CSV, NetCDF)
- NumPy based array computations allowing calculations such as monthly means or heat indices optionally on temporally grouped data slices
- Decorators to expose existing Python API as a RESTful API
- Simple to use, lightweight Web-framework and JavaScript libraries for analyzing and visualizing geospatial datasets using D3 and WebGL.