Description
In the last 40 years over a petabyte of publicly available earth observation imagery has been produced. In the near future, many petabytes of imagery per year will become available from a combination of public satellite missions and private satellite constellations. At the same time, commercial cloud providers are competing to provide the lowest cost alternative to on-premise compute capabilities. By combining the dramatic rise in available imagery with low cost of high performance storage, network, and compute capabilities, we have a unique opportunity to combine analysis techniques from remote sensing, machine learning algorithms, and scalable compute infrastructure. Combined, they allow for global scale investigations into how our planet is changing.
Here we will report on how we leverage the commercial cloud to generate a tiled spatio-temporal mosaic of the Earth and how it enables fast iteration for the development of both traditional model based predictions and machine learning algorithms. As part of our effort, we have processed, in less than 24 hours, over a petabyte of compressed raw data from the combination of the US Landsat and MODIS programs, totalling nearly 3 petapixels. We will detail the challenges and benefits to moving from traditional remote sensing workbenches to the commercial cloud, with particular emphasis on the benefits for researchers.