Description
In the SciPy community, HDF5 has rapidly emerged as the de facto technology for storing and sharing large volumes of numerical data. There has also been increased interest in the SciPy community with using cloud based computing to tackle challenging computational requirements. With the Highly Scalable Data Service (HSDS), a Python-based web-service, HDF5 data can now be read and written to object-based storage (e.g. AWS S3) and accessed over http using a RESTful api. As a web service this opens up many avenues to utilize HDF in ways that would have been difficult to achieve previously. In addition to the service itself, h5pyd is a package written specifically for Python clients that enables access to the HDF REST API in a way that is compatible with the popular h5py package.