Contribute Media
A thank you to everyone who makes this possible: Read More

"Enhancing Civics Data with Elasticsearch, pandas and Eland" - Jay Miller (PyCascades 2021)


Jay Miller

After discovering that many major cities in the United States have open data projects that allow for a level of transparency. I was happy to discover that my own city had data available but quickly realized that the data had many issues that would make it difficult to analyze as-is.

In this talk, I will show how I was able to consolidate "Calls for Service" to the San Diego Police Department since 2015. This was done by importing the city-provided CSV files into Pandas DataFrames and making modifications to standardize data.

I will also convert approximate location data to a geo_point using Public geoJSON data and the filter capabilities in Elasticsearch.

Lastly, I will show how you can bulk upload the data into and out of Elasticsearch via eland so that it can be visualized in Kibana or a public graphing platform and how I automated this process to check for updates.

After three amazing in-person conferences, this time we're moving PyCascades online.

PyCascades is a regional PyCon in the Pacific Northwest, celebrating the west coast Python developer and user community. Our organizing team includes members of the Vancouver, Seattle, and Portland Python user groups.

Videos are released as CC BY-NC-SA 4.0.

Produced by Next Day Video Australia:

#pycascades #pycon #python #conference

Sat Feb 20 11:45:00 2021 at Prerecorded Talks

Improve this page