Description
PyData London 2016
Starting in the Q4, 2015, I wrote the financials data pipeline that collates ~200 data points and calculates ~300 metrics for ~80M account filings from ~11M private companies.
As I write, this is in production: http://bit.ly/1T3CzDG, http://bit.ly/1Q8iBBq.
I used Python, Spark and loads of good fortune to make this. I would like to share my journey with the PyData community - purely to give something back, as I have learned so much out of the meetups.
My talk would include takeaways, patterns, anti-patterns, mistakes and big mistakes that I made and learned from. I think this will be very useful for beginner-intermediate data wranglers.
Slides available here: https://github.com/alixedi/PyData2016/blob/master/Enhanced%20Financials.pdf