Description
The pandas library represents a very efficient and convenient tool for data manipulation, but sometimes hides unexpected pitfalls which can arise in various and sometimes unintelligible ways.
By briefly referring to some aspects of the implementation, I will review specific situations in which a change of approach can make code based on pandas more robust, or more performant.
Some examples:
- inefficient indexing
- multiple dtypes and efficiency
- implicit type casting
- HDF5 storage overhead
- GroupBy.apply()... when you don't actually need it
UPDATE: slides and materials can be found at http://pietrobattiston.it/python:pycon#europython_rimini_july_2017