Description
There are audio issues with this video that cannot be fixed. We recommend listening to the tutorial without headphones to minimize the buzzing sound.
Tutorial information may be found at https://scipy2017.scipy.org/ehome/220975/493423/
Data Science and Machine learning have been synonymous with languages like Python. Libraries like numpy and Pandas have become the de facto standard when working with data. The DataFrame object provided by Pandas gives us the ability to work with heterogeneous unstructured data that is commonly used in "real world" data.
New learners are often drawn to Python and Pandas because of all the different and exciting types of models and insights the language can do and provide, but are awestruck when faced with the initial learning curve.
This tutorial aims to guide the learner from using spreadsheets to using the Pandas DataFrame. Not only does moving to a programming language allow the user to have a more reproducible workflow, but as datasets get larger, some cannot even be opened in a spreadsheet program. The goal is to have an absolute beginner proficient enough with Pandas that they can start working with data in Python.
We will cover how to load and view our data. Then, some basic methods to do quick visualizations of our data for exploratory data analysis. We will then work on combining and working multiple datasets (concatenating and merging), and introduce what Dr. Hadley Wickham has coined "tidy data". Tidy data is an important concept because the process of tidying data will fix a host of data problems that are needed to perform analysis. We then cover functions and applying methods to our data with a focus on data cleaning, and how we can use the concept of split-apply-combine (groupby) to summarize or reduce our data.
Finally, we cover the basics of string manipulation and how to use it to clean data before briefly covering the role of Pandas in analysis packages such as scikit learn. The tutorial will with a fitted model.
The goal is to get people familiar with Python and Pandas so they can learn and explore many other parts of the Python ecosystem.