This tutorial was presented by Kevin Markham at PyCon on May 10, 2018.

The tutorial code is available as a Jupyter notebook. The notebook includes 4 additional exercises that were not covered during the tutorial.

1. Introducing the dataset (19:40)
2. Removing columns (6:27)
3. Comparing groups (8:42)
4. Examining relationships (8:44)
5. Handling missing values (5:02)
6. Using string methods (5:55)
7. Combining dates and times (9:11)
8. Plotting a time series (8:48)
9. Creating useful plots (8:47)
10. Fixing bad data (16:31)

The pandas library is a powerful tool for multiple phases of the data science workflow, including data cleaning, visualization, and exploratory data analysis. However, proper data science requires careful coding, and pandas will not stop you from creating misleading plots, drawing incorrect conclusions, ignoring relevant data, including misleading data, or executing incorrect calculations.

In this tutorial, you'll perform a variety of data science tasks on a handful of real-world datasets using pandas. With each task, you'll learn how to avoid either a pandas pitfall or a data science pitfall. By the end of the tutorial, you'll be more confident that you're using pandas for good rather than evil!

You will get the most out of this tutorial if you are an intermediate pandas user, since the tutorial will not cover pandas basics. If you are new to pandas or just need a refresher, I recommend watching some videos from my free pandas course. Alternatively, you can review all of the code from my pandas course in this Jupyter notebook.

• police.csv is the Rhode Island dataset from the Stanford Open Policing Project, made available under the Open Data Commons Attribution License.
• ted.csv is the TED Talks dataset from Kaggle Datasets, made available under the CC BY-NC-SA 4.0 license.

Here are three options that will work equally well:

• If you know how to use git, you can click the green button above and clone the repository.
• If you know how to open a ZIP file, you can click the green button above and download the repository.
• If you want to download the files individually, right click on these links and select "Save As": police.csv, ted.csv, tutorial.ipynb.

1. Move the CSV files into your working directory. (This is usually the directory where you create Python scripts or notebooks.)

2. Open the Python environment of your choice.

3. If you're using Jupyter notebook, run the following code:

   import pandas as pd
   import matplotlib.pyplot as plt
   %matplotlib inline
   ri = pd.read_csv('police.csv')
   ted = pd.read_csv('ted.csv')
   ri.driver_age.plot()

4. If you're using any other Python environment, run the following code:

   import pandas as pd
   import matplotlib.pyplot as plt
   ri = pd.read_csv('police.csv')
   ted = pd.read_csv('ted.csv')
   ri.driver_age.plot()
   plt.show()

If you don't get any error messages, and a plot appears on your screen, then it's very likely that pandas and matplotlib are installed correctly.

Kevin Markham is the founder of Data School, an online school for learning data science with Python. He is passionate about teaching data science to people who are new to the field, regardless of their educational and professional backgrounds. Previously, Kevin was the lead data science instructor for General Assembly in Washington, DC. Currently, he teaches machine learning and data analysis to over 10,000 students each month through the Data School YouTube channel. He has a degree in Computer Engineering from Vanderbilt University and lives in Asheville, North Carolina with his wife and son.

Sure! You can email kevin@dataschool.io.