Preparation

This session is mostly concerned with pandas: representing tabular data in DataFrames and Series, loading and saving data, inspecting and summarizing it, and cleaning it with a pipeline of pure functions.

To prepare, work through the following materials:

1. Accessing the file system in Python

   • File paths with pathlib

2. Basic Pandas: Practical commands

   • What is (modern) pandas

   • DataFrames and Series

   • Datatypes

   • Loading and saving data

   • Inspecting and summarizing data

   • Creating variables

3. Functional data cleaning

   • Data management: Definitions and example

   • Functional data cleaning: The how: the three rules we will apply throughout: start with an empty DataFrame, touch every variable just once, touch it with a pure function

   • Functional data cleaning: The why

Optional material

• Setting and renaming columns and indices: the functional approach makes most renaming unnecessary because you assign each column of a new DataFrame exactly once, but it is useful to recognise the patterns

If the functional approach feels unfamiliar and you would like to see how it connects to the way you have cleaned data so far, but in Python, watch Imperative data cleaning.