lecture for today
This commit is contained in:
Andrew Straw
parent
cae59ac414
commit
153d998716
208
lectures/lecture-06/1 - Classes.ipynb
Normal file
208
lectures/lecture-06/1 - Classes.ipynb
Normal file
|
|
@ -0,0 +1,208 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Reminder\n",
|
||||
"\n",
|
||||
"Do you need to register for \"Studienleistung\" in HisInOne?"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Speedrun: command line treasure hunt (schnitzeljagd)\n",
|
||||
"\n",
|
||||
"We will do a quick speedrun of the schnitzeljagd exercise starting with `ssh your-user-name@python-course.strawlab.org`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Classes\n",
|
||||
"\n",
|
||||
"Here is an example of a simple class:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class MyClass:\n",
|
||||
" def my_method(self):\n",
|
||||
" print(\"This was printed from a method inside the class.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We'll discuss the \"self\" parameter below. First, let's create an instance of this class:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"x = MyClass()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"x.my_method()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Why classes?\n",
|
||||
"\n",
|
||||
"The great benefit of classes is that they keep data and functions organized togther. The functions within a class are called \"methods\" and always take the first parameter, \"self\".\n",
|
||||
"\n",
|
||||
"With small pieces of code, this is not so important, but as the size of software grows larger, this is handy for keeping things organized. Most Python libraries make extensive use of classes.\n",
|
||||
"\n",
|
||||
"A class is like a template and multiple instances of this template can be created.\n",
|
||||
"\n",
|
||||
"## Creating an instance of a class\n",
|
||||
"\n",
|
||||
"Class definitions can have a special method caled `__init__()`. This initialization method, also called constructor, is called when an instance of a class is created. It is used to store variables and perform other setup."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class MyClass:\n",
|
||||
" def __init__(self):\n",
|
||||
" self.my_variable = \"foo\"\n",
|
||||
" def my_method(self):\n",
|
||||
" print(\"My variable is {}.\".format(self.my_variable))\n",
|
||||
" def set_var(self,new_value):\n",
|
||||
" self.my_variable = new_value\n",
|
||||
" \n",
|
||||
"x = MyClass()\n",
|
||||
"x.my_method()\n",
|
||||
"\n",
|
||||
"x.set_var(\"bar\")\n",
|
||||
"x.my_method()\n",
|
||||
"\n",
|
||||
"x.my_variable = \"zzz\"\n",
|
||||
"x.my_method()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## What is this `self` thing?\n",
|
||||
"\n",
|
||||
"As mentioned, `self` is always the first argument of any method. It contains the data (variables) for a specific instance of a class."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"class Insect:\n",
|
||||
" def __init__(self, species_name, mass, mass_units='milligrams'):\n",
|
||||
" self.species_name = species_name\n",
|
||||
" self.mass = mass\n",
|
||||
" self.mass_units = mass_units\n",
|
||||
" def print_description(self):\n",
|
||||
" print(\"The insect (species {}) has a mass of {} {}.\".format(\n",
|
||||
" self.species_name, self.mass, self.mass_units))\n",
|
||||
" def eat(self,amount):\n",
|
||||
" # (This could alternatively be done with `self.mass += amount`.)\n",
|
||||
" self.mass = self.mass + amount\n",
|
||||
" def exercise(self,amount):\n",
|
||||
" self.mass = self.mass - amount\n",
|
||||
" \n",
|
||||
"x = Insect(\"Bombus terrestris\", 200)\n",
|
||||
"x.print_description()\n",
|
||||
"x.eat(10)\n",
|
||||
"x.exercise(5)\n",
|
||||
"x.print_description()\n",
|
||||
"\n",
|
||||
"y = Insect(\"Apis mellifera\", 100)\n",
|
||||
"y.print_description()\n",
|
||||
"# y.eat(10)\n",
|
||||
"# y.print_description()\n",
|
||||
"\n",
|
||||
"z = Insect(\"Tarantula gigantus\", 10, mass_units=\"grams\") # yes, it's not really an insect...\n",
|
||||
"z.print_description()\n",
|
||||
"\n",
|
||||
"# print(x.species_name,x.mass)\n",
|
||||
"# print(y.species_name,y.mass)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You can see we access the value of a variable within a method using `self.variable_name`. Outside the function, we can also access the \"instance variables\" but we need to use the instance name and a dot:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(x.species_name)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(y.species_name)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As mentioned, most Python libraries make use of classes to organize their code. All objects in Python act a lot like instances of classes. For example, the string methods `.strip()` and `.format()` could be defined on a hypothetical `String` class exactly like the methods above. (In reality, they are likely implemented differently for performance reasons.)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.10"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
556
lectures/lecture-06/2 - Pandas.ipynb
Normal file
556
lectures/lecture-06/2 - Pandas.ipynb
Normal file
|
|
@ -0,0 +1,556 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Pandas\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\"high-performance, easy-to-use data structures and data analysis tools\" https://pandas.pydata.org/\n",
|
||||
"\n",
|
||||
"Pandas is typically imported as `pd`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import pandas as pd"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# We will start with the Iris dataset from a previous lecture\n",
|
||||
"iris_dataset = {'sepal length (cm)': [5.1, 4.9, 4.7, 4.6, 5.0, 5.4, 4.6, 5.0, 4.4, 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7, 5.4, 5.1, 5.7, 5.1, 5.4, 5.1, 4.6, 5.1, 4.8, 5.0, 5.0, 5.2, 5.2, 4.7, 4.8, 5.4, 5.2, 5.5, 4.9, 5.0, 5.5, 4.9, 4.4, 5.1, 5.0, 4.5, 4.4, 5.0, 5.1, 4.8, 5.1, 4.6, 5.3, 5.0, 7.0, 6.4, 6.9, 5.5, 6.5, 5.7, 6.3, 4.9, 6.6, 5.2, 5.0, 5.9, 6.0, 6.1, 5.6, 6.7, 5.6, 5.8, 6.2, 5.6, 5.9, 6.1, 6.3, 6.1, 6.4, 6.6, 6.8, 6.7, 6.0, 5.7, 5.5, 5.5, 5.8, 6.0, 5.4, 6.0, 6.7, 6.3, 5.6, 5.5, 5.5, 6.1, 5.8, 5.0, 5.6, 5.7, 5.7, 6.2, 5.1, 5.7, 6.3, 5.8, 7.1, 6.3, 6.5, 7.6, 4.9, 7.3, 6.7, 7.2, 6.5, 6.4, 6.8, 5.7, 5.8, 6.4, 6.5, 7.7, 7.7, 6.0, 6.9, 5.6, 7.7, 6.3, 6.7, 7.2, 6.2, 6.1, 6.4, 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 6.3, 6.4, 6.0, 6.9, 6.7, 6.9, 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9], 'sepal width (cm)': [3.5, 3.0, 3.2, 3.1, 3.6, 3.9, 3.4, 3.4, 2.9, 3.1, 3.7, 3.4, 3.0, 3.0, 4.0, 4.4, 3.9, 3.5, 3.8, 3.8, 3.4, 3.7, 3.6, 3.3, 3.4, 3.0, 3.4, 3.5, 3.4, 3.2, 3.1, 3.4, 4.1, 4.2, 3.1, 3.2, 3.5, 3.6, 3.0, 3.4, 3.5, 2.3, 3.2, 3.5, 3.8, 3.0, 3.8, 3.2, 3.7, 3.3, 3.2, 3.2, 3.1, 2.3, 2.8, 2.8, 3.3, 2.4, 2.9, 2.7, 2.0, 3.0, 2.2, 2.9, 2.9, 3.1, 3.0, 2.7, 2.2, 2.5, 3.2, 2.8, 2.5, 2.8, 2.9, 3.0, 2.8, 3.0, 2.9, 2.6, 2.4, 2.4, 2.7, 2.7, 3.0, 3.4, 3.1, 2.3, 3.0, 2.5, 2.6, 3.0, 2.6, 2.3, 2.7, 3.0, 2.9, 2.9, 2.5, 2.8, 3.3, 2.7, 3.0, 2.9, 3.0, 3.0, 2.5, 2.9, 2.5, 3.6, 3.2, 2.7, 3.0, 2.5, 2.8, 3.2, 3.0, 3.8, 2.6, 2.2, 3.2, 2.8, 2.8, 2.7, 3.3, 3.2, 2.8, 3.0, 2.8, 3.0, 2.8, 3.8, 2.8, 2.8, 2.6, 3.0, 3.4, 3.1, 3.0, 3.1, 3.1, 3.1, 2.7, 3.2, 3.3, 3.0, 2.5, 3.0, 3.4, 3.0], 'petal length (cm)': [1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1.5, 1.6, 1.4, 1.1, 1.2, 1.5, 1.3, 1.4, 1.7, 1.5, 1.7, 1.5, 1.0, 1.7, 1.9, 1.6, 1.6, 1.5, 1.4, 1.6, 1.6, 1.5, 1.5, 1.4, 1.5, 1.2, 1.3, 1.4, 1.3, 1.5, 1.3, 1.3, 1.3, 1.6, 1.9, 1.4, 1.6, 1.4, 1.5, 1.4, 4.7, 4.5, 4.9, 4.0, 4.6, 4.5, 4.7, 3.3, 4.6, 3.9, 3.5, 4.2, 4.0, 4.7, 3.6, 4.4, 4.5, 4.1, 4.5, 3.9, 4.8, 4.0, 4.9, 4.7, 4.3, 4.4, 4.8, 5.0, 4.5, 3.5, 3.8, 3.7, 3.9, 5.1, 4.5, 4.5, 4.7, 4.4, 4.1, 4.0, 4.4, 4.6, 4.0, 3.3, 4.2, 4.2, 4.2, 4.3, 3.0, 4.1, 6.0, 5.1, 5.9, 5.6, 5.8, 6.6, 4.5, 6.3, 5.8, 6.1, 5.1, 5.3, 5.5, 5.0, 5.1, 5.3, 5.5, 6.7, 6.9, 5.0, 5.7, 4.9, 6.7, 4.9, 5.7, 6.0, 4.8, 4.9, 5.6, 5.8, 6.1, 6.4, 5.6, 5.1, 5.6, 6.1, 5.6, 5.5, 4.8, 5.4, 5.6, 5.1, 5.1, 5.9, 5.7, 5.2, 5.0, 5.2, 5.4, 5.1], 'petal width (cm)': [0.2, 0.2, 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.4, 0.4, 0.3, 0.3, 0.3, 0.2, 0.4, 0.2, 0.5, 0.2, 0.2, 0.4, 0.2, 0.2, 0.2, 0.2, 0.4, 0.1, 0.2, 0.2, 0.2, 0.2, 0.1, 0.2, 0.2, 0.3, 0.3, 0.2, 0.6, 0.4, 0.3, 0.2, 0.2, 0.2, 0.2, 1.4, 1.5, 1.5, 1.3, 1.5, 1.3, 1.6, 1.0, 1.3, 1.4, 1.0, 1.5, 1.0, 1.4, 1.3, 1.4, 1.5, 1.0, 1.5, 1.1, 1.8, 1.3, 1.5, 1.2, 1.3, 1.4, 1.4, 1.7, 1.5, 1.0, 1.1, 1.0, 1.2, 1.6, 1.5, 1.6, 1.5, 1.3, 1.3, 1.3, 1.2, 1.4, 1.2, 1.0, 1.3, 1.2, 1.3, 1.3, 1.1, 1.3, 2.5, 1.9, 2.1, 1.8, 2.2, 2.1, 1.7, 1.8, 1.8, 2.5, 2.0, 1.9, 2.1, 2.0, 2.4, 2.3, 1.8, 2.2, 2.3, 1.5, 2.3, 2.0, 2.0, 1.8, 2.1, 1.8, 1.8, 1.8, 2.1, 1.6, 1.9, 2.0, 2.2, 1.5, 1.4, 2.3, 2.4, 1.8, 1.8, 2.1, 2.4, 2.3, 1.9, 2.3, 2.5, 2.3, 1.9, 2.0, 2.3, 1.8], 'species': ['setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica']}"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# pandas `DataFrame`\n",
|
||||
"\n",
|
||||
"The primary interest in pandas is the `DataFrame`. A `DataFrame` is a type, conceptually related to a numpy array, for containing large amounts of data and operating efficiently on it. With `DataFrame`s, however, there is typically more structure. A `DataFrame` is always two dimensional, with every element in a column having the same data type. There are multiple columns, each with a name and potentially different datatypes. The easiest way to think about a `DataFrame` is like a well-organized spreadsheet. Indeed, `DataFrame`s are great for doing the kind of calculations you might do in spreadsheets.\n",
|
||||
"\n",
|
||||
"## Creation\n",
|
||||
"\n",
|
||||
"One way to create a pandas `DataFrame` is by using its constructor, `DataFrame()`. If provided one argument, a dictionary, it will create a new `DataFrame` instance with a column from each item in the dict. The dict key becomes the column name and the dict value (a Python sequence) becomes are the column data values. Pandas will infer the datatype for the column. It is required that the length of all sequences in the dict are identical so that each column in the `DataFrame` has the same length.\n",
|
||||
"\n",
|
||||
"As an example, let's load our Iris dataset."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df = pd.DataFrame(iris_dataset)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `head()` and `tail()` methods both return dataframes which are a subset of the original dataframe, with the top and bottom rows, respectively:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df.tail()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Note that jupyter and pandas work nicely together to give the nicely formatted output you see above. Here is plain `print()`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"print(df.tail())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"What's happening behind the scenes is that Pandas knows how to use the `diplay()` function from IPython."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from IPython.display import display"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"display(df.head())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# The `DataFrame.groupby()` method\n",
|
||||
"\n",
|
||||
"One of the most useful aspects of dataframes is the `groupby()` method, which returns an iterator that steps through the original dataframe by returning subsets (groups) which all have been selected based on a common value. An example will make this more clear.\n",
|
||||
"\n",
|
||||
"Here we will step through our original dataframe grouping by species. The iterator from `groupby()` returns, on each iteration, a tuple of `(group_value, group_data_frame)`. Let's look at this in action:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for species, gdf in df.groupby('species'):\n",
|
||||
" print(f\"species: Iris {species}\")\n",
|
||||
" display(gdf.head())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's take a closer look at this iteration aspect:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"my_iter = df.groupby('species')\n",
|
||||
"print(type(my_iter))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"my_iter = df.groupby('species')\n",
|
||||
"for x in my_iter:\n",
|
||||
" # species = x[0]\n",
|
||||
" # gdf = x[1]\n",
|
||||
" species, gdf = x\n",
|
||||
" # (species, gdf) = x\n",
|
||||
" print(species, len(gdf))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for species, gdf in df.groupby('species'):\n",
|
||||
" print(species, len(gdf))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for species, gdf in df.groupby('species'):\n",
|
||||
" print(f\"=============== {species} ============\")\n",
|
||||
" display(gdf)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df.describe()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"for species, gdf in df.groupby('species'):\n",
|
||||
" print(f\"=============== {species} ============\")\n",
|
||||
" display(gdf.describe())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# More about Pandas `DataFrame`s\n",
|
||||
"\n",
|
||||
"Let's get started by making a sample dataframe with fake data:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sample_df = pd.DataFrame({'number':[1,2,3,6,2,3,2,2,1,2], 'color':['blue','blue','red','red','red','blue','blue','red','green','yellow']})\n",
|
||||
"display(sample_df)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Getting a `Series` from a `DataFrame` in Pandas\n",
|
||||
"\n",
|
||||
"You can get a `Series` (a Pandas 1D array) from a dataframe column by indexing with the column name:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"colors = sample_df['color']\n",
|
||||
"display(colors)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Above we got a `Series` from a `DataFrame` column with dictionary-like get item using square brackets and a string with the name of the column. In addition to this approach, Pandas also has an ergonomic feature where columns with names that are valid Python can be used with a dot (`.`) as if they were variables (also called \"attributes\") of the `DataFrame` instance."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# This does the same thing as above because \"color\" is a valid Python attribute name.\n",
|
||||
"\n",
|
||||
"colors = sample_df.color\n",
|
||||
"display(colors)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"type(colors)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"A `Series` has many useful methods, such as `.unique()` and `.mean()`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"colors.unique()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sample_df['number'].mean()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Pandas `read_csv`\n",
|
||||
"\n",
|
||||
"[CSV files](https://en.wikipedia.org/wiki/Comma-separated_values) are a very common and very good way to save data. Pandas has a good (and fast) reader for CSV files in the `read_csv()` function."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df = pd.read_csv('iris.csv')"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df.head()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df['species'].unique()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Conditions and selecting part of the data from a `DataFrame`.\n",
|
||||
"\n",
|
||||
"Let's consider an equality condition. Let's check every row to test if the 'color' column is equal to `'blue'`:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sample_df"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sample_df['color']"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sample_df['color']=='blue'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"As usual, we can assign this result to a variable:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"condition = sample_df['color']=='blue'"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We can see the type of this result is another `Series`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"type(condition)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"One very useful thing in Pandas is to create a new `DataFrame` based on a condition from an old one. Let's make a new `DataFrame` from only the rows with a blue color:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"condition = sample_df['color']=='blue'\n",
|
||||
"blue_sample_df = sample_df[ condition ]\n",
|
||||
"display(blue_sample_df)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"This could of course be written so:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"blue_sample_df = sample_df[ sample_df['color']=='blue' ]\n",
|
||||
"display(blue_sample_df)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## A key concept in Pandas is iterating over a dataframe, grouping by values in one (or more) columns\n",
|
||||
"\n",
|
||||
"This allows doing a lot of powerful datascience work which requires nothing more than storing your data in a well-organized format. This of course has other advantages as well. Let's have a look at our `groupby()` example again:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"df = pd.read_csv('iris.csv')\n",
|
||||
"for species, gdf in df.groupby('species'):\n",
|
||||
" print(f\"=============== {species} ============\")\n",
|
||||
" display(gdf.describe())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# matplotlib + pandas + ❤️ = seaborn\n",
|
||||
"\n",
|
||||
"[Seaborn](https://seaborn.pydata.org/) is \"a Python data visualization library based on matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.\" It makes heavy use of pandas to make your life easy."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import seaborn as sns"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sns.stripplot(x=\"species\", y=\"sepal_width\", data=df);"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"sns.stripplot(x=\"species\", y=\"sepal_width\", data=pd.read_csv('iris.csv'));"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's take a minute to look at the [seaborn gallery](https://seaborn.pydata.org/examples/index.html).\n",
|
||||
"\n",
|
||||
"And while we are at it, we should not forget the [matplotlib gallery](https://matplotlib.org/stable/gallery/index.html)."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.10"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 4
|
||||
}
|
||||
151
lectures/lecture-06/iris.csv
Normal file
151
lectures/lecture-06/iris.csv
Normal file
|
|
@ -0,0 +1,151 @@
|
|||
sepal_length,sepal_width,petal_length,petal_width,species
|
||||
5.1,3.5,1.4,0.2,setosa
|
||||
4.9,3.0,1.4,0.2,setosa
|
||||
4.7,3.2,1.3,0.2,setosa
|
||||
4.6,3.1,1.5,0.2,setosa
|
||||
5.0,3.6,1.4,0.2,setosa
|
||||
5.4,3.9,1.7,0.4,setosa
|
||||
4.6,3.4,1.4,0.3,setosa
|
||||
5.0,3.4,1.5,0.2,setosa
|
||||
4.4,2.9,1.4,0.2,setosa
|
||||
4.9,3.1,1.5,0.1,setosa
|
||||
5.4,3.7,1.5,0.2,setosa
|
||||
4.8,3.4,1.6,0.2,setosa
|
||||
4.8,3.0,1.4,0.1,setosa
|
||||
4.3,3.0,1.1,0.1,setosa
|
||||
5.8,4.0,1.2,0.2,setosa
|
||||
5.7,4.4,1.5,0.4,setosa
|
||||
5.4,3.9,1.3,0.4,setosa
|
||||
5.1,3.5,1.4,0.3,setosa
|
||||
5.7,3.8,1.7,0.3,setosa
|
||||
5.1,3.8,1.5,0.3,setosa
|
||||
5.4,3.4,1.7,0.2,setosa
|
||||
5.1,3.7,1.5,0.4,setosa
|
||||
4.6,3.6,1.0,0.2,setosa
|
||||
5.1,3.3,1.7,0.5,setosa
|
||||
4.8,3.4,1.9,0.2,setosa
|
||||
5.0,3.0,1.6,0.2,setosa
|
||||
5.0,3.4,1.6,0.4,setosa
|
||||
5.2,3.5,1.5,0.2,setosa
|
||||
5.2,3.4,1.4,0.2,setosa
|
||||
4.7,3.2,1.6,0.2,setosa
|
||||
4.8,3.1,1.6,0.2,setosa
|
||||
5.4,3.4,1.5,0.4,setosa
|
||||
5.2,4.1,1.5,0.1,setosa
|
||||
5.5,4.2,1.4,0.2,setosa
|
||||
4.9,3.1,1.5,0.1,setosa
|
||||
5.0,3.2,1.2,0.2,setosa
|
||||
5.5,3.5,1.3,0.2,setosa
|
||||
4.9,3.1,1.5,0.1,setosa
|
||||
4.4,3.0,1.3,0.2,setosa
|
||||
5.1,3.4,1.5,0.2,setosa
|
||||
5.0,3.5,1.3,0.3,setosa
|
||||
4.5,2.3,1.3,0.3,setosa
|
||||
4.4,3.2,1.3,0.2,setosa
|
||||
5.0,3.5,1.6,0.6,setosa
|
||||
5.1,3.8,1.9,0.4,setosa
|
||||
4.8,3.0,1.4,0.3,setosa
|
||||
5.1,3.8,1.6,0.2,setosa
|
||||
4.6,3.2,1.4,0.2,setosa
|
||||
5.3,3.7,1.5,0.2,setosa
|
||||
5.0,3.3,1.4,0.2,setosa
|
||||
7.0,3.2,4.7,1.4,versicolor
|
||||
6.4,3.2,4.5,1.5,versicolor
|
||||
6.9,3.1,4.9,1.5,versicolor
|
||||
5.5,2.3,4.0,1.3,versicolor
|
||||
6.5,2.8,4.6,1.5,versicolor
|
||||
5.7,2.8,4.5,1.3,versicolor
|
||||
6.3,3.3,4.7,1.6,versicolor
|
||||
4.9,2.4,3.3,1.0,versicolor
|
||||
6.6,2.9,4.6,1.3,versicolor
|
||||
5.2,2.7,3.9,1.4,versicolor
|
||||
5.0,2.0,3.5,1.0,versicolor
|
||||
5.9,3.0,4.2,1.5,versicolor
|
||||
6.0,2.2,4.0,1.0,versicolor
|
||||
6.1,2.9,4.7,1.4,versicolor
|
||||
5.6,2.9,3.6,1.3,versicolor
|
||||
6.7,3.1,4.4,1.4,versicolor
|
||||
5.6,3.0,4.5,1.5,versicolor
|
||||
5.8,2.7,4.1,1.0,versicolor
|
||||
6.2,2.2,4.5,1.5,versicolor
|
||||
5.6,2.5,3.9,1.1,versicolor
|
||||
5.9,3.2,4.8,1.8,versicolor
|
||||
6.1,2.8,4.0,1.3,versicolor
|
||||
6.3,2.5,4.9,1.5,versicolor
|
||||
6.1,2.8,4.7,1.2,versicolor
|
||||
6.4,2.9,4.3,1.3,versicolor
|
||||
6.6,3.0,4.4,1.4,versicolor
|
||||
6.8,2.8,4.8,1.4,versicolor
|
||||
6.7,3.0,5.0,1.7,versicolor
|
||||
6.0,2.9,4.5,1.5,versicolor
|
||||
5.7,2.6,3.5,1.0,versicolor
|
||||
5.5,2.4,3.8,1.1,versicolor
|
||||
5.5,2.4,3.7,1.0,versicolor
|
||||
5.8,2.7,3.9,1.2,versicolor
|
||||
6.0,2.7,5.1,1.6,versicolor
|
||||
5.4,3.0,4.5,1.5,versicolor
|
||||
6.0,3.4,4.5,1.6,versicolor
|
||||
6.7,3.1,4.7,1.5,versicolor
|
||||
6.3,2.3,4.4,1.3,versicolor
|
||||
5.6,3.0,4.1,1.3,versicolor
|
||||
5.5,2.5,4.0,1.3,versicolor
|
||||
5.5,2.6,4.4,1.2,versicolor
|
||||
6.1,3.0,4.6,1.4,versicolor
|
||||
5.8,2.6,4.0,1.2,versicolor
|
||||
5.0,2.3,3.3,1.0,versicolor
|
||||
5.6,2.7,4.2,1.3,versicolor
|
||||
5.7,3.0,4.2,1.2,versicolor
|
||||
5.7,2.9,4.2,1.3,versicolor
|
||||
6.2,2.9,4.3,1.3,versicolor
|
||||
5.1,2.5,3.0,1.1,versicolor
|
||||
5.7,2.8,4.1,1.3,versicolor
|
||||
6.3,3.3,6.0,2.5,virginica
|
||||
5.8,2.7,5.1,1.9,virginica
|
||||
7.1,3.0,5.9,2.1,virginica
|
||||
6.3,2.9,5.6,1.8,virginica
|
||||
6.5,3.0,5.8,2.2,virginica
|
||||
7.6,3.0,6.6,2.1,virginica
|
||||
4.9,2.5,4.5,1.7,virginica
|
||||
7.3,2.9,6.3,1.8,virginica
|
||||
6.7,2.5,5.8,1.8,virginica
|
||||
7.2,3.6,6.1,2.5,virginica
|
||||
6.5,3.2,5.1,2.0,virginica
|
||||
6.4,2.7,5.3,1.9,virginica
|
||||
6.8,3.0,5.5,2.1,virginica
|
||||
5.7,2.5,5.0,2.0,virginica
|
||||
5.8,2.8,5.1,2.4,virginica
|
||||
6.4,3.2,5.3,2.3,virginica
|
||||
6.5,3.0,5.5,1.8,virginica
|
||||
7.7,3.8,6.7,2.2,virginica
|
||||
7.7,2.6,6.9,2.3,virginica
|
||||
6.0,2.2,5.0,1.5,virginica
|
||||
6.9,3.2,5.7,2.3,virginica
|
||||
5.6,2.8,4.9,2.0,virginica
|
||||
7.7,2.8,6.7,2.0,virginica
|
||||
6.3,2.7,4.9,1.8,virginica
|
||||
6.7,3.3,5.7,2.1,virginica
|
||||
7.2,3.2,6.0,1.8,virginica
|
||||
6.2,2.8,4.8,1.8,virginica
|
||||
6.1,3.0,4.9,1.8,virginica
|
||||
6.4,2.8,5.6,2.1,virginica
|
||||
7.2,3.0,5.8,1.6,virginica
|
||||
7.4,2.8,6.1,1.9,virginica
|
||||
7.9,3.8,6.4,2.0,virginica
|
||||
6.4,2.8,5.6,2.2,virginica
|
||||
6.3,2.8,5.1,1.5,virginica
|
||||
6.1,2.6,5.6,1.4,virginica
|
||||
7.7,3.0,6.1,2.3,virginica
|
||||
6.3,3.4,5.6,2.4,virginica
|
||||
6.4,3.1,5.5,1.8,virginica
|
||||
6.0,3.0,4.8,1.8,virginica
|
||||
6.9,3.1,5.4,2.1,virginica
|
||||
6.7,3.1,5.6,2.4,virginica
|
||||
6.9,3.1,5.1,2.3,virginica
|
||||
5.8,2.7,5.1,1.9,virginica
|
||||
6.8,3.2,5.9,2.3,virginica
|
||||
6.7,3.3,5.7,2.5,virginica
|
||||
6.7,3.0,5.2,2.3,virginica
|
||||
6.3,2.5,5.0,1.9,virginica
|
||||
6.5,3.0,5.2,2.0,virginica
|
||||
6.2,3.4,5.4,2.3,virginica
|
||||
5.9,3.0,5.1,1.8,virginica
|
||||
|
Loading…
Reference in a new issue