for today
This commit is contained in:
Andrew Straw
parent
cff8bbd085
commit
9f59285c76
470
lectures/lecture-05/1 - Command line, files, and processes.ipynb
Normal file
470
lectures/lecture-05/1 - Command line, files, and processes.ipynb
Normal file
|
|
@ -0,0 +1,470 @@
|
|||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a3c07551",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# What is the unix command line (also known as shell)? \n",
|
||||
"\n",
|
||||
"# Why would I use the unix command line?\n",
|
||||
"\n",
|
||||
"- faster when you know what you are doing (slower when you are new)\n",
|
||||
"- full capabilities\n",
|
||||
"- automation\n",
|
||||
"- remote access (especially SSH)\n",
|
||||
"- running self-hosted cloud services\n",
|
||||
"- your router, NAS drive, and other devices probably run it\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9a0018d3",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# How can I get access to a unix command line?\n",
|
||||
"\n",
|
||||
"## On your existing computer\n",
|
||||
"\n",
|
||||
"* On Windows, [install WSL2](https://docs.microsoft.com/en-us/windows/wsl/install) to run linux. Also native Windows command prompt very similar conceptually to unix command line, but most individual commands are slightly different.\n",
|
||||
"\n",
|
||||
"* On Mac or Linux: native terminal window\n",
|
||||
"\n",
|
||||
"## On a new computer\n",
|
||||
"\n",
|
||||
"* Puchase a [Raspberry Pi computer](https://www.raspberrypi.com/products/). I recommend something like the [Raspberry Pi Zero 2 W full starter kit](https://www.berrybase.de/raspberry-pi-zero-2-w-full-starter-kit) with HDMI adapter, SD card, case, USB adapter, and power supply. This should cost about 45 EUR. This particular seller is listed as an example that I know from personal experience is OK.\n",
|
||||
"* Inexpensive normal PC or laptop. Also: Linux runs great on old hardware.\n",
|
||||
"\n",
|
||||
"## Remotely\n",
|
||||
"\n",
|
||||
"* Accounts on university computers (Rechenzentum, the course server we will use for the exercise, ...).\n",
|
||||
"* Online \"cloud\" providers: https://www.hetzner.com/cloud, https://www.linode.com/pricing/#compute-shared, https://www.digitalocean.com/products/droplets and many more. Typically starting around 5 EUR per month with hourly billing."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "18c6cb20",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Concepts\n",
|
||||
"\n",
|
||||
"- SSH\n",
|
||||
"- Shell prompt\n",
|
||||
"- Running programs from the command line\n",
|
||||
"- Reading command line arguments in Python\n",
|
||||
"- Exit codes\n",
|
||||
"- Navigating in the command line\n",
|
||||
"- Environment variables, especially PATH"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "99568b9a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# SSH\n",
|
||||
"\n",
|
||||
"Image from https://www.hostinger.com/tutorials/ssh-tutorial-how-does-ssh-work\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Also great with [`tmux`](https://en.wikipedia.org/wiki/Tmux) to preserve your session if you logout or if your network connection drops."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "10345779",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Login to the course server using SSH\n",
|
||||
"\n",
|
||||
"## Step 1 - ensure you are in the Uni Freiburg network\n",
|
||||
"\n",
|
||||
"If you are not connected via a Uni Freiburg computer (including EduRoam), you must setup a VPN (\"virtual private network\"). See [this page](https://www.rz.uni-freiburg.de/de/services/netztel/vpn) for details about how to install the relevant software and use the university VPN.\n",
|
||||
"\n",
|
||||
"## Step 2 - start a terminal program\n",
|
||||
"\n",
|
||||
" - On Windows, go to the start menu and type \"Cmd\" and then `<Enter>` to start the \"Command Prompt\" program. (Alternative: the oldie-but-goodie [PuTTY](https://www.putty.org/) program.)\n",
|
||||
" - On macOS, go to \"Applications -> Utilities -> Terminal\" (German: \"Programme -> Dienstprogramme -> Terminal\")\n",
|
||||
" - On linux, start the \"Terminal\" program.\n",
|
||||
"\n",
|
||||
"## Step 3 - your username and passoword\n",
|
||||
"\n",
|
||||
"Your username is your the lowercase version of your lastname (e.g. Cem Özdemir would have a username of `oezdemir`). Your password is your matriculation number followed by \"x\" (e.g. `12345678x`).\n",
|
||||
"\n",
|
||||
"## Step 4 - login with SSH\n",
|
||||
"\n",
|
||||
"The following steps will only work from the Uni Freiburg network. If you are remote, you can use the VPN solution provided by the university.\n",
|
||||
"\n",
|
||||
"Type this command in your terminal program:\n",
|
||||
"\n",
|
||||
" ssh <username>@python-course.strawlab.org\n",
|
||||
"\n",
|
||||
"### First login with SSH\n",
|
||||
"\n",
|
||||
"For your first login, the terminal should print something like:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"The authenticity of host 'python-course.strawlab.org (10.4.66.100)' can't be established.\n",
|
||||
"ED25519 key fingerprint is SHA256:MWMheGkCSvu8drY2gjS/9SQ7ZBBTRh+bQlj0SNblzyo.\n",
|
||||
"This key is not known by any other names\n",
|
||||
"Are you sure you want to continue connecting (yes/no/[fingerprint])? \n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Type `yes` to continue.\n",
|
||||
"\n",
|
||||
"Then you will see something like:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"Warning: Permanently added 'python-course.strawlab.org' (ED25519) to the list of known hosts.\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"### Entering password\n",
|
||||
"\n",
|
||||
"After your first login, the steps in the section above will not happen as the SSH server will be \"known\" to your own system.\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"<username>@python-course.strawlab.org's password:\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Here you type your password. You will not be able to see the characters as you type them.\n",
|
||||
"\n",
|
||||
"Upon success, the following should be printed:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"Linux dragon 6.1.0-27-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.115-1 (2024-11-01) x86_64\n",
|
||||
"\n",
|
||||
"The programs included with the Debian GNU/Linux system are free software;\n",
|
||||
"the exact distribution terms for each program are described in the\n",
|
||||
"individual files in /usr/share/doc/*/copyright.\n",
|
||||
"\n",
|
||||
"Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent\n",
|
||||
"permitted by applicable law.\n",
|
||||
"straw@dragon:~$\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"The final line (`<username>@dragon:~$ `) is called the command prompt and is where you type your commands.\n",
|
||||
"\n",
|
||||
"When you press the `enter` (or `return`) key, the command will be executed by the remote computer. Your commands must be given in the \"shell\" language. Although there are some slight differences between different shell languages, we are going to stick with a common core of unix shell in this class."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "4b92c017-ad75-4147-8e8f-034cda135700",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Editing files from the command line\n",
|
||||
"\n",
|
||||
"Demo of the use of the `nano` program as a plain text editor.\n",
|
||||
"\n",
|
||||
"Note that the mouse usually does not work in the terminal!"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fd62cd31",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Viewing files from the command line\n",
|
||||
"\n",
|
||||
"Use of the `cat` program (which stands for concatenate).\n",
|
||||
"\n",
|
||||
"Also the `less` program."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7d9d8b0a-c5e0-473b-a073-e93217b9e238",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Demo: create and run Python from command line\n",
|
||||
"\n",
|
||||
"Let's make `my_program1.py` which will print `\"Hello world\"`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c13ff2ea",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Using our anaconda environment\n",
|
||||
"\n",
|
||||
"The python we used above is not Python managed by Anaconda but rather the built-in python to the server's operating system. To use all of our libraries, we need to load our Anaconda environment. From the command line, run this:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"eval \"$(/anaconda3/bin/conda shell.bash hook)\"\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Upon success, your prompt should change from\n",
|
||||
"\n",
|
||||
" straw@dragon:~$\n",
|
||||
"\n",
|
||||
"to\n",
|
||||
"\n",
|
||||
" (base) straw@dragon:~$\n",
|
||||
" \n",
|
||||
"Notice the extra `(base) ` at the beginning. This shows that you are in the \"base\" anaconda environment.\n",
|
||||
"\n",
|
||||
"We can confirm this by typing `which python`, which should return `/anaconda3/bin/python`.\n",
|
||||
"\n",
|
||||
"Finally, make sure our simple program still runs with this python. Type `python my_program1.py`:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"(base) straw@dragon:~$ python my_program1.py\n",
|
||||
"Hello world\n",
|
||||
"(base) straw@dragon@strawlab-course-server:~$\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "58174f26",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Saving a plot of data\n",
|
||||
"\n",
|
||||
"Let's now make a program to do some plotting and data analysis. Let's first make a program which simply plots some fake data and saves it to a .png file.\n",
|
||||
"\n",
|
||||
"Create a new text file called `plot1.py` and set it to this content:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"import pandas as pd\n",
|
||||
"import seaborn as sns\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"\n",
|
||||
"df = pd.DataFrame({'sepal length (cm)': [5.1, 4.9, 4.7, 4.6, 5.0, 5.4, 4.6, 5.0, 4.4, 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7, 5.4, 5.1, 5.7, 5.1, 5.4, 5.1, 4.6, 5.1, 4.8, 5.0, 5.0, 5.2, 5.2, 4.7, 4.8, 5.4, 5.2, 5.5, 4.9, 5.0, 5.5, 4.9, 4.4, 5.1, 5.0, 4.5, 4.4, 5.0, 5.1, 4.8, 5.1, 4.6, 5.3, 5.0, 7.0, 6.4, 6.9, 5.5, 6.5, 5.7, 6.3, 4.9, 6.6, 5.2, 5.0, 5.9, 6.0, 6.1, 5.6, 6.7, 5.6, 5.8, 6.2, 5.6, 5.9, 6.1, 6.3, 6.1, 6.4, 6.6, 6.8, 6.7, 6.0, 5.7, 5.5, 5.5, 5.8, 6.0, 5.4, 6.0, 6.7, 6.3, 5.6, 5.5, 5.5, 6.1, 5.8, 5.0, 5.6, 5.7, 5.7, 6.2, 5.1, 5.7, 6.3, 5.8, 7.1, 6.3, 6.5, 7.6, 4.9, 7.3, 6.7, 7.2, 6.5, 6.4, 6.8, 5.7, 5.8, 6.4, 6.5, 7.7, 7.7, 6.0, 6.9, 5.6, 7.7, 6.3, 6.7, 7.2, 6.2, 6.1, 6.4, 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 6.3, 6.4, 6.0, 6.9, 6.7, 6.9, 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9], 'sepal width (cm)': [3.5, 3.0, 3.2, 3.1, 3.6, 3.9, 3.4, 3.4, 2.9, 3.1, 3.7, 3.4, 3.0, 3.0, 4.0, 4.4, 3.9, 3.5, 3.8, 3.8, 3.4, 3.7, 3.6, 3.3, 3.4, 3.0, 3.4, 3.5, 3.4, 3.2, 3.1, 3.4, 4.1, 4.2, 3.1, 3.2, 3.5, 3.6, 3.0, 3.4, 3.5, 2.3, 3.2, 3.5, 3.8, 3.0, 3.8, 3.2, 3.7, 3.3, 3.2, 3.2, 3.1, 2.3, 2.8, 2.8, 3.3, 2.4, 2.9, 2.7, 2.0, 3.0, 2.2, 2.9, 2.9, 3.1, 3.0, 2.7, 2.2, 2.5, 3.2, 2.8, 2.5, 2.8, 2.9, 3.0, 2.8, 3.0, 2.9, 2.6, 2.4, 2.4, 2.7, 2.7, 3.0, 3.4, 3.1, 2.3, 3.0, 2.5, 2.6, 3.0, 2.6, 2.3, 2.7, 3.0, 2.9, 2.9, 2.5, 2.8, 3.3, 2.7, 3.0, 2.9, 3.0, 3.0, 2.5, 2.9, 2.5, 3.6, 3.2, 2.7, 3.0, 2.5, 2.8, 3.2, 3.0, 3.8, 2.6, 2.2, 3.2, 2.8, 2.8, 2.7, 3.3, 3.2, 2.8, 3.0, 2.8, 3.0, 2.8, 3.8, 2.8, 2.8, 2.6, 3.0, 3.4, 3.1, 3.0, 3.1, 3.1, 3.1, 2.7, 3.2, 3.3, 3.0, 2.5, 3.0, 3.4, 3.0], 'petal length (cm)': [1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1.5, 1.6, 1.4, 1.1, 1.2, 1.5, 1.3, 1.4, 1.7, 1.5, 1.7, 1.5, 1.0, 1.7, 1.9, 1.6, 1.6, 1.5, 1.4, 1.6, 1.6, 1.5, 1.5, 1.4, 1.5, 1.2, 1.3, 1.4, 1.3, 1.5, 1.3, 1.3, 1.3, 1.6, 1.9, 1.4, 1.6, 1.4, 1.5, 1.4, 4.7, 4.5, 4.9, 4.0, 4.6, 4.5, 4.7, 3.3, 4.6, 3.9, 3.5, 4.2, 4.0, 4.7, 3.6, 4.4, 4.5, 4.1, 4.5, 3.9, 4.8, 4.0, 4.9, 4.7, 4.3, 4.4, 4.8, 5.0, 4.5, 3.5, 3.8, 3.7, 3.9, 5.1, 4.5, 4.5, 4.7, 4.4, 4.1, 4.0, 4.4, 4.6, 4.0, 3.3, 4.2, 4.2, 4.2, 4.3, 3.0, 4.1, 6.0, 5.1, 5.9, 5.6, 5.8, 6.6, 4.5, 6.3, 5.8, 6.1, 5.1, 5.3, 5.5, 5.0, 5.1, 5.3, 5.5, 6.7, 6.9, 5.0, 5.7, 4.9, 6.7, 4.9, 5.7, 6.0, 4.8, 4.9, 5.6, 5.8, 6.1, 6.4, 5.6, 5.1, 5.6, 6.1, 5.6, 5.5, 4.8, 5.4, 5.6, 5.1, 5.1, 5.9, 5.7, 5.2, 5.0, 5.2, 5.4, 5.1], 'petal width (cm)': [0.2, 0.2, 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.4, 0.4, 0.3, 0.3, 0.3, 0.2, 0.4, 0.2, 0.5, 0.2, 0.2, 0.4, 0.2, 0.2, 0.2, 0.2, 0.4, 0.1, 0.2, 0.2, 0.2, 0.2, 0.1, 0.2, 0.2, 0.3, 0.3, 0.2, 0.6, 0.4, 0.3, 0.2, 0.2, 0.2, 0.2, 1.4, 1.5, 1.5, 1.3, 1.5, 1.3, 1.6, 1.0, 1.3, 1.4, 1.0, 1.5, 1.0, 1.4, 1.3, 1.4, 1.5, 1.0, 1.5, 1.1, 1.8, 1.3, 1.5, 1.2, 1.3, 1.4, 1.4, 1.7, 1.5, 1.0, 1.1, 1.0, 1.2, 1.6, 1.5, 1.6, 1.5, 1.3, 1.3, 1.3, 1.2, 1.4, 1.2, 1.0, 1.3, 1.2, 1.3, 1.3, 1.1, 1.3, 2.5, 1.9, 2.1, 1.8, 2.2, 2.1, 1.7, 1.8, 1.8, 2.5, 2.0, 1.9, 2.1, 2.0, 2.4, 2.3, 1.8, 2.2, 2.3, 1.5, 2.3, 2.0, 2.0, 1.8, 2.1, 1.8, 1.8, 1.8, 2.1, 1.6, 1.9, 2.0, 2.2, 1.5, 1.4, 2.3, 2.4, 1.8, 1.8, 2.1, 2.4, 2.3, 1.9, 2.3, 2.5, 2.3, 1.9, 2.0, 2.3, 1.8], 'species': ['setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'setosa', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'versicolor', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica', 'virginica']})\n",
|
||||
"\n",
|
||||
"sns.stripplot(x=\"species\", y=\"sepal width (cm)\", data=df)\n",
|
||||
"plt.savefig('plot1.png')\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"If we use the Jupyter Notebook, we can open our newly saved file and verify that it looks like this:\n",
|
||||
"\n",
|
||||
""
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d295c5ef",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Command line arguments and exit codes\n",
|
||||
"\n",
|
||||
"Running a program from the command line has many conceptual similarities to calling a function within Python. There are *arguments* passed to the program, the program returns a final result, and there are \"side effects\". Typically the programs run more for the \"side effects\" in this view. The final result of a program is merely a single integer that is `0` in the case of no error, and any non-zero number in case of error. This final result, an integer, is called the \"exit code\" or the \"error code\".\n",
|
||||
"\n",
|
||||
"On the command line, arguments are given with a space. In fact, when we call python with our script, we are actually running the program `python` with our program as the first command line argment, e.g. `my_program1.py`.:\n",
|
||||
"\n",
|
||||
" python my_program1.py\n",
|
||||
" \n",
|
||||
"We can call it with additional arguments like this:\n",
|
||||
"\n",
|
||||
" python my_program1.py arg1 arg2\n",
|
||||
" \n",
|
||||
"If we run our program above, we see it still runs as before (printing `Hello world`) but at least there is no error. In a moment we will see how to read these arguments from Python. For now, let's check the exit code. We do this with the command:\n",
|
||||
"\n",
|
||||
" echo $?\n",
|
||||
" \n",
|
||||
"What this does is call the `echo` program with a special variable named `?`. At the command line, the `$` means to get the variable, and `?` means the last exit code. So you must be careful to run this command immediately after a command where you want to know the exit code. So, this command is again using a command line argument (`$?`) to the program `echo`. It should print `0` in case of no error.\n",
|
||||
"\n",
|
||||
"Now let's use these command line arguments from our python program. For this we `import sys` and use `sys.argv` which is a list of all command line arguments. If we create a program `my_program2.py` to have the following contents:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"import sys\n",
|
||||
"print(sys.argv)\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Run it like so:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"(pm21-tiger) <username>@strawlab-course-server:~$ python my_program2.py\n",
|
||||
"['my_program2.py']\n",
|
||||
"(pm21-tiger) <username>@strawlab-course-server:~$\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"We can also run it with arguments:\n",
|
||||
"\n",
|
||||
"```\n",
|
||||
"(pm21-tiger) astraw@strawlab-course-server:~$ python my_program2.py arg1 arg2\n",
|
||||
"['my_program2.py', 'arg1', 'arg2']\n",
|
||||
"(pm21-tiger) astraw@strawlab-course-server:~$\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Note that all arguments are strings.\n",
|
||||
"\n",
|
||||
"It is common to pass filenames as command line arguments. The python program will receive filenames (like all command line arguments) as strings and your program will have to interpret them as filenames.\n",
|
||||
"\n",
|
||||
"(Power user: [argparse](https://docs.python.org/3/library/argparse.html) library for parsing command line arguments.)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "44a66d09-1e87-4b44-b532-ff5a6e95de24",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# topic: \"current working directory\"\n",
|
||||
"\n",
|
||||
"Every program has a \"current working directory\".\n",
|
||||
"\n",
|
||||
"To view it in Python:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"import os\n",
|
||||
"\n",
|
||||
"print(os.getcwd())\n",
|
||||
"```"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b5693c06",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Navigation at the command line\n",
|
||||
"\n",
|
||||
"(Mouse typically does not work.)\n",
|
||||
"\n",
|
||||
"Up arrow, down arrow, `<tab>`.\n",
|
||||
"\n",
|
||||
"(Power user: `<Ctrl-r>` search)\n",
|
||||
" \n",
|
||||
"`pwd` Present working directory\n",
|
||||
" \n",
|
||||
"`ls` list files\n",
|
||||
" \n",
|
||||
"`ls -a` list all files, including hidden files\n",
|
||||
"\n",
|
||||
"`ls -l` list \"long\"\n",
|
||||
" \n",
|
||||
"`ls -al` list all files \"long\"\n",
|
||||
" \n",
|
||||
"`cd` change directory (to home dir)\n",
|
||||
"\n",
|
||||
"`cd somedir` change directory to `somedir`\n",
|
||||
" \n",
|
||||
"(Power user: `cd -`)\n",
|
||||
"\n",
|
||||
"`cat filename` will print the contents of the file `filename` to the screen"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "caaa26b6",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Paths (also known as filesystem locations)\n",
|
||||
"\n",
|
||||
"Directories are separated with `/`. Paths can be \"absolute\" and start with `/` or relative, in which case they may start with `./`.\n",
|
||||
"\n",
|
||||
"- `.` is the current directory\n",
|
||||
"- `..` is the parent directory"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9e49e94a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Environment variables at the command line\n",
|
||||
"\n",
|
||||
"Environment variables are set by the operating system in the command line environment and are used to pass information to programs without explicitly using arguments. They are thus like variables in python in the global frame which are accessed from within a function.\n",
|
||||
"\n",
|
||||
"We already saw `$?`.\n",
|
||||
"\n",
|
||||
"`$HOME`\n",
|
||||
"\n",
|
||||
"`$PATH` To start a program, this is searched unless the program name has a path (either absolute: `/some/path/program-name` or relative: `./program-name`)\n",
|
||||
"\n",
|
||||
"Note: anaconda works by setting the `$PATH` and related `$PYTHONPATH`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9814d3c9",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Useful unix programs\n",
|
||||
"\n",
|
||||
"`ssh`\n",
|
||||
"\n",
|
||||
"`cat` view a file (\"con**cat**enate\" it to the command line)\n",
|
||||
"\n",
|
||||
"`less` view a file, page by page\n",
|
||||
"\n",
|
||||
"`nano` (or, for a power user: Visual Studio Code with Remote SSH extension)\n",
|
||||
"\n",
|
||||
"Programs we won't use, but are very useful:\n",
|
||||
"\n",
|
||||
"`rg` ripgrep - quickly search in recursively descended file tree\n",
|
||||
"\n",
|
||||
"`curl` download from URLs\n",
|
||||
"\n",
|
||||
"`scp` secure copy\n",
|
||||
"\n",
|
||||
"`tmux` terminal multiplexer"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "c13ae6e1",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Reading and plotting data from the command line\n",
|
||||
"\n",
|
||||
"Copy the `iris.csv` file\n",
|
||||
"\n",
|
||||
" cp /data/iris.csv .\n",
|
||||
" \n",
|
||||
"Now create `plot2.py` to have the following contents:\n",
|
||||
"\n",
|
||||
"```python\n",
|
||||
"import pandas as pd\n",
|
||||
"import seaborn as sns\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import sys\n",
|
||||
"\n",
|
||||
"fname = sys.argv[1]\n",
|
||||
"print(f'reading {fname}')\n",
|
||||
"\n",
|
||||
"df = pd.read_csv(fname)\n",
|
||||
"sns.stripplot(x=\"species\", y=\"sepal_width\", data=df)\n",
|
||||
"plt.savefig('plot2.png')\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Now we can run this with: `python plot2.py iris.csv`"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "157597ea",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Further resources\n",
|
||||
"\n",
|
||||
"* [Software Carpentry, \"The Unix Shell\"](https://swcarpentry.github.io/shell-novice/)\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3 (ipykernel)",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.7"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
BIN
lectures/lecture-05/Matrix-SSH.png
Normal file
BIN
lectures/lecture-05/Matrix-SSH.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 2.8 MiB |
BIN
lectures/lecture-05/plot1.png
Normal file
BIN
lectures/lecture-05/plot1.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
BIN
lectures/lecture-05/ssh.jpg
Normal file
BIN
lectures/lecture-05/ssh.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 45 KiB |
Loading…
Reference in a new issue