diff --git a/README.md b/README.md index 67c0575..89187ec 100644 --- a/README.md +++ b/README.md @@ -42,6 +42,7 @@ http://pythontutor.com/ | exercise-06 | 2024-11-20 | | exercise-07 | 2024-11-27 | | exercise-08 | 2024-12-04 | +| exercise-09 | 2024-12-11 | ## Submitting assignments diff --git a/exercises/release/exercise-09/1__Image_analysis_1.ipynb b/exercises/release/exercise-09/1__Image_analysis_1.ipynb new file mode 100644 index 0000000..64e2255 --- /dev/null +++ b/exercises/release/exercise-09/1__Image_analysis_1.ipynb @@ -0,0 +1,419 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# You must run this cell, but you can ignore its contents.\n", + "\n", + "import hashlib\n", + "\n", + "def ads_hash(ty):\n", + " \"\"\"Return a unique string for input\"\"\"\n", + " ty_str = str(ty).encode()\n", + " m = hashlib.sha256()\n", + " m.update(ty_str)\n", + " return m.hexdigest()[:10]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Numpy array for images" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "638fe83e8c7237afbb3ac1af8f16e079", + "grade": false, + "grade_id": "cell-6a738215de5622a5", + "locked": true, + "schema_version": 3, + "solution": false + } + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Images are saved in computers as arrays of numbers. In Python, the library called `numpy` (often abbreviated `np` for short). is the most common way to manipulate arrays of numbers. Here we will create an 8x8 pixel image and put it in the variable `check`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create a variable called `check` which will contain an 8x8 array of numbers.\n", + "check = np.zeros((8, 8))\n", + "check[::2, 1::2] = 1\n", + "check[1::2, ::2] = 1\n", + "check" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Now lets view our 8x8 pixel image:\n", + "plt.imshow(check, cmap='gray');" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Questions Part A - Image representation\n", + "\n", + "Use numpy slicing to set the values in an array:\n", + "\n", + "- Use `plt.imshow` to show a new 8x8 pixel image in which the top half is white and the bottom half is black.\n", + "- Use `plt.imshow` to show a new 8x8 pixel image in which the left half is white and the right half is black." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "54a6b9c8846af369d5753911d916851b", + "grade": true, + "grade_id": "cell-4ee9d19bc490e8bf", + "locked": false, + "points": 1, + "schema_version": 3, + "solution": true, + "task": false + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "9b9342fa9b869febd0d2be08cc84a619", + "grade": true, + "grade_id": "cell-9d1baa0ef55db4c6", + "locked": false, + "points": 1, + "schema_version": 3, + "solution": true, + "task": false + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Questions Part B - More image representation\n", + "\n", + "Write the numerical value corresponding with each pixel. In other words, what number corresponds with the black color and which number corresponds with the white color?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "05a82888baf180f282cff34c8466f850", + "grade": false, + "grade_id": "cell-2fd29ad721f075e0", + "locked": false, + "schema_version": 3, + "solution": true + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "e0766ab97487f394e00e43e62cdc3297", + "grade": true, + "grade_id": "cell-4f11d45d5fbbe454", + "locked": true, + "points": 1, + "schema_version": 3, + "solution": false + } + }, + "outputs": [], + "source": [ + "# If this runs without error, it means the answer in your previous cell was correct.\n", + "assert ads_hash(float(black))=='8aed642bf5'\n", + "assert ads_hash(float(white))=='d0ff5974b6'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Questions Part C - More image representation and contrast enhancement\n", + "\n", + "Now let's look at a more interesting image. We will use data included with the library `skimage` (also called scikit-image). The homepage for scikit-image is [here](http://scikit-image.org/)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# If import skimage fails, you can install it like this in Jupyter:\n", + "# !pip install scikit-image\n", + "# or like this at the command line:\n", + "# python -m pip install scikit-image\n", + "from skimage import data\n", + "\n", + "# Load the sample image data into a variable called `camera`.\n", + "camera = data.camera()\n", + "\n", + "plt.figure(figsize=(4, 4))\n", + "plt.imshow(camera, cmap='gray', interpolation='nearest', vmin=0, vmax=255)\n", + "plt.axis('off')\n", + "\n", + "plt.tight_layout()\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So, let's checkout some things about the image. How many pixels are here? Put your answer in the `num_pixels` variable." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "9724b1acb021773cfeba10403ce47eae", + "grade": false, + "grade_id": "cell-35aec8ec167eaa1a", + "locked": false, + "schema_version": 3, + "solution": true, + "task": false + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "c45d4e8bc58c1a92790a5307c61594df", + "grade": true, + "grade_id": "cell-9eda0f4283380a85", + "locked": true, + "points": 1, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "# If this runs without error, it means the answer in your previous cell was correct.\n", + "assert ads_hash(num_pixels)=='54faea9b3e'" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "What is the distribution of luminance values in the image?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "plt.hist(camera.flat,bins=256);\n", + "plt.xlabel('luminance');\n", + "plt.ylabel('number of occurances');" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can see that there are two main peaks in the intensity histogram. We can also see that the intensities go from 0 to 255. This happens to be the range of values that fit in an 8 bit *byte*. This is the unit of memory size on modern computers. Is it correct to think that each pixel in this image is stored as a byte?\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "camera.dtype" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `dtype` of a numpy array is the \"data type\" - the type for each individual element of the array. Above we see the answer is `uint8` which means \"unsigned integer, 8 bits\". So, yes, each pixel is stored here as a byte." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you look at the histogram, you can see that there is some part of the luminance space which has very few occurances, namely above a luminance value of about 220. We can probably make better use of the 0-255 dynamic range available.\n", + "\n", + "In the cell below, enter a scale factor which fills the histogram of possible luminances more completely but does not cause too much clipping of the image values. The tower in the background should still be visible, for example. Look at the figures below to judge the effect of changing the scale factor. Put this in the variable `scale_factor`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "d2db0a18898480013b2a7637e06a4225", + "grade": false, + "grade_id": "cell-9ff4a4d60c8485d0", + "locked": false, + "schema_version": 3, + "solution": true + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "d258c5be175387e003b6f1dd91dfdfcb", + "grade": true, + "grade_id": "cell-484ef1503b6e25a0", + "locked": true, + "points": 1, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "# assert that scale_factor is a number\n", + "assert scale_factor - 0.0 == scale_factor" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's look at the histogram of rescaled luminances." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "rescaled = np.clip(scale_factor*camera.astype(np.float32),0,255).astype(np.uint8)\n", + "plt.hist(rescaled.flat,bins=256);\n", + "plt.xlabel('luminance');\n", + "plt.ylabel('number of occurances');\n", + "\n", + "plt.figure(figsize=(4, 4))\n", + "plt.imshow(rescaled, cmap='gray', interpolation='nearest', vmin=0, vmax=255)\n", + "plt.axis('off')\n", + "plt.tight_layout()\n", + "plt.show()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.10" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/exercises/release/exercise-09/2__Image_analysis_2.ipynb b/exercises/release/exercise-09/2__Image_analysis_2.ipynb new file mode 100644 index 0000000..dce7a61 --- /dev/null +++ b/exercises/release/exercise-09/2__Image_analysis_2.ipynb @@ -0,0 +1,910 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "b82b3443534132b3771ea1bd5dce7292", + "grade": false, + "grade_id": "cell-1d6811de1e292425", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "In this notebook, we explore how to detect a moving object on a stationary background." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "c182d2be009c8c695eef8625a316ad12", + "grade": false, + "grade_id": "cell-c101bf39d596ef9a", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from scipy import ndimage\n", + "\n", + "# If import imageio fails, you can install it like this in Jupyter:\n", + "# !pip install 'imageio[ffmpeg]'\n", + "# or like this at the command line:\n", + "# python -m pip install 'imageio[ffmpeg]'\n", + "import imageio" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "e2c79f491e8c8c3c610c17a2c388b4ac", + "grade": false, + "grade_id": "cell-7161f550b1a10a31", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "# Here we define a helper function which we call below to make sure\n", + "# an image is an \"unsigned 8 bit integer\". This way, we know they\n", + "# take only a single byte per pixel and have a value from 0..255.\n", + "\n", + "def ensure_dtype_uint8(arr):\n", + " return arr.astype(np.uint8)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "ff7991ce2a103a43b9fcbdd9c12f49d4", + "grade": false, + "grade_id": "cell-108186f7f14d851d", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Here we load the first frame of a movie file saved in the Straw Lab of a fly walking." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "10aa39ce56f805abd276ebf6158c31cf", + "grade": false, + "grade_id": "cell-721f647abba6218f", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "fname = \"data/short-movie20170810_182130.mp4\"\n", + "reader = imageio.get_reader(fname)\n", + "for frame in reader:\n", + " frame0 = frame[:,:,1] # take only green channel\n", + " break" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "2190e9954dbf33018d3260e5707d16ef", + "grade": false, + "grade_id": "cell-6589753ba9570c6b", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now, we plot the first frame (the first image) in the video. We do this once in grayscale and once in false color." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "2d78181caa0fb6a2c3400add13dfdad9", + "grade": false, + "grade_id": "cell-f8d6f9c412400ac4", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "plt.imshow(frame0, cmap='gray')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "4c4dc1664274ccb8975819355360a08b", + "grade": false, + "grade_id": "cell-79ff5ac403dbec51", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "plt.imshow(frame0, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "c136511dff9e32106ddf90e1e3b30a3d", + "grade": false, + "grade_id": "cell-b4d51617498cdf00", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Note how the false color image lets you see more easily approximately what pixel intensity values are present." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "16d3565781d1ef6c69094952f33fd0f1", + "grade": false, + "grade_id": "cell-bb6b6c1075212bfa", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now, list make a histogram of the pixel intensity values." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "9973e9aa97f86e66e016d6c94a3e4270", + "grade": false, + "grade_id": "cell-1f03feb12c388aa0", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "plt.hist(frame0.flat, bins=30);" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "3f08ed66615d8dd3ab93e8976c026f5b", + "grade": false, + "grade_id": "cell-b8ef9c796dcc7da1", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now we will load all the frames into a single large data structure from disk into memory." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "af923e6f1df358188b7bf9d73afc8bef", + "grade": false, + "grade_id": "cell-d8d4b833e722c306", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "# get all frames into big 3D array\n", + "(height,width) = frame0.shape\n", + "all_frames = []\n", + "reader.set_image_index(0) # return to start of file\n", + "for frame in reader:\n", + " all_frames.append( frame[:,:,1] )\n", + "all_frames = np.array(all_frames)\n", + "print(all_frames.shape)\n", + "n_frames = len(all_frames)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "02d2ad93fdb67de427d32f3fc0c7e833", + "grade": false, + "grade_id": "cell-3ef9393de49f1ec7", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now the entire movie is stored as a large 3D array of size: number of frames x width x height." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "0b950953dde6ce892c13c4217c9ad3a4", + "grade": false, + "grade_id": "cell-c2722d719e790f09", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now, we want to find a fly in the image. The fly is moving, but the background is stationary, so we should be able to determine a single \"background image\". We can try to find the background image in at least two ways: with a \"mean image\" (also \"average image\"), which should average over the fly's position. And with a \"median image\" which should completely eliminate the fly from the backround image if the fly was absent for most of the time from each pixel." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "de347ea8c88b9edf97f3b28e0f1ebf5a", + "grade": false, + "grade_id": "cell-2f4ea267c6c2ccd9", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "mean_frame = np.mean(all_frames, axis=0)\n", + "median_frame = np.median(all_frames, axis=0)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "1b60dd865c78115948b238ed3c5a44f1", + "grade": false, + "grade_id": "cell-e2365cb18d5b20d1", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "plt.imshow(mean_frame, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "e1117202150673346ec746b98ce66971", + "grade": false, + "grade_id": "cell-58ec1fa04a85c5db", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "plt.imshow(median_frame, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "41b6632af41f7da6e03705041ef382d1", + "grade": false, + "grade_id": "cell-5faa20bd7e208895", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now that we have the \"background\", we can find the difference from background, which should emphasize anything that is moving." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "074b256543714ac267d6a652a9c816bf", + "grade": false, + "grade_id": "cell-074247293d712cac", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "frame0_absdiff = abs(frame0 - median_frame)\n", + "plt.imshow(frame0_absdiff, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "33636375e38961ac449739f12d0c677b", + "grade": false, + "grade_id": "cell-798add9cb796580b", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now lets make a histogram of the `frame0_absdiff` image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "b743c16e5470a4e0ecba1b1830b7b644", + "grade": false, + "grade_id": "cell-c71b6c10e0ee3018", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "plt.hist(frame0_absdiff.flat, bins=30);" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "c470d43abe6189c7992104c8517126cb", + "grade": false, + "grade_id": "cell-2e5d36068fff4d1d", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "We want to find a threshold for this \"absolute difference image\" which will separate the fly from the background. From frame to frame the background changes very little - only noise in the light levels, camera sensor and caused by lossy compression to a movie format cause changes. But when the fly moves over the background, there is a large change in luminance values. This happens only in very few pixels. So few that they do not show on the histogram. However, matplotlib will automatically scale the axes, so we know there must be few pixels with values at 150 or above." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "b0af9598ca270ca26efc5400cf2e19b5", + "grade": false, + "grade_id": "cell-d05a54e4c069f9aa", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "## Questions Part A\n", + "\n", + "Enter a threshold below and describe why you chose this value. Put your answer in the variable `threshold`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "71fcb58983ffdeb9114e35ad94690907", + "grade": false, + "grade_id": "cell-1ef3868b1b1b045c", + "locked": false, + "schema_version": 3, + "solution": true + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "77322ebaaa5f2b4b29f0e3d0488add63", + "grade": true, + "grade_id": "cell-c848bbfece87e521", + "locked": false, + "points": 1, + "schema_version": 3, + "solution": true + } + }, + "source": [ + "YOUR ANSWER HERE" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "fdcf39f570e2d445f8fddba75e926a82", + "grade": true, + "grade_id": "cell-edadac5626867908", + "locked": true, + "points": 1, + "schema_version": 3, + "solution": false + } + }, + "outputs": [], + "source": [ + "# If this runs without error, it means the answer in your previous cell was a number.\n", + "assert(threshold+0==threshold)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "f2f208aff8f5e0442143c952eb9bbcb0", + "grade": false, + "grade_id": "cell-f6fc50107a0d129a", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now we want to \"binarize\" or \"threshold\" the image so that it is all zeros and ones (or true and false). This will let use use analyses like connected components labeling later." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "07af0cac3c4f77c9758189d926502bac", + "grade": false, + "grade_id": "cell-fcef9de824e56bd5", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "binarized = ensure_dtype_uint8(frame0_absdiff>threshold)\n", + "plt.imshow(binarized);\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "ff1db2b54f0da4954fa6a7d47ee0a5ad", + "grade": false, + "grade_id": "cell-0338555f51c06105", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Now we will use **connected components labeling** via the `ndimage.label()` function.\n", + "\n", + "We will do more with **connected components labeling** later. For now, you can [read about connected components labeling](https://en.wikipedia.org/wiki/Connected-component_labeling).\n", + "\n", + "We will use these labels to perform object detection. We will not bother to detect the fly on every single frame. Only on the frames where we have exactly one label do we say that we detected the fly. We skip the other frames, ignoring them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "c33765aa2c06d7fa5451d15226f9e5c0", + "grade": false, + "grade_id": "cell-9dae893dac3022a4", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "labels, num_labels = ndimage.label(binarized)\n", + "plt.imshow(labels, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "a62a1a753e77ef467e8ba67f399843eb", + "grade": false, + "grade_id": "cell-bf4c1b87ad01a8bb", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "## Questions Part B\n", + "\n", + "Run the object detection algorithm for each frame and plot the result. Here, we are going to make lists `frames`, `xs`, `ys` to save the results of our object detection step.\n", + "\n", + "Try re-running the above cells (from \"Questions Part A\") to the cell below with varying values of the `threshold` value until you get a good tracking result. You should ideally have zero warnings printed, but one or two are OK. " + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "bd98d21e0aa72c9a4ea72292a21e56a8", + "grade": false, + "grade_id": "cell-cb7ec6b50e3d024a", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "outputs": [], + "source": [ + "frames = []\n", + "xs = []\n", + "ys = []\n", + "for i in range(n_frames):\n", + " frame = all_frames[i,:,:]\n", + " frame_absdiff = abs(frame - median_frame)\n", + " binarized = ensure_dtype_uint8(frame_absdiff>threshold)\n", + " labels, num_labels = ndimage.label(binarized)\n", + " if num_labels!=1:\n", + " print('WARNING: num_labels %d on frame %d, skipping' % (num_labels,i))\n", + " continue\n", + " y,x = np.mean(np.nonzero(labels==1),axis=1)\n", + " frames.append(i)\n", + " xs.append(x)\n", + " ys.append(y)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "c55362805bd45921198f09157c1d21f2", + "grade": false, + "grade_id": "cell-c1e4e9b97c4899cb", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Make with an image of the first frame (saved in the variable `frame0` above) in the background and then the fly trajectory overlaid on top of the image.\n", + "\n", + "You can do this by first calling `plt.imshow()` and then `plt.plot()`. Your result should look like this:\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "d156a2cb11d8b53eca4ce0c2c6d98c5a", + "grade": true, + "grade_id": "cell-7250d526d95bf828", + "locked": false, + "points": 1, + "schema_version": 3, + "solution": true + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "e7bff9942e7efca03e28e36dafa297a3", + "grade": false, + "grade_id": "cell-06fde843bcd5b77e", + "locked": true, + "schema_version": 3, + "solution": false, + "task": false + } + }, + "source": [ + "Make a plot of the timeseries of the x and y values.\n", + "\n", + "We want a plot that looks like the following. You need to fix the following code to do this:\n", + "\n", + "```\n", + "fig, axes = plt.subplots(nrows=2, ncols=1, sharex=True)\n", + "axes[0].plot(time_values,x_values,'x')\n", + "axes[0].set_ylabel('x (px)')\n", + "axes[1].plot(time_values,y_values,'x')\n", + "axes[1].set_ylabel('y (px)')\n", + "axes[1].set_xlabel('time (frames)')\n", + "```\n", + "\n", + "As a side note, the `sharedx` links the two panels together so they have the same X axis. When using matplotlib interactively (highly recommended!!!), panning and zooming in one panel also pans and zooms the other panel. This is incredibly useful.\n", + "\n", + "" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "a1268da25c1f3b257c2e16acfde6b9d4", + "grade": true, + "grade_id": "cell-8680c2c7c9c5cb22", + "locked": false, + "points": 1, + "schema_version": 3, + "solution": true, + "task": false + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Questions Part C\n", + "\n", + "Describe: How did we get the background image? How did we compute a binary image? What did we do with the binary image?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "8f8394bf41b5ede928f69dff168e85f4", + "grade": true, + "grade_id": "cell-0a341a483941e5cd", + "locked": false, + "points": 2, + "schema_version": 3, + "solution": true + } + }, + "source": [ + "YOUR ANSWER HERE" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.10" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/exercises/release/exercise-09/3__Bean_counting.ipynb b/exercises/release/exercise-09/3__Bean_counting.ipynb new file mode 100644 index 0000000..8dc8873 --- /dev/null +++ b/exercises/release/exercise-09/3__Bean_counting.ipynb @@ -0,0 +1,402 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Bean counting using connected components and more to do multiple object detection\n", + "\n", + "Now we have an image of many coffee beans and we want to count the number of coffee beans in the image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "import scipy.misc\n", + "from scipy import ndimage\n", + "import imageio.v2 as imageio" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "im_rgb = imageio.imread('data/IMG_4272.JPG')\n", + "original_gray = im_rgb[:,:,1] # take green channel to make grayscale image" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "plt.imshow(original_gray, cmap='gray')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's view our image is pseudocolor. This will be helpful for getting absolute intensity values from the images." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "plt.imshow(original_gray, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "One thing we may notice are some very small details in the image which will not help us perform our bean counting task." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "im = ndimage.gaussian_filter(original_gray, sigma=10.0)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "plt.imshow(im, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "9c04a893014b0c518a098943bb0db0f4", + "grade": false, + "grade_id": "cell-349688ed72efb990", + "locked": true, + "schema_version": 3, + "solution": false + } + }, + "source": [ + "## Question Part A\n", + "\n", + "We now want to pick a single threshold. Ideally, this will perfectly decompose our image into connected components of coffee beans and a background.\n", + "\n", + "Enter a threshold and describe why you chose this value. Use the variable `threshold`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "7b8f1f4d5f6da8b2b58cd0f6e3d1bbf4", + "grade": false, + "grade_id": "cell-6e1c2b89bd8a66f5", + "locked": false, + "schema_version": 3, + "solution": true + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "040f56c800c3a3a9d8073030823d80e1", + "grade": true, + "grade_id": "cell-4590afe7907cebe0", + "locked": false, + "points": 0, + "schema_version": 3, + "solution": true + } + }, + "source": [ + "YOUR ANSWER HERE" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "thresholded = (im < threshold).astype(np.uint8)\n", + "plt.imshow(thresholded, cmap='gray')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now, we are once again going to use [connected components labeling](https://en.wikipedia.org/wiki/Connected-component_labeling) to detect objects in our binary image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "labels, num_labels = ndimage.label(thresholded)\n", + "plt.imshow(labels, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Detour: erosion and dilation (morphological operations)\n", + "\n", + "Here we will take a small detour to demonstrate some morphological operations. You can read about morphological image processing [here](https://en.wikipedia.org/wiki/Mathematical_morphology).\n", + "\n", + "We will first perform `erosion` 30 times in a row on our thresholded image. What does the erosion do?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "eroded = thresholded\n", + "for i in range(30):\n", + " eroded = ndimage.binary_erosion(eroded)\n", + "plt.imshow(eroded.astype(np.uint8), cmap='gray')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we will perform `dilation` 30 times in a row on our thresholded image. What does the dilation do?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "dilated = thresholded\n", + "for i in range(30):\n", + " dilated = ndimage.binary_dilation(dilated)\n", + "plt.imshow(dilated.astype(np.uint8), cmap='gray')\n", + "plt.colorbar()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now what about the combination of `erosion` with `dilation`?" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "ed = thresholded\n", + "for i in range(30):\n", + " ed = ndimage.binary_erosion(ed)\n", + "for i in range(30):\n", + " ed = ndimage.binary_dilation(ed)\n", + "plt.imshow(ed.astype(np.uint8), cmap='gray')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# End of detour - no more morphological operations\n", + "\n", + "Let's view our connected components labels a bit larger." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(10, 10))\n", + "labels, num_labels = ndimage.label(thresholded)\n", + "plt.imshow(labels, cmap='jet')\n", + "plt.colorbar();" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "So, above are the labels for each connected component. Note that some individual coffee beans may be broken into two or more components and that sometimes multiple coffee beans may have been combined into one connected component." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can calculate the \"center of mass\" of each connected component." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coms = ndimage.center_of_mass(thresholded, labels, index=range(1,num_labels))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "4f893ff960cb1a4875a63476f8f3a29a", + "grade": false, + "grade_id": "cell-1213340a2a7c110e", + "locked": true, + "schema_version": 3, + "solution": false + } + }, + "source": [ + "## Question Part B\n", + "\n", + "Change the parameters (and, if you like, change the processing steps) to label each coffee bean individually. Make a plot where each bean is marked with an \"x\".\n", + "\n", + "You may like to change:\n", + "\n", + " - the image channel used (red, green, blue, an average of all channels, ...)\n", + " - gaussian blur size\n", + " - threshold used for binarization\n", + " - optionally, any morphological image operations" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "code", + "checksum": "5d08800fec54d4202136e3f732d48e59", + "grade": true, + "grade_id": "cell-9a32e77f790bb4da", + "locked": false, + "points": 2, + "schema_version": 3, + "solution": true + } + }, + "outputs": [], + "source": [ + "# YOUR CODE HERE\n", + "raise NotImplementedError()" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "editable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "72b5665ee0d8d36a5eab437cb746837a", + "grade": false, + "grade_id": "cell-5b9512404535239b", + "locked": true, + "schema_version": 3, + "solution": false + } + }, + "source": [ + "## Question Part C\n", + "\n", + "Describe what can go wrong when performing this connected components analysis to count and identify individual coffee beans. Name two possible problems and what do you see if each problem happens?" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "deletable": false, + "nbgrader": { + "cell_type": "markdown", + "checksum": "f3b280e2ac990b166198df66eeadf05c", + "grade": true, + "grade_id": "cell-b109315aba1b058b", + "locked": false, + "points": 2, + "schema_version": 3, + "solution": true + } + }, + "source": [ + "YOUR ANSWER HERE" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.10" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/exercises/release/exercise-09/data/IMG_4272.JPG b/exercises/release/exercise-09/data/IMG_4272.JPG new file mode 100644 index 0000000..2a0d84f Binary files /dev/null and b/exercises/release/exercise-09/data/IMG_4272.JPG differ diff --git a/exercises/release/exercise-09/data/short-movie20170810_182130.mp4 b/exercises/release/exercise-09/data/short-movie20170810_182130.mp4 new file mode 100644 index 0000000..9421e91 Binary files /dev/null and b/exercises/release/exercise-09/data/short-movie20170810_182130.mp4 differ