From 9edfecccde90d07be1c4b89e5ef63c138fffe9aa Mon Sep 17 00:00:00 2001
From: "gilson.m" <matthieu.gilson@univ-amu.fr>
Date: Wed, 20 Nov 2024 23:55:02 +0100
Subject: [PATCH] chg basic_python -> nb1
---
basics/python/basic_python.ipynb | 1497 ------------------------------
1 file changed, 1497 deletions(-)
delete mode 100644 basics/python/basic_python.ipynb
diff --git a/basics/python/basic_python.ipynb b/basics/python/basic_python.ipynb
deleted file mode 100644
index 35c1cc9..0000000
--- a/basics/python/basic_python.ipynb
+++ /dev/null
@@ -1,1497 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "markdown",
- "id": "0b9ae567-11a5-4565-9cae-d35a5729c4a5",
- "metadata": {},
- "source": [
- "# Short Introduction to Jupyter and Python for Scientific Calculations\n",
- "\n",
- "## Cells in Notebook\n",
- "\n",
- "Remember to double click on cells to edit them, then run them to process the text or execute the code to peform calculations (shortcut by pressing MAJ+ENTER or CTRL+ENTER on the keyboard). Change the cell to be text or code. The output of next cell is not the same whether it is text or code..."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "eae69337-7f22-465c-ab1c-58b9d34850ba",
- "metadata": {
- "tags": []
- },
- "source": [
- "1+1"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "47b066a2-d866-426a-ad2a-3f5be2f41fa8",
- "metadata": {},
- "source": [
- "Text in markdown cells can be formated to display titles, subtitles, lists, etc.\n",
- "\n",
- "Check `#` for titles, `-` for lists, `$` for mathematical expressions, etc."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "368cdad0-444c-4a86-b037-77f2d40281e9",
- "metadata": {},
- "source": [
- "## Starting with Python\n",
- "\n",
- "Now let's switch to Python basic programming. In Python, everything is an object, defined by a 'type'. For example, numbers can be integers or floats (among other types, but these are the most common types). Let's create a variable $a$."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e6fbb24b-f5fa-4734-8c31-b5d5fb96cca7",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "a = 1"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8025f157-d03b-4056-9d36-aa2ebbbb4f57",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "type(a)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1916774d-91bb-4d3e-afe9-e60baf15410e",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "b = 1.0"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "cccc4090-fa5b-4560-bfa3-159a1cb1f07e",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "type(b)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7217d639-f05c-4219-9879-11a23bbeac1e",
- "metadata": {},
- "source": [
- "Many other types are available, like strings of characters for text (use quotes like `'`, `\"` or `\"\"\"`) or booleans for logic."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "af3232b7-258d-4192-9bea-b3ae1fdbcdb2",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "c = 'char'"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "2ba66437-4943-4e6d-bd59-325f21d03755",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "type(c)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1b7e6acd-8ada-48a6-9c0b-c71376252206",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "d = True"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "3c1ccfba-6281-42af-8257-30e13edec965",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "type(d)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "3a2a06f8-9970-4686-965f-869c018adc37",
- "metadata": {},
- "source": [
- "We can add an integer and a float, but not an integer with a string... While doing that, you ask Python to convert one of the object to the \"broader\" class (e.g. int to float)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "41c10d3d-a36d-4f77-9c18-d79c22f7a8d1",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "a+b"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1f209864-faf5-48d7-9311-64671af99d59",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "type(a+b)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "f2bb7efa-0c7a-47f9-a8e8-a5acecc719ab",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "a+c"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "4397d2b6-5f33-4b28-b08d-194984d5ca4d",
- "metadata": {},
- "source": [
- "Equality between variables is strictly well-defined for integers, but only up to a tolerance for floats"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "34663a32-b3ca-4b45-a600-806b2a17a9f4",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "1+4==5"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1ec232d1-9423-49d9-8ea3-eb7af71d02bd",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "0.3==0.1+0.1+0.1"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "60f47ba6-08fd-4d91-9dbb-f66e3c27b742",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "0.1+0.1+0.1"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7f652788-b688-42cf-999d-695db129e631",
- "metadata": {},
- "source": [
- "You can now write several lines of code in a cell (note that only the last output of the last line is displayed if `print` is not used"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "68cbf7ab-c18a-419b-9a73-5bc87ec455df",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "a\n",
- "print(a)\n",
- "a"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "b39c84c6-c55f-4f14-8ef8-f9af74efa185",
- "metadata": {},
- "source": [
- "Lists are a type with successive items, that can be of various types. They can be created using square brackets and commas to separate items."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c1f2b609-84b1-4c2b-8920-f6c0322a8ebc",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "l = [a, b, c, d]\n",
- "print(l)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "a867add9-aad6-4d35-96f4-4a9ec7b1bd77",
- "metadata": {},
- "source": [
- "A list has a length corresponding to the number of items, which can be called using the position index. In Python, the first index is 0..."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "d479220f-3edf-4326-b3ee-c0e555fe27fc",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(l[1])"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e204b04b-786a-4b19-8a5e-fe468c708417",
- "metadata": {},
- "source": [
- "A list can be empty..."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "d453913b-0e92-4a8d-9fa7-e0945de9ae58",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "l = []\n",
- "print(l)\n",
- "print(len(l))"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "74ccbe9c-336c-4838-b74e-ad0242658305",
- "metadata": {},
- "source": [
- "Dictionaries are similar to lists, but indexed by arbitrary objects as 'keys'. This means that you can store for example number indexed by strings. Disctionaries can be created using curvy brackets, with the syntax `key: value`."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "67707155-57fd-43b8-ab98-241463c1ac99",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "dic = {}\n",
- "dic['weight'] = 12.3\n",
- "dic['height'] = 50.9"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "0da2e874-6a0a-4bc2-93ef-fd4ccdd5486c",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(dic)\n",
- "print(dic['weight'])"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "303959fd-869c-4a29-b8f1-730794443812",
- "metadata": {},
- "source": [
- "Check what happens when you ask for an inexisting key...\n",
- "\n",
- "We now consider loops that repeat some code for several values of a variable (here $i$). Loops are defined with the keywords `for` and `in`, while the commands to repeat are in the same indented block."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c9287678-a090-4d0a-b5a7-8ab9a3c60993",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "for i in [a, b, c, d]:\n",
- " print(i)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "56dce070-2088-40a3-ab11-df872aa033cf",
- "metadata": {},
- "source": [
- "CAREFUL HERE: indentation is part of the syntax in Python!!!"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "846f1081-00c2-4764-b42b-c06bee4ee0bc",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "for i in [a, b, c, d]:\n",
- " print(i)\n",
- " print(i+i)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "cd163079-abcb-42e7-b9a7-ee69c5b1bf53",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "for i in [a, b, c, d]:\n",
- " print(i)\n",
- "print(i+i)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "3fb34113-0858-4864-bd27-a4e3f7d21233",
- "metadata": {},
- "source": [
- "Finally, we can use tests to decide which calculation to apply, like:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c514b68a-5649-41f4-860d-b7f2c0884b7c",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "for i in [a, b, c, d]:\n",
- " if type(i)==int:\n",
- " print('{} is an integer'.format(i))\n",
- " else:\n",
- " print('{} is not an integer'.format(i)) "
- ]
- },
- {
- "cell_type": "markdown",
- "id": "5e76cc73-a114-46ef-962f-ec0a97b1ac1d",
- "metadata": {},
- "source": [
- "Above, the double equal sign is the test for equality. The `format` function is useful to format strings by inserting variables in place holders (curvy brackets).\n",
- "\n",
- "Finally, we can define a function that takes an \"argument\" as input to produce an output or result. The keyword here is `def` and the body of the function (code to be executed) is indented:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "4cb7d440-6970-4cf0-82d5-bec3427b06a5",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "def double(x):\n",
- " return x+x\n",
- "\n",
- "print(double(a))\n",
- "print(double(b))\n",
- "print(double(c))\n",
- "print(double(d))"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "315bce1c-1268-4e54-879a-ff956f9bd7ff",
- "metadata": {},
- "source": [
- "CAREFUL with functions: variables not defined inside have to be defined at the time of execution:"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "5c70e711-cb30-4517-bd1a-3535059f05e1",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "def shift(x):\n",
- " return x + e"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8b16423e-cbe2-4aa6-8e46-6e65cbb9bd66",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(shift(a))"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "16a4b173-5bac-411d-86a1-0e2129c10ba8",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "e = -3\n",
- "\n",
- "print(shift(a))"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9ae0e02a-c9b3-49aa-ac7f-579bf332f459",
- "metadata": {},
- "source": [
- "## Numpy Library\n",
- "\n",
- "To use functions in a library, we import it and call its member functions as follows. Here we create an \"alias\" or short name for numpy because it is shorter to write."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e595a780-410b-41c6-90fc-0b85bee25c76",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "import numpy as np\n",
- "\n",
- "x = np.array([0, 2, 8, -3])\n",
- "\n",
- "print(x)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7018fb84-2c5d-46cf-b466-d066f5ed7451",
- "metadata": {},
- "source": [
- " All items must be of the same type."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "37e20460-30c4-455a-9c0a-fe53149a0be0",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "type(x)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "6d0b1514-ba52-4ab6-8b5b-1c02af21bff9",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "x.dtype"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e9d0ee7d-46af-4c1d-be0b-fad654d5f1da",
- "metadata": {},
- "source": [
- "The advantage of arrays over list is that mathematical operations can be used on them (they have been made for that...)."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1b8c908e-f154-4496-b001-14f068309c3f",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "y = np.array([10, 4, 6, 3])\n",
- "\n",
- "print(x + y)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "26e9a780-4d89-4202-a218-0df183e6d888",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(x * y)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9df6a48c-73ec-405f-8dbd-084c159f327a",
- "metadata": {},
- "source": [
- "The operator `+` has a different behavior for arrays, lists and strings."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "2f02d7c3-42f7-4475-a542-5cd7918f6866",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(x.tolist() + y.tolist())"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "a9448563-65e2-41ad-bcab-369d2a4a6e53",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print('EMN' + 'online')"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "3eb9dba8-f1ab-44b5-a6c4-a077a52cefc3",
- "metadata": {},
- "source": [
- "Arrays can be constructed with arbitrary dimensions ('shape'), some operations require that they have matching shapes, see above or the scalar product below. Recall that the dot product above is the sum of the product of corresponding items:\n",
- "$$ \\sum_i x_i y_i $$"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "252e69eb-9a86-42db-bd3e-0a2d02758cc3",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(np.dot(x, y))\n",
- "print(x @ y)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "390dad88-eafc-4757-a93b-05538c021c5f",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(x.shape, y.shape)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "630dd115-7cd7-4f4a-867a-1454ef0693c9",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "z = np.array([2, 4])\n",
- "\n",
- "print(np.dot(x, z))"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "7cef99eb-c866-4090-8446-5575a1d7958a",
- "metadata": {},
- "source": [
- "Numpy has a lot of operators (like `**` for power), constructors for arrays, functions for linear algebra (search for 'linalg' in the doc), like matrix operations."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "0c81ad2b-9604-4165-8d51-be8954059fae",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print('square of 3 =', 3**2, '; 4 to the power of 1.5 =', 4**1.5)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "fda360fb-2bcd-49ab-b93f-c5bbdd36a156",
- "metadata": {},
- "outputs": [],
- "source": [
- "# array with values determined by a step, like from 0.0 included to 2.0 excluded with step 0.4\n",
- "print(np.arange(0.0, 2.0, 0.4))"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "48f23718-ed4c-4319-aabb-4dcea4e6cd60",
- "metadata": {},
- "outputs": [],
- "source": [
- "M = np.load('ex_array.npy')"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "401a7bc5-712c-4a8d-997f-30f7035e06c9",
- "metadata": {},
- "outputs": [],
- "source": [
- "M.shape"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c21e081e-d3aa-4eb4-9d18-0a4248677ce8",
- "metadata": {},
- "outputs": [],
- "source": [
- "M[0,:,:].shape"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "5d76fc18-abb5-4637-bf7e-d5a4663767ae",
- "metadata": {},
- "outputs": [],
- "source": [
- "M[[0,10],:,:].shape"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "36d5853a-1bbe-4e0a-b211-78021a485749",
- "metadata": {},
- "outputs": [],
- "source": [
- "sel_region = np.arange(50)<10\n",
- "print(sel_region)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8eec640b-5f46-4a59-a38c-ddf227349450",
- "metadata": {},
- "outputs": [],
- "source": [
- "M[:,sel_region,sel_region].shape"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e6eed839-f925-491f-a8ee-5d5be3feffef",
- "metadata": {},
- "outputs": [],
- "source": [
- "M[:,sel_region,:][:,:,sel_region].shape"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "66970bf3-e998-4a4f-ab8f-9b0a95e7349e",
- "metadata": {},
- "outputs": [],
- "source": [
- "sel_inter = np.tri(50,50,0,dtype=bool)\n",
- "print(sel_inter)\n",
- "print(sel_inter.shape)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "ce901745-3bfe-4cff-8b5a-2113d5242141",
- "metadata": {},
- "outputs": [],
- "source": [
- "M[:,sel_inter].shape"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8c17e521-a6e6-4584-b1cb-7db0fb0ab491",
- "metadata": {},
- "outputs": [],
- "source": [
- "A = np.arange(6)\n",
- "print(A)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "6f72f937-eb42-499a-996c-8e280e9fe0f2",
- "metadata": {},
- "outputs": [],
- "source": [
- "B = A.reshape([2,3])\n",
- "print(B)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "b52ad811-a7c1-49fd-9920-e938642948c2",
- "metadata": {},
- "outputs": [],
- "source": [
- "print(B.flatten())"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "59304706-9ddf-4e9e-8a76-d8b8c07d821b",
- "metadata": {},
- "outputs": [],
- "source": [
- " A.reshape([2,-1])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8909ee1b-afef-48fe-806e-86892b2e6e47",
- "metadata": {},
- "outputs": [],
- "source": [
- " A.reshape([4,-1])"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "5c9e4d1c-3b2e-406a-b752-c64ae20e0b65",
- "metadata": {},
- "source": [
- "# Pandas"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c13dfd24-25fe-495b-82dd-b30ced76a669",
- "metadata": {},
- "outputs": [],
- "source": [
- "import pandas as pd"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c6de2ac9-c8bf-4851-a83b-fc129737feae",
- "metadata": {},
- "outputs": [],
- "source": [
- "df = pd.DataFrame()\n",
- "print(df)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "f4415c80-e395-4e10-9cdb-32f87f0001ce",
- "metadata": {},
- "outputs": [],
- "source": [
- "df['session'] = np.random.permutation(np.arange(5))\n",
- "print(df)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "aa82ac0b-2af7-4fe2-8071-2cf139714d99",
- "metadata": {},
- "outputs": [],
- "source": [
- "df['value'] = np.random.rand(5)\n",
- "print(df)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "5eabb413-5f89-42a0-a58d-326db5f706ea",
- "metadata": {},
- "outputs": [],
- "source": [
- "df['type'] = ['train', 'train', 'train', 'test', 'test']\n",
- "print(df)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c8c8bd80-a9cf-4bef-b7a0-85974a7576fa",
- "metadata": {},
- "outputs": [],
- "source": [
- "sel_df = df['type']=='train'\n",
- "print(sel_df)\n",
- "print(type(sel_df))"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "03ef2a0c-f12a-47c0-ac94-7877f4285d55",
- "metadata": {},
- "outputs": [],
- "source": [
- "df[sel_df]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "2f72e8e9-8c0c-41fc-8b03-6836d21b45b1",
- "metadata": {},
- "outputs": [],
- "source": [
- "for df2 in df.groupby('type'):\n",
- " print(df2)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "e6b7b9fc-4b50-4dc0-a2b8-6bfc3c1afb26",
- "metadata": {},
- "source": [
- "## Matplotlib Library\n",
- "\n",
- "This library has functions for visualization, like line plots, scatter plots, etc. \n",
- "\n",
- "Here arange (from Numpy) provides integers from 0 to 3=4-1, the indices of the arrays."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "6fa7f0fe-2631-4ad0-ab71-6d1af30b8ef1",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "import matplotlib.pyplot as plt\n",
- "\n",
- "plt.plot(np.arange(4), x, label='x')\n",
- "plt.plot(np.arange(4), y, label='y')\n",
- "plt.legend()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "17f07d66-4a90-4c0e-9398-9d0c05861bfe",
- "metadata": {},
- "source": [
- "We can add labels, change the ofnt size, etc."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "79c71418-13df-401a-bcc0-ebb1d5a466db",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# change the default font size to 16\n",
- "import matplotlib as mpl\n",
- "mpl.rcParams['font.size'] = 16\n",
- "\n",
- "plt.scatter(x, y, marker='s', color='green')\n",
- "plt.xlabel('x')\n",
- "plt.ylabel('y')\n",
- "plt.tight_layout() # to save space in the figure, avoid labels outside, etc."
- ]
- },
- {
- "cell_type": "markdown",
- "id": "cef54c43-0165-4d74-92e1-e51b405a8046",
- "metadata": {},
- "source": [
- "The function plot can also display the markers without the line by using arguments (see the documentation https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.plot.html)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "fd829716-2f37-4844-822a-95a820515c4b",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "plt.plot(x, y, marker='s', color='green', ls='')\n",
- "plt.xlabel('x')\n",
- "plt.ylabel('y')\n",
- "plt.tight_layout()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "8146146f-0971-4f70-954c-f9457ce90f63",
- "metadata": {},
- "source": [
- "Here the graduations are linear, which is appropriate to display a vector of values that are regurlarly distributed in a linear manner between. For example, the `numpy` function `linspace` generates a givne number of such values a minimum and a maximum (see https://numpy.org/doc/stable/reference/generated/numpy.linspace.html)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "0941cad8-7916-4556-9b69-5ef813c6a5ff",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# generate 20 values between 0 and 10\n",
- "lin_x = np.linspace(0,10,20)\n",
- "print(lin_x)\n",
- "\n",
- "plt.plot(lin_x, lin_x, marker='x', color='gray', ls='')\n",
- "plt.xlabel('linear scale')\n",
- "plt.ylabel('linear scale')\n",
- "plt.tight_layout()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "1d871772-603b-4416-ab77-4f5da94642ec",
- "metadata": {},
- "source": [
- "If we want to plot values that are distributed in a different scale, this may not be appropriate anymore..."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e2ef077f-f64d-4823-8071-32ecb601ac78",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# generate 20 values between 0.01=10**(-2) and 100=10**2 distributed exponentially (linearly spaced on a log scale)\n",
- "log_x = np.logspace(-2,2,20)\n",
- "print(log_x)\n",
- "\n",
- "plt.plot(log_x, log_x, marker='o', color='gray', ls='')\n",
- "plt.xlabel('linear scale')\n",
- "plt.ylabel('linear scale')\n",
- "plt.tight_layout()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "ca813943-78af-4204-bd97-e5cc37a96569",
- "metadata": {},
- "source": [
- "Most of the 20 dots are located around the origin. We can zoom to see some of them but there is still a larger concentration around the origin..."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "576789e3-5fcd-4074-99b9-068f3e710fec",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "plt.plot(log_x, log_x, marker='o', color='gray', ls='')\n",
- "plt.xlabel('linear scale')\n",
- "plt.ylabel('linear scale')\n",
- "plt.axis(xmin=0, xmax=5, ymin=0, ymax=5)\n",
- "plt.tight_layout()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9ed61675-c545-41f0-948b-c3e75dcb32c0",
- "metadata": {},
- "source": [
- "To see the dots better, we can use a log plot that shows the points regularly distributed on the log scale."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "89dd5015-77da-4720-9f13-c130a05f7bc7",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "plt.loglog(log_x, log_x, marker='o', color='gray', ls='')\n",
- "plt.xlabel('log scale')\n",
- "plt.ylabel('log scale')\n",
- "plt.tight_layout()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "dda41aae-6c46-4b78-946f-6ebfbfedd2c0",
- "metadata": {},
- "source": [
- "For futher examples of plotting options, see tutorials: https://matplotlib.org/stable/tutorials/index.html.\n",
- "\n",
- "Now considering a function $f(t)$ of a time variable $t$, we can plot the function to then see its profile, extrema (minima, maxima), etc."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "1a3b810f-9e1b-4292-862c-7a5ebbac85ce",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# definition of the function\n",
- "def f(t):\n",
- " # calculate the result\n",
- " res = t / (1 + t**2)\n",
- " # return the result\n",
- " return res"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "61913d0c-4d79-4207-8ac0-66877473205b",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# definition of vector of time for x-axis: 20 points uniformaly spaced from -10.0 to 10.0\n",
- "vt = np.linspace(-10.0, 10.0, 20)\n",
- "\n",
- "print(vt)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "9ac07566-3009-424d-a847-313f09401735",
- "metadata": {},
- "source": [
- "Instead of passing a single argument to the function, we can pass the vector $vt$ directly and the function output is the images of the corresponding arguments in $vt$."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "9b75b5ce-6919-4f77-bf75-09c3210753b7",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(f(vt))"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "293fac9a-25bc-4214-bc7f-b9ead2771368",
- "metadata": {},
- "source": [
- "We can now plot $f(t)$ as a function of $t$ for each t in $vt$."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "cfc78baf-0503-4081-b12c-432d799602e4",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "plt.plot(vt, f(vt))\n",
- "plt.xlabel('$t$')\n",
- "plt.ylabel('function $f(t)$')"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "afbd45b9-0410-4d4e-9f0f-9ea08582dfd9",
- "metadata": {},
- "source": [
- "The function doesn't seem smooth, we don't have a sufficient resolution for the points in $vt$, let's use 100 points."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "02addb8c-488e-433b-9bd8-6eda2c465ec4",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "vt = np.linspace(-10.0, 10.0, 100)\n",
- "\n",
- "plt.plot(vt, f(vt))\n",
- "plt.xlabel('$t$')\n",
- "plt.ylabel('function $f(t)$')"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "3376050e-fbdc-4c68-adb7-e176b88fd694",
- "metadata": {},
- "source": [
- "Finally, let's find the maximum of the plotted curve (in the range given by $vt$):"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "fad332b5-b94b-4432-869e-7e52142f145d",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# index of largest element of f(vt)\n",
- "i_t = np.argmax(f(vt))\n",
- "\n",
- "print(i_t)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "71b5a36d-86ce-47bf-ac8a-faaddac6c68e",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "print(vt[i_t])"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "09824ea6-af10-40fc-ac6f-56575a447e53",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "plt.plot(vt, f(vt), label='curve')\n",
- "plt.plot(vt[i_t], f(vt[i_t]), 'xr', ms=10, label='max') # xr stands for red cross, ms for marker size\n",
- "plt.legend(fontsize=12) # to show labels associated with plotted elements\n",
- "plt.xlabel('$t$')\n",
- "plt.ylabel('function $f(t)$')"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "8183cccf-c848-4336-8117-c2d62295c526",
- "metadata": {},
- "source": [
- "## Pandas and Seaborn Libraries\n",
- "\n",
- "Pandas is a library to store data in various formats in `DataFrame`, which can seen as an extension of arrays with more flexibility.\n",
- "\n",
- "Seaborn has plotting functions like Matplotlib that is conveniently interfaced with `DataFrame` objects."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "2c2236dd-f3c0-499d-b8a1-a9416403637c",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# import libraries\n",
- "import pandas as pd\n",
- "import seaborn as sb"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "5b50b5a8-81bc-4895-b8d0-62a7665c85f0",
- "metadata": {},
- "source": [
- "In a `DataFrame`, data are organized in \"columns\" that each have a title or label. Note that elements can be of various types. Check the documentation for further reference: [https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "cf2e74a2-7ac5-4638-a827-b2104a29e12a",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "df = pd.DataFrame(columns=['type', 'value'])\n",
- "df['type'] = ['low', 'high']\n",
- "df['value'] = [0.1, 10.4]\n",
- "\n",
- "print(df)"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "f4801e0e-8555-4446-8346-8d5138d0a294",
- "metadata": {},
- "source": [
- "The integers at the beginning of each row is the index of the row. Elements from rows (all columns) can be accessed from the index with the function `iloc`. The columns can be retrieved using the corresponding label."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "e17d07c1-57f6-4bed-b0ed-254e27f6922c",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "df.iloc[1]"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "385a4deb-8c67-4d40-8490-688cf61cc7e4",
- "metadata": {},
- "source": [
- "Once defined, data frames of numerical values can be used directly with seaborn functions, simply specifying the columns used for the plotting. In other words, constructing the data frame correctly may take a bit of lines of code, but then the plotting is made easier. Check [https://seaborn.pydata.org/tutorial/introduction.html](https://seaborn.pydata.org/tutorial/introduction.html) for first steps."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c975ef4a-900f-42be-965b-02d943a6219d",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# create a data frame with normally distributed numbers\n",
- "df = pd.DataFrame(columns=['x', 'y'])\n",
- "df['x'] = np.random.randn(20)\n",
- "df['y'] = np.random.randn(20) * 2.0\n",
- "\n",
- "# scatter plot\n",
- "sb.scatterplot(data=df, x='x', y='y')\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "13223b82-833f-4b0d-8cb2-89bedf7127e5",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "df['type']"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "92094f00-844d-484f-a7bf-61dd0a855663",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# add a column 'time'\n",
- "df['time'] = np.arange(20)\n",
- "print(df)\n",
- "\n",
- "# \n",
- "sb.lineplot(data=df, x='time', y='x', color='red', label='x')\n",
- "sb.lineplot(data=df, x='time', y='y', color='blue', label='y')\n",
- "plt.ylabel('value')\n",
- "plt.show()"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "89592b21-e0e2-4a8b-a7ef-823ad6e21b3c",
- "metadata": {
- "tags": []
- },
- "source": [
- "Note that some warnings may be displayed concerning seaborn, they should be fixed in future versions. You can ignore them.\n",
- "\n",
- "To represent distributions, `violinplot` and `swarmplot` are useful tools to visualize them with details of the samples. Here we have several samples of three classes, denoted by integers in column 'type'."
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "c9109349-24a2-4753-a27f-4f485e7a0c9b",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "# create data frame with two columns, wifor each sample its 'type' and 'value'\n",
- "df = pd.DataFrame(columns=['type', 'value'])\n",
- "# types of the 30 samples\n",
- "v_type = np.zeros([30], dtype=int)\n",
- "v_type[10:20] = 1\n",
- "v_type[20:30] = 2\n",
- "df['type'] = v_type\n",
- "# values\n",
- "v_val = np.random.randn(30)\n",
- "v_val[10:20] += 1.0 # add an offset for the samples of class 1\n",
- "df['value'] = v_val\n",
- "\n",
- "print(df)"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "d1af7728-0dab-4e9f-a6ac-20052b9af562",
- "metadata": {
- "tags": []
- },
- "outputs": [],
- "source": [
- "sb.violinplot(data=df, x='type', y='value')\n",
- "sb.swarmplot(data=df, x='type', y='value', color='black', size=10)\n",
- "plt.show()"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.11.7"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
--
GitLab