Introduction to programming in engineering Syllabus G.H.P. Campmans M. Pezij July 12, 2022 Preface This reader is part of the Introduction to Programming in Engineering course. This course is part of the Civil Engineering programme of the University of Twente. The course focuses on introducing the concepts of programming for engineering students using the Python programming language. No prior knowledge of programming is required. This course is linked to the project of the Water Management module (Blue Nile), so the applications and advantages of programming should become self-evident. We would like to thank Georgi Nikolov and Frank Hemme for their contributions to this syllabus. This course material is compiled specifically for educational uses, and used various online sources such as tutorialspoint.com, stackoverflow.com and realpython.com. 3 4 Contents Preface 3 1 Introduction 1.1 Automating your work as a civil engineer 1.2 Programming . . . . . . . . . . . . . . . . 1.3 Why Python? . . . . . . . . . . . . . . . . . 1.4 Learning goals . . . . . . . . . . . . . . . . 1.5 Reading guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 7 7 7 8 2 Getting started 2.1 Introduction . . . . . . . . . . . . . . . . 2.2 Download and install Python in six steps 2.3 Starting your Python environment . . . 2.4 Running your first Python code . . . . . 2.5 Spyder and scripts . . . . . . . . . . . . . 2.6 Synthesis . . . . . . . . . . . . . . . . . . 2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 9 9 9 10 10 13 13 3 Variable types 3.1 Introduction . . . . . . . . . . . 3.2 Variable names . . . . . . . . . . 3.3 Variable types . . . . . . . . . . 3.4 Keywords and built-in functions 3.5 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 15 15 16 19 20 . . . . . . . . . . . . . . . . . . . . . . . . . 4 Numerical operations 21 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.2 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 5 Conditional statements and loops 5.1 Introduction . . . . . . . . . . 5.2 Conditional statements . . . . 5.3 Logical operators . . . . . . . 5.4 While loop . . . . . . . . . . . 5.5 For loop . . . . . . . . . . . . . 5.6 Synthesis . . . . . . . . . . . . 5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 23 23 25 26 27 28 28 6 Importing external packages 6.1 Introduction . . . . . . . . . . 6.2 Packages . . . . . . . . . . . . 6.3 Use Python code of a package 6.4 Advanced importing . . . . . 6.5 Synthesis . . . . . . . . . . . . 6.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 31 31 31 32 32 32 5 7 Read and write data 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . 7.2 Files . . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 File paths and navigation . . . . . . . . . . . . . . 7.4 Open and close data files . . . . . . . . . . . . . . 7.5 Use existing tools to read and write files: pandas 7.6 Synthesis . . . . . . . . . . . . . . . . . . . . . . . 7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 35 35 36 37 38 40 41 8 Creating figures 8.1 Introduction . . . . . . . 8.2 Pyplot . . . . . . . . . . . 8.3 Plotting your first figure 8.4 Plot customization . . . . 8.5 Synthesis . . . . . . . . . 8.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 43 43 44 44 45 45 9 Functions 9.1 Introduction . . . . . . . . . . . . . . . . 9.2 Why functions are useful . . . . . . . . . 9.3 Calling a function . . . . . . . . . . . . . 9.4 Creating a function . . . . . . . . . . . . 9.5 Use functions to break down a problem . 9.6 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 47 47 47 47 49 49 10 Debug code and reading error messages 10.1 Introduction . . . . . . . . . . . . . . 10.2 Errors . . . . . . . . . . . . . . . . . . 10.3 Unexpected behavior and bugs . . . 10.4 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 51 51 52 52 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Modular programming using packages, modules and functions 53 11.1 Creating a module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 11.2 Packages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 11.3 Path locations suitable for packages and modules . . . . . . . . . . . . . . . . . . . . 55 12 Solving nonlinear equations 57 12.1 Solving systems of equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 13 Numerical integration of ordinary differential equations 61 13.1 Solving systems of ordinary differential equations . . . . . . . . . . . . . . . . . . . . 62 13.2 Numerical errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 14 Exercises combining previous chapters 15 Answers to the exercises 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2 Getting started . . . . . . . . . . . . . . . . . . . . . . . . . 15.3 Variable types . . . . . . . . . . . . . . . . . . . . . . . . . 15.4 Numerical operations . . . . . . . . . . . . . . . . . . . . . 15.5 Loops and conditional statements . . . . . . . . . . . . . . 15.6 Importing external packages . . . . . . . . . . . . . . . . . 15.7 Read and write data . . . . . . . . . . . . . . . . . . . . . . 15.8 Creating figures . . . . . . . . . . . . . . . . . . . . . . . . 15.9 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.10 Numerical integration of ordinary differential equations 15.11 Exercises combining previous chapters . . . . . . . . . . . 6 67 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 71 71 71 72 72 73 73 74 75 75 76 Chapter 1 Introduction 1.1 Automating your work as a civil engineer Becoming a civil engineer requires you to be able to process and analyse all sorts of data. Performing these tasks manually is fine for small problems. However, as you move towards larger datasets, manually processing and analysing data will become inefficient and time-consuming. Many engineerings work with large datasets. Programming skills will help them (and you!) automating data processing, data analysis, and data visualization, enabling you to work faster and more efficiently. 1.2 Programming Programming can be defined as a way to instruct a computer system to perform various tasks. Instructing the computer means that you provide the computer with a set of instructions. This set is generally based on a specific programming language. A programming language is a coded language which instructs a computer to perform tasks. Generally, the code is saved as a program (also known as a script). This reader will show you how to provide the set of instructions in a script. We will provide short scripts in the reader. An example of a script is: 1 2 # This is a Python script! print(’This is a Python script ’) 1.3 Why Python? The author of the Python programming language is the Dutchman Guido van Rossum. He created Python in 1989. The current Python version is 3.9.7, which we will use in this course. In recent years, Python has become one of the most used programming languages in the world. Python is open-source, which means you can install and use the software for free. Additionally, the Python community offers an enormous set of free tools and libraries that can help in your project, such as tools for visualization, data processing and machine-learning/artificial intelligence. Generally, Python is considered as a relatively easy programming language to learn due to its clear syntax and readability. The syntax can be interpreted as the spelling and grammar of a programming language. 1.4 Learning goals After this course, you should be able to: • write and run your own scripts/programs; • explain how existing scripts/programs work; • debug code and understand error messages; 7 • use a script to read and write various data files; • perform basic data analysis; • create and save figures; • use conditional statements; • independently find solutions (online). 1.5 Reading guide The next chapters will focus on introducing you to various aspects of programming. Each chapter will end with exercises to practice the newly study material introduced. First, we will help you get started, so you can run your own code. Then, variable types will be introduced. We will subsequently move to various numerical operations, after which we will focus on loops and conditional statements. In addition, we will show you how to use the enormous set of free tools and libraries provided by the Python community. Also, we will focus on topics which are important for civil engineers. We will show how you can read and write various data types, how to create and save figures, and how to use functions. Independent programmers are able to debug their own code and identify bugs. At the end of the course, we will show you how to debug your code, understand the Python error messages, and how to look for solutions on the internet. Finally, exercises (and answers) are provided that combine the study material of all chapters of this reader. 8 Chapter 1 Chapter 2 Getting started 2.1 Introduction This section will help you get started with Python. Among others, you will learn how to download, install and use Python. 2.2 Download and install Python in six steps You can run Python scripts after installing the Python software on your laptop or desktop. Often, you will need additional libraries to enhance your Python installation. These libraries are often bundled in Python distributions. A popular distribution is Anaconda (for Windows, Mac, and Linux). We will use a variant of Anaconda called Miniconda, which provides us with a basic installation of Python. We will add some well-known libraries to our installation, such as NumPy and Matplotlib, which will be addressed later in this. Please follow these steps to setup your Python environment: 1. You can get Miniconda by downloading the installer at https://docs.conda.io/en/latest/ miniconda.html. Download the Miniconda3 Windows 64-bit version if you are working on a Windows laptop/desktop. If you do not have a 64-bit system, please use the 32-bit installer. 2. Install Miniconda using the installer. 3. Open Anaconda Prompt (by typing Anaconda Prompt in the start menu). If Miniconda has been installed correctly, you will see the following: 1 (base) C:\ Users\Pezij > 4. Next, move the provided file python environment.yml to the directory that is listed in the previous step (in the example case C:\Users\Pezij, this directory is probably different on your laptop or desktop). The environment file contains a description of all additional libraries we want to install. 5. Then, run the following command in Anaconda Prompt: 1 conda env create -f python_environment .yml Type Y when requested. 6. Let the download and installation process finish. Voila, you now have a working Python environment for this course and are ready to go! 2.3 Starting your Python environment Every time you want to start your Python environment, you will have to open Anaconda Prompt. By default the ‘(base)’ environment is active. For this course, you have to use the python IPE environment which contains Spyder. To start this environment, type: 9 activate python_IPE 1 You can check whether the Python environment has been started correctly if the word python IPE is showing in front of your current directory: ∗ 1 (python_IPE) C:\ Users\Pezij > 2.4 Running your first Python code In this section you can run your very first line of Python code directly from the Anaconda Prompt terminal. Please start your Python environment using Anaconda Prompt as shown in section 2.3 and type: 1 python You will now start the Python interpreter in which you can type all your Python code: 1 2 3 Python 3.8.8 (default , Apr 13 2021 , 15:08:03) [MSC v.1916 64 bit (AMD64)] Type "help", "copyright", "credits" or "license" for more information. >>> The very first code most programmers write in a new programming language is Hello World!. This code displays the words “Hello World!”. You can write this code by typing the following in the Python interpreter and press Enter. 1 >>> print("Hello World!") If you typed in the line correctly you will see the text “’Hello World!” in the terminal. Congratulations, you just ran your first Python code! Note that we will not use python directly via the terminal in this course. However, it is good to know that this is a possibility. Later you might want to use this to quickly run a program that you programmed in the spyder environment. Likely you will only use the spyder environment to write code and to run programs. You will learn how to do so in the next section. You can exit the Python interpreter using the exit command: 1 >>> exit () Then you will return to the terminal environment, and you have the terminal reader to start spyder, which will be explained in the next section. 2.5 Spyder and scripts The previous section showed how to code in Python using the command line. However, you cannot save your Python code using the command line. Therefore, we will introduce two key aspects which help us to program more efficiently: coding environments and scripts. 2.5.1 Integrated Development Environments Instead of programming in the command line, we will use an Integrated Development Environment (IDE) in this course. IDEs are text editors that are used for programming, providing us with a graphical interface in which we can code. IDEs provide us with many tools for programming, for example: • IDEs can run programming code in the same interface; • IDEs often provide suggestions to improve your code; ∗ The python IPE environment contains all required libraries for this course. You might need other libraries during other courses, for which you can install your own environment. You can find all information on installing other libraries and new environments at https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments. html#creating-an-environment-with-commands. 10 Chapter 2 • IDEs provide tools to identify errors and bugs in your code. This process is commonly known as debugging. We will use the Spyder IDE in this course. You already installed Spyder using the pythonenvironment.yml file. If you followed the instructions from the previous section you already have the ‘(python IPE)’ environment active in the terminal. But if you start Anaconda Promt, remember to first activate the conda environment ‘(python IPE)’ before you start Spyder using: 1 (base) C:\ Users\campmansghp > activate python_IPE Next, type spyder in Anaconda Prompt to start Spyder: 1 (python_IPE) C:\ Users\campmansghp > spyder Exercise 2.1. After having followed the previous steps, close spyder and anaconda prompt. Familiarize yourself to start Spyder with the steps above. If you are having trouble starting the Spyder environment do not hesitate ask the teacher during the lectures! Once Spyder has started, you should see the interface as shown in Figure 2.1. Three main panels are shown. 1. The editor covers the left part of the screen. The editor can be used to type your Python code and shows your scripts. Scripts are introduced in section 2.5.2; 2. The upper right panel contains the Help tab, the Variable Explorer, Plots and Files. The Help tab provides you with information on your code. The Variable Explorer allows you to interactively browse and manage the objects generated by your code. The Plots tab shows the figures you will create later this course. Last, the Files tab gives an overview of all files located in the folder you are currently working in; 3. The lower right panel is the IPython console, which is similar to the command prompt discussed in section 2.4. The scripts you run in the editor will be executed in this console. Also, you can use the console to enter lines of Python code. Figure 2.1: The interface of Spyder. On the left is the editor in which scripts can be edited. The upper right panel shows the Help tab, Variable Explorer, Plots and Files. The lower right panel is the console. Python code can be executed in the console, such as the print command shown in the figure. Note that Spyder contains the console, editor and additional information tabs in one convenient interface. Also note that the text in the editor has various colours for different code fragments. Chapter 2 11 Spyder conveniently displays different variable types in different colors. The various variable types will be introduced in chapter 3. 2.5.2 Scripts You can use scripts to save your code. Scripts are text files containing the code statements that your program should carry out. The editor panel of Spyder is used to create and execute scripts. In this section you will learn how to create and execute your first script. Creating a script you can create and save a Python script in Spyder by: 1. Go to the File menu in the top left of your screen and click on New File; 2. Save the script using the File menu and clicking on Save 3. Give your script a descriptive filename and save them in a proper folder on your computer, so you can easily find your script. For example, a logical folder for this course would be C:\Users\<user> \Documents\Courses\Intro to Programming in Engineering\Scripts\; The script created in exercise 2.2 might be saved as ex 2 1 print a string.py. 4. Spyder adds by default a short descriptive section in the script indicating the author and file creation date. You can change this description. However, do not remove the first line, as this line allows Spyder to read your script: 1 # - * - coding: utf -8 - * - 2 Running a script Now that you have created your first python script, let’s find out how you can run your script: 1. First, we have to add some code. Let us use the code we used before. Add the code of the Hello World example to your script: 1 print("Hello World!") 2 2. Please remember to save your script often, so you will not loose your progress; 3. Next, the Run file button can be used to execute/run the script. The ’Run File’ button is indicated by the red circle in Figure 2.1. Alternatively the F5 key can be used on a Windows laptop to run the script. Please note that the F5 key is not present on the Chromebooks which will be used during the exam; 4. When running a script for the first time, the window shown in Figure 2.2 will appear. The default settings as shown in this figure are okay to use. 5. Run your script now and compare the result in the console with the result you got in the command line. Great, now we know how to open Spyder, create a script, and how to run the code of the script. Before we move to the next chapter and focus on variable types, we will introduce comments. 2.5.3 Comments Often we want to provide an explanation of the code in scripts. We can put statements that are not executed in a script using comments. Commenting is important for programmers. Not only do you help yourself in writing understandable code, you also provide documentation for other users of your script. A command is simply added by adding the # sign in front of your line: 1 2 3 # - * - coding: utf -8 - * """ Created on Tue Aug 24 14:38:29 2021 12 Chapter 2 Figure 2.2: Settings for running your first script in Spyder. 4 5 6 7 8 @author: CampmansGHP Note that upon creating a new script a comment block is automatically created This is another way comments can be added to a script """ 9 10 11 12 print("Hello world!") # this line of code prints the text ’Hello world!’ # A ’#’ sign indicates the start of a comment. The comment ends the end of the line print(’on the next line after a comment code can be written ’) Another option to place comments in a program is to create a comment block, which is starting and ending with three double quotes (”””). These aspects provide us with all the tools necessary for this course. We are now ready to start programming! 2.6 Synthesis You will now be able to start programming in Python yourself. Let’s have a look at the various variable types that can be used in Python in the next chapter. 2.7 Exercises Exercise 2.2. Create a new script in Spyder, and write a line with print(’some text’) in the script. Run the script by clicking the green triangle, or pressing the F5 key. Exercise 2.3. The script in the previous exercise should have printed the string (the characters between apostrophes) in the console. A script executes code line by line. Add another print statement to your script, run it and see that indeed first the first string is printed, followed by the second string. Exercise 2.4. Now add the # symbol in front of a line with a print statement. What happens to the output of your program when you run it again? Chapter 2 13 Exercise 2.5. Add the following command between two print statements that print a string: 1 input(’Press enter in the console to continue ’) Run the program and test what this line of code does. 14 Chapter 2 Chapter 3 Variable types 3.1 Introduction Python can be used as a interactive calculator. For example, the following code will lead to the number 4 as output: 1 2+2 2 3 # output = 4 This line is a typical example of an expression in Python. Expressions can consist of values (such as 2) and operators (such as +). Expressions always lead to a single value. We often want to use such an intermediate result at a later moment in the script. We can store (or assign) the result of an expression in a variable. In general, variables are often used in programming. Various variable types (or data types) exist in Python. This chapter will introduce the most common types used. The standard types are: • Number • String • List • Tuple • Dictionary 3.2 Variable names Variables can have almost any name containing letters, underscores and numbers. A variable name must start with a letter, either capital or lowercase. Starting a variable name with an underscore is possible, but it is not recommended, as it does have specific effects on variables that are out of scope of this course). If we would make a script in which the number of bridges must be assigned, we could use the following variable declarations: 1 2 3 4 5 6 7 N = 5 n = 5 Nbridge = 5 Number_of_bridges = 5 NumberOfBridges = 5 _N = 5 # not recommended . puppies = 5 # not recommended. Above, seven different variables are defined which all have the same value (5). Note that when developing a program, variables can have any name. However, typically logical and short names 15 are preferred. A program where the number of bridges are called ‘puppies’ will work just fine, as long as the script uses this name consistently. There are some conventions that Python users tend to follow. Last, variable names are not allowed to start with a number or a special character. 1 2 3 5N = 5 @ = 5 5 = 5 3.3 Variable types As mentioned earlier, various variable types exist. The standard types are Numbers, Strings, Lists, Tuples and Dictionary, but various other types exist e.g. from other packages (as we will see in Chapter 6). In the previous section we have seen which variable names are valid. 3.3.1 Numbers Two types of numbers exist in Python: integers (whole numbers) and floating point numbers (decimals): 1 2 N=5 # integer print(type(N)) The function type() can be used to get the variable type. In this case the code fragment above will output <class ’int’>. This means that the variable N is of the type integer, i.e. one of the number types. The variable N is of the type integer because we assigned an integer value to it. Non integer numbers are known as floats, for instance: 1 2 N=5.5 # but also N=5.0 print(type(N)) will return <class ’float’>. For many applications integers and floats are sufficient. In some applications, complex numbers may be encountered ( for instance if you type print(type((-1)**0.5)) you will see <class ’complex’> ). 3.3.2 Strings A type that we have already encountered early on is the string type. Strings are set ofs characters enclosed in quotes. Either single or double quotes can be used (but not mixed, i.e. once you start a string with single quotes it should also end with a single quote). 1 2 3 var1 = ’String with single quotes ’ var2 = "String with double quotes" print(type(var1),type(var2)) Both variables var1 and var2 are of the type <class ’str’>, i.e. string. Strings are frequently used to print text as output of a program, or for instance to provide a directory or filename for a file which needs to be opened. The latter will be explained in section 7. Note that one of the methods to make a comment block, shown in Section 2.5.3, is actually also a string type that allows for multiple lines, however when used as a comment, there typically is no variable assigned to it. 1 2 3 4 var3=""" Note that the comment block is actually a multi -line string type """ print(type(var3)) 3.3.3 Lists Lists are a basic data structure in python. Lists contain multiple elements (e.g. strings or numbers). A list is defined using square brackets. Each element is separated by a comma. For instance the code fragment below defines three different lists. 16 Chapter 3 1 2 3 list1 = [1,2,3] list2 = [’one’,’two’,’three ’] list3 = [’one’ ,2,3.0] Each of the elements can have a different type. An element of a list can also be another list, e.g.: 1 list4 =[[1 ,2 ,3] ,[’one’,’two ’,’three ’],[’one ’ ,2 ,3.0]] We can access elements of a list using the index of the list element and the square brackets notation. It is important to realize that the first index in Python is always zero instead of one. If multiple lists (or other variable types within a list) are used, square brackets can be used sequentially to access firstly the element of the outer list, and subsequently the element of the inner list. 1 2 print(list1 [0]) # print the first element of list1 print(list4 [0][1]) # from the first list within list4 print the second element Note that lists can be modified, also called mutable. Elements can be added to a list using the append() function, deleted using the del() or remove() function, or modified by assigning a new value to an element using the index of that element, e.g.: 1 2 3 4 5 6 7 8 var1 = [1,2,3] # define a list print(var1 ,’before changes ’) var1.append (4) # add an element print(var1 ,’after appending 4’) del(var1 [2]) # delete an element print(var1 ,’after deleting the element with index 2’) var1 [0]=0 # modify an element print(var1 ,’after modifying an element ’) To access multiple elements at once, we can provide a range of indexes. The result is a new list with a subset of the original list. This process is known as slicing.. In the code example below, some examples are given how ranges of indices can be used to retrieve specific parts of a list. Indices can be positive and negative. Positive indices start counting from the left side of a list, while negative indices count from the right side of a list. Please keep in mind that the number zero is used to access the first element of a list from the left side. 1 2 3 # 0 1 2 3 4 5 list1 = [0, 1, 2, 3, 4, 5] # -6 -5 -4 -3 -2 -1 index counting from the left index counting from the right 4 5 6 7 8 # to access an indidual element of the list you can use its index print(list1 [3]) # prints the element with index 3 # sometimes you may want to access elements counting from the end of the list print(list1 [ -2]) # print the second element from the end of the list 9 10 11 12 13 14 15 # To access a range print(list1 [0:3]) # print(list1 [4:6]) # print(list1 [:]) # print(list1 [3:]) # print(list1 [:3]) # of elements from a print the elements print the elements print the elements print the elements print the elements list with with from from from so called slicing can be used: index 0 untill (but not including) 3 index 4 untill (but not including) 6 start until the end 3 until the end start untill (but not including) 3 16 17 18 19 20 # The negative index can also be used here. print(list1 [: -1]) # print the entire list exept the last element print(list1 [ -3:]) # print the last 3 elements of the list print(list1 [1: -2])# combinations of counting from left and right can be used 21 22 23 24 # Indices are not allowed to exceed the list length. This results in an IndexError. print(list1 [6]) # The sevents element from the left does not exist print(list1 [ -7]) # The sevent element from the right does not exist either Exercise 3.1. Create a list with the integers 5,4,3,2,1,0. Print the first and last element of this list. Chapter 3 17 Exercise 3.2. Create a list with 10,20,30,40,50,60 as its elements. Then create a second list that is a subset of the first one, such that the new list only includes 20,30,40. Do not create a new list with those values, but use slicing. Exercise 3.3. Try if you can also apply the slicing technique to a string. Define a string and print out the first 4 and the last 4 characters from that string. Exercise 3.4. Create a list with the following elements: 12,22,313,10,6,78,69,420,11,88,100,120,300,500,12,3,4,1,55,23,21,65,34,54. Now sort it in ascending order and print the result. Tip: look online how you can easily sort the elements of a list! Exercise 3.5. Insert in the list from exercise 3.4 the number 1000 so that it is in the 4th position and print the result. Then sort it and print the sorted list. 3.3.4 Tuples Tuples are quite similar to lists in the sense that they also contain elements separated by comma’s. However, tuples are defined using round brackets () rather than the square brackets [] for lists. 1 2 3 4 tup1 = (1,2,3) # this is a tuple list1 = [1, 2, 3] # this is a list tup2 = (’one’,’two’,’three ’) tup3 = (’one’ ,2,3.0) Also similar to lists, elements of a tuple can be of any type. So, a tuple can contains others tuples and lists: 1 tup4 =((1 ,2 ,3) ,[’one’,’two’,’three ’],(’one’ ,2,3.0)) Accessing elements of tuples is done in the same way as lists, using square brackets and the index of the element. 1 2 print(tup1 [2]) # print the third element of tup1 print(tup4 [0][1]) # from the outer tuple take the first element , then from the first inner tuple take the second element and print that value What is the difference between lists and tuples? Unlike lists, tuples are immutable, so you cannot change the elements of a tuple once defined: 1 tup1 [2] = 4 # trying to change the value of the tuple -element with index 2 will give an error However, the values of a list within a tuple can be modified. 1 2 3 print(tup4 [1][2]) tup4 [1][2]= ’four ’ print(tup4 [1][2]) You may wonder where you would use a tuple. A common use of tuples is to retrieve multiple outputs from some function. 1 2 3 4 def some_function_with_two_outputs (): variable1=’some text ’ variable2 =7 return variable1 , variable2 5 6 output = some_function_with_two_outputs () 18 Chapter 3 7 8 9 10 11 print(type(output)) var1 = output [0] var2 = output [1] print(var1) print(var2) Function definition will be explained in Chapter 9. Exercise 3.6. Make a tuple and try if you can take a slice of that tuple in the same way as you can for a list. Exercise 3.7. Make a tuple and try to modify one of its elements. 3.3.5 Dictionaries Lists and tuples use indices starting at 0 to keep track of the positions of the elements. Dictionaries are another type of a way to store sequences of data. Dictionaries use unique keys to identify elements of a dictionary. A dictionary is a list of items. Each item consists of a key and its value, separated by a colon (:). A key can be a number, a string, or a tuple. The values in the dictionary can be of any type. Importantly, the key-value pairs are not ordered in a dictionary. Curly braces are used to define a dictionary. Examples of dictionaries are: 1 2 3 4 dict1 = {’fruit ’:’apple ’,’quantity ’:5,’price(euro)’:0.50} dict2 = {’key’:’key of type string ’ ,2:’key of type number ’ ,(1,0):’key of type tuple ’} dict3 = {’Countries ’:{’NL’:’The Netherlands ’,’BE’:’Belgium ’,’DE’:’Germany ’}, ’Capitals ’:{’NL’:’Amsterdam ’,’BE’:’Brussels ’,’DE’:’Berlin ’}} To access a dictionary element the key needs to be provided between square brackets: 1 2 print(dict2 [(1 ,0)]) print(dict3[’Capitals ’][’NL’]) Exercise 3.8. Make a grades register for John, Peter and Jessica for the following subjects: Math, Structural Mechanics, Fluid Mechanics. Make a dictionary which contains the grades for the students and courses. Then, display the grade of Peter for Fluid Mechanics. The grades are: • John: Math, 5; Structural Mech, 6; Fluid Mechanics, 7; • Peter: Math, 6; Structural Mech, 9; Fluid Mechanics, 8; • Jessica: Math, 8; Structural Mech, 4; Fluid Mechanics, 9; 3.4 Keywords and built-in functions Probably you have already seen several words getting specific colors in the Spyder editor. By default Spyder displays keywords in orange and built-in functions in purple. These words have a special meaning in Python. Keywords cannot be used as a variable name as they are reserved words in Python (even though they are composed of letters, as explained in Section 3.2). Examples of keywords are for, del, import. Built-in functions, are not reserved words. This means that builtin functions are valid variable names. However, in general it is not advised to use these names as variables as you might experience unexpected behavior of your programs. Some examples of built-in functions are max, min, type. The type of problem that you might run into when using a built-in function as a variable name is as follows: Chapter 3 19 1 2 3 4 print(type(max)) # prints: <class ’builtin_function_or_method ’> max = max (1 ,2) print(type(max)) # prints: <class ’int ’> max (1 ,5) # this will cause an error , because the variable ’max ’ has priority over the builtin function ’max ’ we now try to cal the variable as a function , which is not possible for the type integer that is assigned to the variable max Note that in general you do not need to worry too much about this. However, when you do find yourself getting an unexpected error messages for a function, then you might have to check if you named a variable the same as a (built-in) function. 3.5 Synthesis You are now introduced to the most common variable types used for programming in Python. The next chapter will focus on numerical operations. 20 Chapter 3 Chapter 4 Numerical operations 4.1 Introduction You will likely perform many calculations as a civil engineering. Programming will allow you to perform many calculations in a small amount of time. Operators were already introduced in Chapter 4. Table 4.1 lists the most common operators in Python. basic numeric operations are written as follows: 1 2 3 4 5 6 7 8 9 print (2+3) print (2 -3) print (2 * 3) print (2/3) print (2 ** 3) print (2 ** 0.5) print (2+3 * 2) print ((2+3) * 2) print(abs(-2)) # # # # # # # # # 2+3 2-3 2 times 3 2 devided by 3 2 to the power 3 2 to the power 0.5 is the same as taking the square root of 2 2 plus 3 times 2 (2 plus 3) times 2 absolute value of -2 In python the same rules apply where it comes to the sequence in which operations are performed, i.e. multiplication is performed prior to addition. Most of the times in programming you will write down an equations with variables, such that you can quickly obtain the outcome of an equation for different parameter values. Also you will likely come across mathematical functions such as sin, cos and exp. These functions are not by default defined in python. However, e.g. the numpy packages does define these – and many more – for us. (More on importing packeges in Chapter 6.) 1 2 import numpy x = 3 3 4 5 6 7 y=2-x y=2+x y=2 * x y=2/x Table 4.1: The most common operators in Python Operation Addition Subtraction Multiplication Division Exponentation Modulo Operator + * / ** % 21 8 9 10 11 12 13 14 y=x ** 2 y=abs(x) y=numpy.sin(x) y=numpy.cos(x) y=numpy.tan(x) y=numpy.arctan(x) y=numpy.exp(x) The above list is by no means extensive. Many more mathematical functions exist. In general when you don’t know how to use a function you can search for the function name online and add ’python’ to the search terms and likely you will find the function and how to use it. 4.2 Synthesis 4.3 Exercises Exercise 4.1. Create a program that computes the volume of a cylinder. 22 Chapter 4 Chapter 5 Conditional statements and loops 5.1 Introduction The previous chapter showed us how to use Python for solving simple mathematical expressions. An advantage of programming is that we can use the program for decision-making using conditionals. Conditionals often contain code sections which only run when the provided condition is true or false. In addition, as a programmer, you will encounter situations where you will want to use part of your code repeatedly in a script. You could write the same line of code again, but you can also make use of loops. Understanding loops is an important skill for a programmer, as they can be used to execute code repeatedly. Using loops prevents the need of copy and pasting lines of code, which will make the program more clearly readable. Moreover, when you need to make changes to the code you will only need to change them in one place, instead of having to make changes in multiple locations in the code. The latter is a large risk for making mistakes if you make changes in one place but forget to do this in some parts of the code. This chapter will introduce the concepts of conditional statements and loops. 5.2 Conditional statements You will likely to encounter a situation in programming in which a condition has to be tested. Figure 5.1 shows an example. The operator of the bridge in the figure has to assess the height of each ship approaching the bridge. If the height of the ship is larger than the bridge deck, he has to raise the bridge deck. The municipality now has replaced the operator, as they have installed a camera which can observe the height of approaching ships. Simultaneously, a Python script has been written to automatically operate the bridge deck using the observations of the camera. The script is run every time a ship is approaching. The camera is passing the height of the ship to the Python script. Both the bridge height (h bridge) and the detected ship height (h ship) are variables in the script. We can use conditionals to assess whether the bridge deck has to be raised. The following logical conditions can be used in Python to assess the situation: 1 2 3 4 5 6 h_bridge h_bridge h_bridge h_bridge h_bridge h_bridge == != < <= > >= h_ship h_ship h_ship h_ship h_ship h_ship # # # # # # Check whether ... the bridge ... the bridge ... the bridge ... the bridge ... the bridge the bridge height is equal to the ship height. height is not equal to the ship height. height is less than the ship height. height is less than or equal to the ship height. height is greater than the ship height. height is greater than or equal to the ship height The result of a logical conditions is known as a Boolean. A boolean is a data type with two possible values: either true or false. Please note that the Boolean operators in Python start with a capital letter: True and False. As an example, the result of the following conditional is false, because the ship height is smaller than the bridge height: 1 h_bridge = 5.0 23 Figure 5.1: A ship passing the Magere brug (Skinny Bridge) in Amsterdam. 2 h_ship = 4.9 3 4 print( h_bridge < h_ship ) 5.2.1 # prints: False If, Else, and Elif Logical conditions are frequently used in If Else statements. The if statement tests whether a provided condition is either true or false. If the condition is true, the script will run the code section belonging to the if statement. If the condition is false, the script will run a different code section indicated by the else statement: 1 2 h_bridge = 5.0 # [m] h_ship = 4.9 # [m] 3 4 5 6 7 8 9 10 if h_bridge < h_ship: # the block of code that is executed if the condition is true print(’Alarm! The height of the approaching ship is too large! We have to raise the bridge deck.’) else: # the (optional) block of code that is executed if the condition is false print(’No worries , the ship can pass.’) print(’the rest of the program ’) You can note that the if and else statements are followed by a block of code that is executed only if the tested condition is true or false respectively. That code block is separated from the rest of the script using indentation. The lines of code are indented by either 4 spaces or one tab (but not mixed!). Spyder will automatically add an indent level after writing the colon (:) enclosing the condition. Please note that when the indentation level is removed, that line of code will be executed regardless of the condition. Therefore, indentation levels are crucial for the functionality of a script in Python. The else block is not always necessary. You can only use an if statement to test whether the ship height is larger than the bridge height: 1 2 h_bridge = 5.0 # [m] h_ship = 4.9 # [m] 3 4 5 if h_bridge < h_ship: # the else block is not required 24 Chapter 5 6 7 print(’Alarm! The height of the approaching ship is too large!’) print(’Continue the execution of the remainder of the script ’) Similarly, you can add more conditions to an If Else statement. You can add another condition using the elif statement: 1 2 h_bridge = 5.0 # [m] h_ship = 4.9 # [m] 3 4 5 6 7 8 9 10 11 12 13 if h_bridge < h_ship: print(’Alarm! The height of the approaching ship is too large! We have to raise the bridge deck.’) elif h_bridge -0.5 < h_ship: # testing with safety margin of 0.5 meter # this block is only executed when the main if condition is false # and when the elif condition is true. print(’Warning: the gap is only small , how are the waves?’) else: print(’No worries , the ship can pass.’) print(’the rest of the program ’) Remember that indentation is necessary for this code section to function properly! A code-block following a elif condition is only executed by the program when the if condition is False and if all prior elif conditions are false as well. 1 2 y = 3 x = 2 3 4 5 6 7 8 9 10 11 if y==4: print(’If code block ’) elif y==3: print(’Elif1 code block ’) elif x==2: print(’Elif2 code block ’) else: print(’Else code block ’) In the above code block the first elif code block is executed. The condition in the second elif is also true, but because the elif statement above it was true, that block was not executed. Exercise 5.1. Change the y variable in the above code fragment to 4 before executing the script. Then, run the script. Can you predict which block will execute, based on the input booleans? Feel free to change the variable to any number and see what the output will be. Exercise 5.2. Make a program that lets the user choose to calculate the area of either a square or a triangle. Subsequently ask the user for the relevant dimensions and calculate and display the area of the chosen geometry. 5.3 Logical operators In the previous section we have checked if one condition is true. However, you may want a program to do something based on a combination of conditions. For instance if one condition is true but another is false. The logical operators and, or and not can be used to construct more complex logical conditions. • The and operator requires both the condition on the left as well as the condition to the right of the and keyword to be true . • The or operator checks if one of the two logical expressions to the left and right of the operator is true. • The not operator reverses the boolean, i.e. not true becomes false and not false becomes true. Chapter 5 25 Some examples are given in the code section below concerning the use of the and, or and not operators: 1 2 3 4 if True and False: # Condition 1 print(’Condition 1 is true ’) else: print(’Condition 1 is false ’) 5 6 7 8 9 if True or False: # Condition 2 print(’Condition 2 is true ’) else: print(’Condition 2 is false ’) 10 11 12 13 14 if not False: # Condition 3 print(’Condition 3 is true ’) else: print(’Condition 3 is false ’) Exercise 5.3. Write down on paper what you think the code section above will print. Next, program the code yourself in a script and check your answers. You can apply these operators as follows: 1 2 x = 1 y = 1 3 4 5 6 7 if x==1 and y==1: print(’Condition is true.’) else: print(’Condition is not true!’) Please check yourself. What is the output of this code block? 5.4 While loop If we want to repeatedly use a code section in our script, we can use loops. Two types of loops exist in Python: while loops and for loops. We will first have a look at the while loop. We can execute a block of code repeatedly as long as the condition of the loop is true using a while loop. Once the script arrives at a while loop, Python will check if the condition is true or false. If the condition is true, the script will run the indented code block immediately following the colon (:). The script will run to the line with the while-statement after running the indented block and checks the condition again. When this condition is still true, Python will execute the indented code block again. If the condition is false, the script will skip the indented code block and continues running the remainder of the script. For instance the following code section repeats a code block as long as the condition is true. Initially, we start with eight apples. Line 2 of the example checks whether the number of apples is larger than zero. That condition is true, so Python moves to line 3. Line 3 applies a numerical operation: one apple is removed, so we end up with seven apples. Again we move to the whilestatement, which again checks whether the number of apples is larger than zero. This process is repeating itself until the number of apples have become zero. 1 apples = 8 2 3 4 5 6 while apples >0: apples = apples - 1 print(’after eating an apple I have ’,apples ,’left ’) print(’I am done eating apples ’) So, the value of the apples variable is modified by the code block. Specifically, the code section decrease the number of apples by one during each iteration of the loop. Please note that if we would have forgotten to add line 3 to the script, the condition would remain true forever: 26 Chapter 5 1 apples = 8 2 3 4 5 6 7 while apples >0: # (without this line) apples = apples - 1 print(’after eating an apple I have ’,apples ,’left ’) # This loop will go on forever! Press Crtl + C in the console to stop it print(’I am done eating apples ’) To stop any program you click in the Console, and press the keyboard combination ‘Ctrl’ and ‘C’. This will stop the program. 5.5 For loop The while loop was introduced in previous section. You can use the while loop until a certain condition is met However, frequently you already know how many times you want to repeat the code block. In such a case you can use the for loop. For instance: you have a Python list consisting of strings. You would like to perform an action (e.g. a numerical operation) on each list element. A for loop is ideal in such a situation: 1 words = [’apples ’,’pears ’,’mangos ’,’bananas ’] 2 3 4 for word in words: print(’the word ’,word ,’cosists of’,len(word),’letters ’) The for statement will loop over each element of the list words (’every word in words’). The value of each element is assigned to the variable word during each iteration. The indented code block is run four times in the example. • During the first iteration, the string apples is assigned to the variable word. • During the second iteration, the string pears is assigned to the variable word. • During the third iteration, the string mangos is assigned to the variable word. • During the fourth iteration, the string bananas is assigned to the variable word. A for loop can run of various items in a sequence. Examples of a sequence are the elements of a list, the elements of a tuple and the elements (characters) of a string: 1 2 for i in [’one’ ,2 ,3.0]: print(i) 3 4 5 for i in (1,2,3): print(i) 6 7 8 for i in ’A line of text ’: print(i) Note that again the indentation is crucial. Only the code block that is indented after the colon (:) symbol is run repeatedly. You can use the range() function to generate a sequence for a for loop. The following code block will print the numbers 0 up to 9: 1 2 for i in range (10): print(i) The function range(10) returns a list of 10 elements long starting (by default) starting at 0 up to and including 9. If you would like to start counting at the number 3 instead of 0, you could use two arguments in the range function: 1 2 for i in range (3 ,10): print(i) Then, the range function will produce i values from 3 until and including 9. Chapter 5 27 5.5.1 Nested for loops Loops can be combined in code blocks. These combinations are known as nested loops. The following example shows how two for loops can be combined: 1 2 3 for n in range (0 ,3): for m in range (1 ,4): print(n,’ * ’,m,’=’,n * m) First, the first (or outer) for loop in line 1 is encountered by Python. During the first iteration of the outer loop, the inner for loop is encountered in line 2. The inner for loop will first finished before the second iteration of the outer loop will start. Again, the inner loop will run completely before the next iteration of the outer loop starts. The output of the code fragment is shown below: 0*1=0 0*2=0 0*3=0 1*1=1 1*2=2 1*3=3 2*1=2 2*2=4 2*3=6 5.6 Synthesis You have learned in this chapter how to use conditionals, while loops and for loops. Next, we will focus on importing external packages which we can use in our scripts. 5.7 Exercises Exercise 5.4. Write a program using a nested for loop which shows all displayed numbers that a stopwatch would show during 3 minutes. tip: 1 print(minute ,’:’,seconds) Exercise 5.5. Since it is almost Christmas holiday, write a program that prints a Chrismas tree of height N. For the program with N = 3 the tree should look as follows: + +++ +++++ tip: you can use the strings ’ ’ (a string with only a space) and ’+’ to construct a larger string, which you will print. Exercise 5.6. A ferry carrying cars and lorries will leave the port once it is filled to 80% of its capacity. The capacity of the ferry is 50 cars. Lorries take up twice as much space as cars. The ferry has 5 lanes, each capable of storing 10 cars. If a lane cannot take the next vehicle it is closed and the next lane will start filling. Write a program that determines if a vehicle is taken onto the ferry, and if so, in which lane it will be parked. Produce a table that shows the number of the vehicle, the vehicle type and the 28 Chapter 5 lane if it is taken onto the ferry. Hint: 26 vehicles are taken on board. The vehicles arrive in the following order: Car, Car, Lorry, Lorry, Car, Lorry, Car, Car, Car, Car, Lorry, Lorry, Car, Car, Lorry, Car, Car, Lorry, Lorry, Car, Lorry, Lorry, Lorry, Lorry, Lorry, Lorry, Car, Car, Car, Lorry, Car, Car, Lorry, Car, Car, Car Exercise 5.7. Find the maximum number of subsequently repeating numbers in a row for a given list. You may use the following list: numbers = [1,3,2,3,3,4,4,5,5,3,5,6,6,6,4,6,3,3,3,3]. Exercise 5.8. In this exercise you are creating a game. The program generates a random integer number between 0 and 100 once. Subsequently the program keeps asking the user to guess what this random number is. The program returns if the guessed value is larger or smaller than the initially generated random number. The program keeps doing so until you guess the correct answer. Hints: you may want to import the package ‘random’ and use the random.randint() function, and use the input() function. Chapter 5 29 30 Chapter 5 Chapter 6 Importing external packages 6.1 Introduction Previous chapters showed you how to program your own code in Python. This chapter will focus on using Python code other programmers developed and shared. 6.2 Packages As many programmers in the world use Python, already many scripts have been developed. Some of these scripts are shared with others in packages. We can use the Miniconda environment (as introduced in section 2.2) to install these packages and import them in our scripts. The environment file that you used to install Python already contains many well-known packages. As an example: in Chapter 4 (Numerical operations), you have already seen the following line of code at the start a script: 1 import numpy This line imports the NumPy package in your active Python environment. The import keyword is followed by a package or a module name. Within this course we will only work with already existing packages. You can create packages yourself. However, that process is beyond the scope of this course. Commonly used packages are NumPy∗ , Matplotlib† and pandas‡ . NumPy is a package containing many mathematical functions, support for arrays and matrices and algebra routines. Matplotlib offers convenient tools to plot data in figures. Pandas is a package containing many tools for data manipulation and analysis. Pandas can be used for similar tasks as often performed in Microsoft Excel. We will use these packages in this course. A convention in Python is that all packages are imported at the start of your script. That way you can always immediately check which packages are imported. A package also only needs to be imported once in a script. In Chapter 2 you have set up your Miniconda environment using the .yml file, which made sure that you have the packages installed you your system. 6.3 Use Python code of a package Since we now understand how to import a package in our scripts, we can now access the Python code of the NumPy package. For example, we can access the sine-function of NumPy to calculate the sine of pi divided by two, as expressed by the following expression: π y = sin( ) 2 (6.1) ∗ https://numpy.org/ † https://matplotlib.org/ ‡ https://pandas.pydata.org/ 31 The corresponding Python code using NumPy would be: 1 2 import numpy print(numpy.sin (3.1415/2)) Try to calculate the result of this expression using both your calculator and a Python script. Do you get the same result? 6.4 Advanced importing We can import a package using it’s full name. However, programmers are sometimes lazy. They do not want to write the full name of the package each time code of the package is used. Therefore, you can provide an abbreviation or alias upon importing a package using the as keyword. The NumPy package is commonly imported using the abbreviation np. We strongly encourage you to use this abbreviation, as other programmers use the same shorthand, which enables more easily exchanging of scripts: 1 2 import numpy as np print(np.sin(np.pi/2)) Please note we have added the NumPy representation of pi to the previous example. Common shorthands for the other packages used in this course are: 1 2 3 import numpy as np import pandas as pd import matplotlib.pyplot as plt In some cases we do not want to import the full package, as we do not need other parts. The third line of previous example imports a subpackage (pyplot) from the main package (matplotlib). According to the Python convention, importing the main package is followed by a dot and then the subpackage is defined. In this course we will mainly use the subpackage pyplot from matplotlib to create figures (see Chapter 8). 6.5 Synthesis Great, now you have learned how to import packages and use the code contained in them. You are now ready to access the active Python community and maybe add your own code to the web in due time. 6.6 Exercises Exercise 6.1. Import the package numpy, and check that sin(0) is indeed zero. Also check that the value of sin(π/2) is one and sin(π/6) is a half. π is defined within the numpy package. Exercise 6.2. In this exercise you are going to compute the mean, median and standard deviation of the length of people. A numpy array with the lengths of 30 people is generated with the following line of code. 1 lengths = np.round(np.random.normal (1.75 ,0.2 ,30) ,2) Print the mean, median and standard deviation from these 30 people. Does the program give the same values each time? Exercise 6.3. Calculate the Body Mass index (BMI) for various persons. BMI is defined as the body mass divided by the square of the body height. 32 Chapter 6 1. Create two lists: one with heights and one with weights. 2. Import NumPy package and convert the lists to NumPy arrays. 3. Perform an element-wise calculation of the BMI (Body Mass Index) for each person. Example: use for heights (m): [1.70, 1.60, 1.80, 1.94, 1.64, 1.55, 1.43, 1.67], for weights (kg): [64.5, 59.2, 70, 88.4, 68.7, 60, 87, 55.5]. Chapter 6 33 34 Chapter 6 Chapter 7 Read and write data 7.1 Introduction You will often use Python for data analysis. Therefore, it is important to have the skills to read and write data from and to files. This chapter will introduce these skills. 7.2 Files If you are going to use Python for data analysis, you first have to get your data in Python. Often, data is stored digitally on your hard drive using various data files. These data files are commonly organized using a structured format which we can use to import the data in Python. Examples of data files are: • Text files • Spreadsheets (like Excel or CSV) • Images (each pixel has a data value stored) • Binary files (contains collections of bytes which are not readable by human minds) Figure 7.1: Example of a student grading spreadsheet in Microsoft Excel. 35 Figure 7.1 shows an example of a data file. The file shows the grades of students for a courses in a spreadsheet. You can clearly observe the structure of the data file. Each row represents an individual student, while the columns provide various types of information concerning that student. We can use this structure to import spreadsheets in Python. 7.3 File paths and navigation Before learning how to create or read files, it is important to understand how to find the files on your hard drive. Each data file is stored at a specific location on your computer. The location of a file on your hard drive is known as a file path. As an example, you might have installed the Python environment of this course in the following directory: C:\tools\ Anaconda3\ envs\ python_IPE . The path of the Python executable in that directory is: C:\tools\ Anaconda3\ envs\ python_ IPE\ python.exe. The Python console in Spyder is started in a specific directory. You can type cd (stands for: Change Directory) in the console to print your current working directory. Just typing cd, without providing a new directory to go to will print the current directory to the console. The current directory is also indicated in the Spyder interface, see figure 7.2. If you run a script, the current working directory will change to the directory of the script. If you want to import or write data in Python, you will have to provide your script the file path. If you only provide a filename without a path, Python will use the current working directory as path. Figure 7.2: Spyder indicating the current working directory (see red rectangular area). Figure 7.3 shows a schematic example of a file structure on a computer. Suppose you created a Python script in the folder Documents, with the name file1. The script in file1 can read file2 by simply providing the filename file2 to read the file. Alternatively, the absolute path to file2 can be provided. The absolute path to the file is: C:\Users\ Your_user_name\ Documents\ file1 . The absolute path is distinguished from the relative path. The relative path refers to the file path relative to the working directory. In this case, the filename alone is the relative path from the directory in which file1 is stored. If the script of file1 needs to read file3, the relative path is: ..\Desktop\file3. The two dots at the start of the relative path tell Python to move to the parent directory of the current directory. The parent directory in this case is the Your user name\ directory. The parent directory can be used to navigate to the Desktop\ directory and access the file. Figure 7.3: Example of a file structure. If you use a Windows laptop, the path seperator will be a backward slash (\) instead of a forward slash (/). 36 Chapter 7 Exercise 7.1. Use the cd command in the Spyder console (or in Anaconda Prompt) to navigate through your computers file system. The command cd alone prints your current directory. When you type cd a directory to navigate to, you will navigate to the directory which you specify. You can use the command ls (or dir in Anaconda Prompt) to view a list of the files and directories inside your current directory. Goal of this exercise is that you become familiar with relative paths. Exercise 7.2. The directory containing the executable mspaint.exe for Microsoft Paint is: C:\Windows\System32. Given the code example below, provide the absolute path to the Paint executable such that the Python script opens Paint. Hint: the subprocess package is used to call executables in Python. 1 2 3 4 import subprocess absolutePath = ... paintProccess = subprocess.run( absolutePath) print(’Once the program is closed , the script continues with its next lines of code ’) 7.4 Open and close data files In this section you will learn how to create a data file. Also, we will write a line of text in this file. Once we have created the file, you will learn how to open the file and read the line of text that is written in the file. To open a file, we will use the Python function open(). Please be aware that when you open a data file using the open() function, you have to close the file at the end of your script with the close() function. If a file is not closed, the data file might become corrupt, so it cannot be used anymore. Note that file closure is not performed if you do not provide code to close the file or if the script has an error before reaching the close() function. In the code example below a file called ’A file with lines of text.txt’ is opened in writing mode (hence the ’w’) in the current directory of the script. Writing mode enables you to make changes to the file. If the file does not yet exists, it will be created in writing mode. The code example then adds two lines of text to the file using the write() function. The second line is started with the \n special character, which indicates that a new line should be started in the data file. As mentioned before, it is important to close the file once Python is done with it. Therefore, the close() function is called at the end of the script. Note that the option ’w’ will overwrite an existing file (if the file ’A file with lines of text.txt’ already exists). If you would like to open a file and append a line of text, the option ’a’ should be provided instead of the option ’w’. 1 2 3 4 5 filename = ’A_file_with_lines_of_text .txt’ file = open(filename ,’w’) file.write(’First line ’) file.write(’\nSecond line ’) file.close () You are probably aware now that an error is easily encountered while programming. Therefore, you want to make sure that the data file is closed even when an error is encountered. One way to make sure that a file is always closed after opening it is usinsg the try and finally statements: 1 2 3 4 5 6 7 8 9 10 11 filename = ’A_file_with_lines_of_text .txt’ file = open(filename ,’w’) try: # do your file processing here. file.write(’First line ’) file.write(’\nSecond line ’) #filename = filename +1 # uncommenting this line will cause an error finally: # even if the program encounters an error in the try code block the file will be closed. file.close () print(’closed the file (even if an error is encountered)’) Chapter 7 37 The try code block will first be executed by Python. The finally code block will always be executed, even if an error is encountered inside the try block. Therefore, it is strongly recommended to use the try and finally construct to guarantee the data file is always closed properly. Exercise 7.3. Create a script in which you write two lines of text in a file with a single write() function call. Exercise 7.4. Create a file called ‘Names.txt’ and add the names of Bob, Peter, Jessica and Susan. Each of them must be on a new line. Exercise 7.5. In the code example for writing two lines of text in a file using the try and finally statements, remove the # before the line filename = filename+1. Run the code and check that the finally block is indeed still executed. In some cases, you only want to read the data in files rather than changing them. In that case, you can use the ’r’ option of the open() function rather than the ’w’ or ’a’ options. Remember that the file still has to be closed afterwards. Hence, we use a similar structure to read a file. In the following code example, we will read the file ’A file with lines of text.txt’, which has been created in the example code blocks above. 1 2 3 4 5 6 7 filename = ’A_file_with_lines_of_text .txt ’ file = open(filename) try: string_content_of_file =file.read () print( string_content_of_file ) finally: file.close () Exercise 7.6. Create a .txt file on your computer and write a line of text in it. Then make a script that reads the .txt file you created and let it print the line of text. 7.5 Use existing tools to read and write files: pandas Frequently text files contain data in the form of numbers. The numbers may represent measurements, experiment results, model results or other calculations. We can use existing packages to read such data files. In this section you will learn how to use the package pandas to write and read csv files (Comma Separated Values). CSV files are in essence spreadsheets. Before we will learn how to write a csv file using pandas, we will learn how pandas treats data. Pandas uses the concept of DataFrames. DataFrames are essentialy two-dimensional tables of data. Each row of the table is indicated using a row index and each column of the table is represented by a column index. Dataframes are very similar to the spreadsheets of Microsoft Excel Figure 7.4 shows an example of a pandas DataFrame in Spyder. The Excel data file of Figure 7.1 was used to create the DataFrame. You can clearly identify the column indices (Students, Grade, Exam date and Course). Please note that the row index starts at zero! 7.5.1 Writing data using pandas Let us have a look at another example. Image a pyramid that was built from cubic blocks, see Figure 7.5. The top layer (layer 1) consists of a single block. The layer below the top layer (layer 2) consists of two by two (four) blocks. The layer below layer 2 consists of three by three (nine) blocks. We can create a script that calculates the number of blocks needed based on the number of layers. Specifically, we can use a pandas DataFrame for this task. In this section we will place the data in a pandas DateFrame. We can then use the to csv() function of the DataFrame object to write the data to a CSV file. This can be done as shown in the following code block: 38 Chapter 7 Figure 7.4: Example of a pandas DataFrame in Spyder. The Excel data file of Figure 7.1 was used to create the DataFrame. Figure 7.5: Schematization of a pyramid built from cubic blocks. Source: nl.wikihow.com 1 import pandas as pd 2 3 4 data = [[1 ,1] ,[2 ,4] ,[3 ,9] ,[4 ,16]] df = pd.DataFrame(data=data ,columns =[’layer ’,’blocks ’]) 5 6 print(df) # to see the dataframe as output 7 8 df.to_csv(’data_written_using_pandas .csv’,index=False) # write data file The print statement will print the following: layer blocks 0 1 1 1 2 4 2 3 9 3 4 16 The layer and blocks data have been added as columns. We also specified the column names: ‘layer’ and ‘blocks’. The column names are also known as headers. Note that the pandas DataFrame contains an additional column with numbers (the first column). These numbers are the indexes of the rows. Chapter 7 39 Additionally, the program has written the DataFrame data to a data file named ‘data written using pandas.csv’ using the to csv() function. The contents of this file are shown in Figure 7.6. The argument ‘index=False’ of the to csv() function prevents writing of the index to the data file. Figure 7.6: The csv file created by the code example above. Note that also the pandas columns indices are included, while the row indices are not included. If you only want to write the data to a csv file (and not the index of the rows and columns from the DataFrame), the argument ‘header=False’ could be added: 1 df.to_csv(’data_written_using_pandas .csv’,index=False ,header=False) 7.5.2 Reading data using pandas You can use the read csv() function of pandas to read csv files (and other text-based data files). The next code fragment provides an example using the csv file that was created in section 7.5.1. 1 2 3 import pandas as pd df = pd.read_csv(’data_written_using_pandas .csv’) print(df) Generally it is advised to add headers to a csv file. Headers give meaning to the data in the file. When reading data, particularly data provided by others or generated by other programs, you might have to read data of which the format is slightly different than explained above. In some cases, data files contain additional text lines above the actual data to provide background information on how to interpret the data. Such information is generally known as metadata. The data from the file shown in Figure 7.7 can be loaded by specifying at which text line the header is located. Blank lines are not counted and the index starts at 0, as always: 1 df = pd.read_csv(’data_underneath_several_lines_of_text .csv’,header =3) Figure 7.7: Data files frequently contain metadata. To read such data files, you need to specify in your script at which line the actual data can be found. In this case, the header is located at the fourth line of text (linebreaks are not counted). Hence we need to specify header=3 as an additional argument in read csv(). 7.6 Synthesis This chapter focused on reading and writing data in Python. We encourage you to explore the various possibilities of using Pandas (explained in more detail here) for these tasks. 40 Chapter 7 7.7 Exercises Exercise 7.7. Make a script that writes the layer numbers and the blocks needed for each layer of the pyramid as discussed in section 7.5.1. Start by extending the example given to 6 layers. Can you also extend the script to 100 layers (without manually typing in 100 layers and the blocks needed yourself of course)? Exercise 7.8. Download the file ‘discharge Lobith.csv’ from Canvas. Make a script that reads the data. What is the mean, minimum and maximum discharge of the river Rhine at Lobith? (The discharge in the data file are given in m3 /s) Hint: you can calculate summary statistics by applying the describe() function on a pandas DataFrame. Exercise 7.9. In this exercise you will import data that includes metadata in the first lines of the file. Also, the data are not provided as a csv file. The file contains additional spaces intended for readability of the file. However these additional spaces could cause issues when reading the data using pandas. Download the meteorological measurements from Twenthe airport from the KNMI website. Read the data and extract the daily average temperature data and daily rainfall from the pandas DataFrame. Hint: 1 pandas.read_csv(filename ,header=N,delimiter=’,’,skipinitialspace =True) We encourage you to search online on how other Python programmers use Python and pandas to read data. Extra: After taking a look at Chapter 8, make a plot of the temperature and rainfall data. Exercise 7.10. Create a DataFrame of data related to famous bridges using the pandas package. Gather data from the internet for the following properties: used material, length, city for the following bridges: Golden Gate Bridge, Brooklyn Bridge, Tower Bridge, Ponte Vecchio, Rialto Bridge. The DataFrame must have the following columns: [‘bridge’, ‘material used’, ‘length’, ‘city’]. Save the DataFrame as a csv file named ‘bridges.csv’. Exercise 7.11. Open the previously created file ‘bridges.csv’ using the open() and close() functions. Manually add a new row to the file: add data about the longest bridge in the world which is located in China. Use the following data: [Danyang–Kunshan Grand Bridge, Concrete,164800, Jiangsu]. Tip: Don’t forget to add a newline at the end of the new row (\n). Exercise 7.12. Create two new directories for this exercise. Then, create and save a script in one of the two new folders. Next, read the data file of the previous exercise (’bridges.csv’) and write the data to a file in the other directory you just created. Chapter 7 41 42 Chapter 7 Chapter 8 Creating figures 8.1 Introduction Figures are a powerful way of visualizing data. In this chapter we will learn how to create figures. We will use the Python package Matplotlib for creating figures, specifically the pyplot subpackage. We import pyplot in our Python script using: 1 import matplotlib.pyplot as plt 8.2 Pyplot We will first provide a brief overview of the pyplot structure. Figure 8.1 shows an example of the structure of a pyplot figure. The outermost (red) container is the Figure object. The Figure object can contain one of multiple Axes objects. The example contains one Axes object, indicated by the blue container. Each Axes object has two axis (the horizontal (x) axis and the vertical (y) axis). The axis provide meaning to the data that are plotted in the Axes area (the blue sine visible in Figure 8.1). Additionally, figure and axis titles can be added for increased readability of the figure. Figure 8.1: Structure of a pyplot figure. Source: https://realpython.com/python-matplotlib-guide/ 43 8.3 Plotting your first figure We first have to create data if we want to create a figure. Let’s create an approximation of a sine wave based on some input values and NumPy: 1 2 import numpy as np import matplotlib.pyplot as plt 3 4 5 x = [0, 1, 2, 3, 4, 5, 6] y = np.sin(x) Note that we also imported the pyplot package already. We can now open a new figure and plot the data using the following code: 1 2 fig , ax = plt.subplots () # open an Axes object in a Figure object ax.plot(x, y, label=’Sine wave ’) # plot the sine wave in the Axes object Note that we added a label to the plot. In the next step, we can add a legend based on this label. Also, we can make the figure more clearly by adding labels and changing the axis limits: 1 2 fig , ax = plt.subplots () ax.plot(x, y, label=’Sine wave ’) 3 4 5 6 ax.set_xlabel(’X-label ’) # define x label ax.set_ylabel(’Y-label ’) # define y label ax.set_title(’Example plot ’) # define title 7 8 9 ax.set_xlim (0, 5) # set range of x axis ax.set_ylim (-1, 1) # set range of y axis 10 11 ax.legend () # add a legend based on the label It is good practice to add labels (including units!) to the axis. Finally, we can save our figure to a png file. If the figure is saved using just the filename, as is done in the example below, it can be found in the current working directory where the script is defined (see Section 7.3). Note that we use the option fig.tight layout() to clean up the figure before saving: 1 2 fig , ax = plt.subplots () ax.plot(x, y, label=’Sine wave ’) 3 4 5 6 ax.set_xlabel(’X-label ’) # define x label ax.set_ylabel(’Y-label ’) # define y label ax.set_title(’Example plot ’) # define title 7 8 9 ax.set_xlim (0, 5) # set range of x axis ax.set_ylim (-1, 1) # set range of y axis 10 11 ax.legend () # add a legend based on the label 12 13 14 15 16 fig.tight_layout () # fit the axes nicely in the figure fig.savefig(’Sine_figure.png’, # save to figure to this file dpi =150 , # export resolution (150 is fine for most applications ) bbox_inches=’tight ’) # Optinally: fit the axes nicely in the figure Now try to plot a figure using this code yourself. Do you get a similar figure as shown in Figure 8.2? Can you add a grid to the figure? Hint: you should use the grid() function on the ax-object. 8.4 Plot customization You can customize many aspects of your plots. For example, you can change the color of your line by adding the color=’red’ argument to your plot statement, as shown by the following code block. 1 2 fig , ax = plt.subplots () ax.plot(x, y, label=’Sine wave ’, color=’red’) # plot a red sine wave 44 Chapter 8 Figure 8.2: Plotting your first figure. You can find a list of all colors at https://matplotlib.org/stable/gallery/color/named_ colors.html. In addition, you can change the linestyle of your line by adding the linestyle=’dotted’ argument to your plot statement: 1 2 fig , ax = plt.subplots () ax.plot(x, y, label=’Sine wave ’, linestyle=’dotted ’) # plot a dotted sine wave You can find a list of all colors at https://matplotlib.org/stable/gallery/lines_bars_and_ markers/linestyles.html. Almost all plot properties can be customized using pyplot. Can you find out how to change the width of a line yourself? 8.5 Synthesis You are now able to plot your data analysis results in a figure. Note that you can use the online manual of Matplotlib to beautify your figures to your own liking! 8.6 Exercises Exercise 8.1. Extend the script that you created in Exercise 7.8 with a figure that plots the discharge of the Rhine at Lobith. Exercise 8.2. Create the figure that is shown in Figure 8.3. Figure 8.3: A plot of a christmas tree. Chapter 8 45 You can copy the following as a start of your script: 1 import matplotlib.pyplot as plt 2 3 4 5 tree_data = [ [0 , -0.05 , -0.05 , -0.3 , -0.1 , -0.2 , -0.05 , -0.12 ,0 ,0.12 ,0.05 ,0.2 ,0.1 ,0.3 ,0.05 ,0.05 ,0] , [0, 0 ,0.1 ,0.07 ,0.2 ,0.19 ,0.35 ,0.33 ,0.5 ,0.33 ,0.35 ,0.19 ,0.2 ,0.07 ,0.1 ,0 ,0]] 6 7 8 decorations = [[-0.1, -0.01, 0.08 , -0.02], [0.12 , 0.2, 0.3, 0.4]] Can you add more decorations to your figure? Exercise 8.3. Read the data from the data file ‘world population data.csv’. The data file can be found at Canvas. This data describe forecasts by the United Nations (UN). Plot the years against the population using ax.plot(). Also add a suitable label for the horizontal (x) axis and the vertical (y) axis. Next, add a figure title. Make sure that the y-axis starts at 0. How do the projections for the future population increase? 46 Chapter 8 Chapter 9 Functions 9.1 Introduction At this point in the reader you have probably already used various functions. Probably the first function that you used was the print() function in Chapter 2. In this chapter we will learn what functions are. Not unimportant, you will also learn why they are useful, and when and how to use them. Next to using functions, you will also learn how you can create functions yourself. In this section you will also learn what scope is. 9.2 Why functions are useful A function is a block of code that runs when that function is called. Many functions already exist. Examples are the built-in functions like max() and len(). Also, many functions are available in packages (see Chapter 6). By using readily existing functions, you can avoid to reinvent solutions to frequent occurring tasks that a program needs to do. The main task of the program that you write may be specific to your needs, but many of the smaller tasks within that program probably already exist. If a function does not exist for your task, you can create, or rather define, a function yourself. Using self written functions can also be a great way to structure your programs. Particularly, when the programming tasks get larger, functions can be used to break down a large tasks into subsequently smaller tasks. Finally, programs often need to carry out a specific task multiple times within a program. Therefore isolating that specific task to a function requires you to write code for that task only in one dedicated place, instead of having to repeat the same lines of code in multiple locations in your program. 9.3 Calling a function If we look in a bit more detail to the print() function, the function name ‘print’ is followed by an opening parenthesis, possibly several arguments separated by commas, and a closing parenthesis. Note that functions do not always need a single argument. For instance the function fig, ax = plt.subplots() encountered in Chapter 8 will create a empty figure and axes without requiring any input arguments. The function print() can also be called without any argument, however, this will not do much at first sight. It does something though; it will print a blank line to the console). If you try to call a function that does not exist, you will encounter an error: “NameError: name ‘function you tried to call’ is not defined”. Or alternatively if the name does exist, but it is not a function but for instance a variable of some type, e.g. int, you will see: “TypeError: ‘int’ object is not callable”. 9.4 Creating a function You can define a function yourself in Python using the def keyword: 47 1 2 def print_hello_world (): print(’Hello world ’) 3 4 print_hello_world () [output] Hello world In the code example above, the function print hello world() is first defined on lines 1-2 and is then called on line 4. A function definition is started with the def keyword, followed by the function name, subsequently parenthesis with optional arguments in between them. A colon (:) at the end of the first line indicates the start of the indented code block that is executed upon function call. In the code example line 4 is no longer part of the function definition as the indentation is ended, similar to the indented blocks of code when executing e.g. a for loop or an if statement. 9.4.1 Functions with input arguments and/or outputs Functions can take input arguments and return output values. In the code example below a function is given that takes a single argument. 1 2 def personalized_hello (first_name): print(’Hello ’+first_name+’, Good morning!’) 3 # The function is defined here # without calling the function these # lines of code are not executed 4 5 personalized_hello (’<type your name here >’) # Here we call the function Note that in the function definition you specify the variable name. If we provide an argument to our function, the variable ‘first name’ is assigned the value that is provided by the function call. The variable ‘first name’ does not exist outside the function. Or in other words the scope of the variable is the code block of the function. Any variable defined inside a function is not accessible outside that function. However, a function can return the values of their variables as output variables. To do so, the return keyword is used at the end of the function, followed by the variables you want to return, separated by commas. Note that the function return does not make these variables accessible outside the scope of the function, but it rather passes the values to the output of the function. The following code example shows a function that accepts two input arguments, and returns two output values: 1 2 3 4 5 6 7 def my_add_and_subtract_function (arg1 ,arg2): print(’arg1 =’,arg1 ,’ arg2 =’,arg2) arg1plus2 = arg1+arg2 arg1min2 = arg1 -arg2 print(’arg1 - arg2 =’,arg1min2) print(’arg1 + arg2 =’,arg1plus2) return arg1plus2 , arg1min2 8 9 10 11 A = 1 B = 2 C,D= my_add_and_subtract_function (A,B) To access the returned output values from a function call, new variable names can be assigned to the returned values produced by the function. Note that the scope of the variables is different inside and outside of functions. The variables A and B have the values 1 and 2 and are defined outside a function. A and B can be called from inside the function. However if changes are made to A or B inside a function, these changes do not take effect outside the function. Variables defined inside a function are not accessible outside a function. Exercise 9.1. Sum the numbers in the list [10, 20, 30, 40]. Make a function and perform the task. Example: for [10, 20, 30, 40] the output should be 100. Exercise 9.2. Create a script in which you define a variable (can be anything), below that define a function 48 Chapter 9 without an input variable that prints the variable defined above the function. Then call the function to see that you can print the variable using the function. Now define another variable inside the function, and now try to print the variable in the script itself (not from inside the function), are you able to print that variable? 9.5 Use functions to break down a problem You now had a look at various tools you can use when program. However, when provided with a problem to solve, programmers do not immediately start programming in Python. They start defining the problem using paper or a whiteboard. Next, they write down how which tasks are necessary to solve the aim of the script. Especially for larger and complex scripts, the general aim of a script is often broken down in several smaller tasks. Each of the tasks could be performed by a function. When you have thought about the different steps you need to take in a program, for example shown in Figure 9.1, you could start to write the program outline. This process is known as Conceptual modelling. Next, the programmers will start programming by implementing these smaller tasks in Python. We strongly encourage you to develop a conceptual model for your script before starting to program in Python itself! Figure 9.1: To solve a problem it is often best to start on a piece of paper. What steps do I need to take to carry out a certain task? Most of the time you can break down a complex task into smaller sub-tasks. The sub-tasks can often be implemented as functions of a program. 9.6 Synthesis You now have the skill to use functions in Python. Remember to break down your programming problems in small tasks which you can solve using functions! Chapter 9 49 50 Chapter 9 Chapter 10 Debug code and reading error messages 10.1 Introduction A computer program may not always perform as expected. While programs are extremely good at performing repetitive tasks and carrying out the precise instructions that we give to the program as programmers, they will not think along with us. In other words, our scripts just carry out exactly what we tell them to do. If we make a mistake in those instructions a program might not perform the tasks which we thought we instructed the script to perform. So in a way scripts are rather stupid themselves, but they can be used in smart ways. 10.2 Errors When we give instructions that the program does not understand, we will encounter an error. These errors tell the programmer that we gave invalid instructions. In other words, we did not provide the instructions according the rules how Python interprets commands. Generally, the error message will indicate where the program encountered something that was incorrect. At first sight error messages may look daunting. However, the error message will try to inform you on the error that was found. The terms used in these messages may at first be unfamiliar to you, but in this section you will learn what certain error messages mean. The common types of errors that you may encounter are: • SyntaxError When the syntax of your script is incorrect • TypeError When an operation/function is applied on a wrong type • NameError The variable name is not found • IndexError The index is out of range • ZeroDivisionError The program tries to divide by zero 10.2.1 Examples Some examples of errors that you might encounter are shown below: SyntaxError 1 2 1 2 3 4 1A = 5 # SyntaxError: invalid syntax # A variable name should not start with a digit , A is a valid variable name for n in range (3) # SyntaxError: invalid syntax print(n,’squared is’,n ** 2) # the indented for loop block starts after a colon (:) # A colon should have been added , i.e.: for n in range (3): 51 TypeError 1 2 3 A = ’some variable ’ A(0) # TypeError: ’str’ object is not callable # The round brackets will try to call the function A(), but it is a variable Here we tried to call the function A(). However A is defined as a variable on the line above. Therefore, a TypeError pops up. NameError 1 2 3 A=1 b=2 # Note it is a lower case b and not an upper case B C=A+B # NameError: name ’B’ is not defined You get a NameError when trying to use a variable that is not yet defined. IndexError 1 2 A = [1,2,3] # valid indexes for this list are: 0,1,2 (not 3) A[3] # IndexError: list index out of range You get an IndexError when you try to access an non-existing element of a list (out of range). ZeroDivisionError 1 2 3 A = 1 B = 1 C = A/(A-B) # ZeroDivisionError : division by zero Maybe more a mathematical issue rather than a programming one, but dividing by zero does not make sense. 10.3 Unexpected behavior and bugs Another type of error might occur. We can give commands that are valid according to the Python syntax, but still lead to unexpected results. An example would be a program that we created which should return the squared value of its input. However, the output that we get is not the squared value of the input number. In that case, clearly the syntax of the program was correct, as we did get an output and we did not encounter a syntax error. However the program did not carry out the task as expected. To figure out what went wrong requires us to have a critical look at the code. You have to analyse where in the script the unexpected result originates from. Finding such a problem in a more complex program can be tedious. A method to track what went wrong is to print (or plot) in between and intermediate results. In more complex programs with several if else statements or various functions unexpected behavior can occur for specif conditions. Wrong behavior could happen depending on the input that we give to the program. When designing a program (or a function within a program), it is often a good idea to test simple solutions for which you know the answer by hand. Finding out later which function it is that does not behave as expected is generally more time consuming than performing some simple checks with a function that you just created. 10.4 Synthesis Great, you now have all the basic tools for programming in Python! Also, you are motivated to critically assess your code in case of errors. You can now have a look at the more advanced exercises which combine previous chapters. These exercises are introduced in the next chapter. 52 Chapter 10 Chapter 11 Modular programming using packages, modules and functions We already used packages and modules in Chapter 6. There we used the import keyword for commonly used packages like numpy, matplotlib or pandas. In this chapter you will learn how you can create a module or package yourself. But what is a module or a package? Why would you want to create them, and what are their benefits. First it is important to know that modules and packages are both mechanisms intended to aid in modular programming. So far we have seen that we can create functions in scripts to break up a larger task in a script into smaller blocks defined in functions. It is very likely that you may want to reuse a function that you created in one script for another task for which you create another script. In such a case you can create a module in which you define such a function, that you would like to call in various programs. A module is thus a means to organize a larger program at another level than only breaking it up in functions in a script. There are thus three levels at which you can structure your program. From the smallest scale towards the largest scale: functions, modules and packages. By defining functions you can structure specific tasks within a script or module. By using modules you can organize groups of related functions together in a single file. And finally, by using packages, related modules can be grouped together at a higher level. 11.1 Creating a module Lets take a look at an example of a module below. Listing 11.1: example of a module file. 1 2 # filename: example_module .py import numpy as np 3 4 5 _a = ’variable starting with underscore ’ b = ’variable not starting with an underscore ’ 6 7 8 def my_function (): print(’The function my_function () has been called ’) 9 10 11 def _underscore_function (): print(’The underscored function _underscore_function () has been called ’) 12 13 14 def print_name (): print(f’__name__ = {__name__}’) 15 16 17 18 19 20 def fibonacci(N): x = np.zeros(N) x[1]=1 for n in range (2,N): x[n]=x[n -1]+x[n -2] 53 return x 21 22 23 print(’being executed anyways , also on module import ’) 24 25 26 27 28 29 if __name__ == ’__main__ ’: print(’If the module is called as a script test the defined functions ’) my_function () print_name () print(’fibonacci (10) = ’,fibonacci (10)) Note that a module could be executed just like a script, although it is not really the purpose of a module. However particularly for testing a module during development it can be useful to be able to run a module like a script. The name variable has the string value ‘ main ’ only when a .py-file is executed as a script. This allows us to use the if-statement to test if the .py-file is called as a main script, or imported as a module. When the module is executed as a script the if-block can be used to test the functions defined in the module, but is otherwise ignored. To be able to import a module, it must be located on a location on the computer where it can be found when using the import keyword. One of these locations is the current directory where a script itself is located. This is likely an okay locations for any project specific modules in the same folder on your computer. However for modules that you would like to be able to import from anywhere on your computer you will have to find out what path locations the import keyword looks for modules (see Section 11.3). To use this module inside another program we can import the module in a script (or module). The script below imports module and prints some of its variables and executes some of its functions. Listing 11.2: import the module example module (Listing 11.1) 1 import example_module 2 3 4 5 6 7 print( example_module ._a) print( example_module .b) example_module .my_function () example_module . _underscore_function () print( example_module .fibonacci (5)) Note that by importing example module the variables a and b are not defined in the so-called namespace of the script. Or in other words, when we try to call print(b), this will result in an error, because there is no variable b defined. The only object defined in the script has the name example module, and to access the values or functions defined in the module they have to be accessed using the dot notations, i.e. to access the function fibonacci() from the module we have to call it using example module.fibonacci(). When you create many different functions in various context of programs it is very likely that you will be reusing simple or convenient variable and function names. By working with modules you thus create separate namespaces in which you safely can use common variables like x, and y or function names that you would use in various different settings like update(). If you do want to have a specific function from a module in the namespace of the script you can import it using from <module name> import <specific object from module>. 1 2 from example_module import fibonacci print(fibonacci (10)) Using the above import only the fibonacci function from the module can be called. When you would like to import all objects from the module you could import them as follows using the asterisk symbol: 1 2 from example_module import * print(fibonacci (10)) Using this asterisk when importing all objects from a module to the current namespace will not import objects with names starting with an underscore, i.e. the variable a and the function underscore function() are not imported. 54 Chapter 11 Exercise 11.1. Create a module in which you define a function that can calculate the number of blocks in a pyramid (such as shown in Figure 7.5). Subsequently make a script in which you import your new module and use the function from the module to calculate the number of blocks in a pyramid of any number of layers. 11.2 Packages A package is not much more than a folder that contains several modules (or subfolders being subpackages). This gives yet another level at which you could organize a program above that of the use of modules. Note however, that at this stage in programming using packages is likely not yet applicable to you. Here we will only show how it is possible to create a package, such that you have an idea of how existing packages are build up, and that you know that this method of structuring exists. Below we show an example of a package. In the current working directory (or in a directory on the path variable) we create a folder called mypackage. Inside this folder we create the following two modules: Listing 11.3: Module called ‘module1.py’ inside the folder mypackage 1 2 def my_function (): print(’function defined in module1.py’) Listing 11.4: Module called ‘module2.py’ inside the folder mypackage 1 2 def my_function (): print(’function defined in module2.py’) If we now make a script at the directory in which the folder mypackage is created (or if the package is defined in another folder known to the path variable), we can import the modules from the packages, and call their functions as follows: 1 2 3 import mypackage.module1 , mypackage.module2 mypackage.module1.my_function () mypackage.module2.my_function () By a file named init .py inside a package, this file is executed on package import, and can be useful to automatically import modules or sub-packages once the package is imported. An example of such an init .py file is shown below: Listing 11.5: The package initialization file ‘ init .py’ located inside the folder mypackage 1 import mypackage.module1 , mypackage.module2 Once the above file is inside the package we can simply import the package name and will be able to use the modules that are imported via the initialization file. 1 2 3 import mypackage mypackage.module1.my_function () mypackage.module2.my_function () 11.3 Path locations suitable for packages and modules To figure out where modules can be placed such that they can be found using the import keyword you can create a script. In this script we will use the sys package from which we print the path variable. This will show a list of directories in which modules can be placed. Chapter 11 55 Listing 11.6: A script that prints the system path variable. Locations where the import keyword will look for modules and packages. Note that the location ‘’ (empty string) is included, which means that it will also look in the current working directory. The output of this script will be different on your own computer of course. 1 2 import sys print(sys.path) 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # output: #""" [’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\ python38.zip’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\ DLLs ’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE ’, ’’, # <-- current directory of the script you created ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib\\site -packages ’, # <-numpy , pandas , matplotlib , ... ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib\\site -packages \\locket -0.2.1 py3 .8. egg’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib\\site -packages \\ win32 ’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib\\site -packages \\ win32 \\lib ’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib\\site -packages \\ Pythonwin ’, ’C:\\ Users \\ campmansghp \\ Anaconda3 \\ envs \\ python_IPE \\lib\\site -packages \\ IPython \\ extensions ’, ’C:\\ Users \\ campmansghp \\. ipython ’] #""" If you would like to create a specific locations for your own modules and packages you could append a directory to the path variable using the append function. 1 2 3 import sys sys.path.append(’C:/ Users/campmansghp/Documents/Python_codes /MyModules ’) print(sys.path) 56 Chapter 11 Chapter 12 Solving nonlinear equations In many engineering problems we encounter equations that are used to describe or predict reality. Most of the time these type of equations are non-linear and difficult to solve. In this chapter we will look at some examples for solving nonlinear equations using the function fsolve from the scipy package. Let us start by defining a nonlinear well known function, a second degree polynomial, given by f (x) = ax2 + bx + c. (12.1) Say we want to know the roots of this function, or in other words find x that satisfies. 0 = ax2 + bx + c. (12.2) To find the values of x we need to solve the above equation. We deliberately choose a function for which we have a formula to determine its roots. However, in most engineering problems there is no such formula that solves the equation directly, therefore we will solve this know problem both using the well know formula as well as using fsolve. Through the known formula for a second order polynomial we know that the roots of this problem are given by √ −b ± b2 − 4ac (12.3) x1,2 = 2a With the values a = 1, b = 0 and c = −4 the roots are x1 = −2 and x2 = 2. To obtain the roots of the above problem using python we can use the following script: 1 from scipy.optimize import fsolve 2 3 4 5 a = 1 b = 0 c = -4 6 7 8 def f(x): return a * x ** 2+b * x+c 9 10 11 12 x1 = fsolve(f,-1) x2 = fsolve(f, 1) x3 = fsolve(f, 0) 13 14 15 16 print(f’solution by fsolve x1 = {x1}, check f(x1)={f(x1)}’) print(f’solution by fsolve x2 = {x2}, check f(x2)={f(x2)}’) print(f’solution by fsolve x3 = {x3}, check f(x3)={f(x3)}’) 17 18 19 20 21 # output: #solution by fsolve x1 = [-2.], check f(x1) =[1.77635684e -15] #solution by fsolve x2 = [2.] , check f(x2) =[1.77635684e -15] #solution by fsolve x3 = [0.] , check f(x3)=[ -4.] 57 22 23 24 #C:\ Users\campmansghp\Anaconda3\envs\python_IPE\lib\site -packages\scipy\optimize\minpack .py :175: RuntimeWarning : The iteration is not making good progress , as measured by the # improvement from the last ten iterations. # warnings.warn(msg , RuntimeWarning ) The output by this script prints the values x1 = −2, x2 = 2 and x3 = 0, along with a warning that the iterative solver used by fsolve did not result in good progress. The steps to take to solve an equation in python are thus 1) to define the function which needs to be solved, 2) to import the fsolve function from the scipy package and 3) to call the fsolve function and provide it with the function that needs to be solved and an initial guess. Note that in the script above the fsolve function indeed obtained the two roots of the given problem. However, it may in some cases not be able to find a root, in this case it has difficulties to find a root, when it starts at an initial guess of zero. Figure 12.1: In blue the second order polynomial (Equation 12.1) with values a = 1, b = 0 and c = −4. The roots are at x = −2 and x = 2. The green and red lines indicate at which root the fsolve algorithm ends up, when that x value is used as initial value. Note that just around x = 0 the line is colored black, indicating that fsolve was not able to find any of the roots. In a similar way we can solve the dispersion relation of surface gravity waves in water, given by σ 2 = gk tanh(kh), (12.4) here σ is the angular frequency (in radians) of the wave, g the gravitational acceleration, h the water depth and k the wavenumber of the surface wave. Without going in too much detail about surface waves, we will focus on how to solve such an equation. We assume that the variables σ , g and h are known and we would like to know the wavenumber k. Since k appears both in the argument of the hyperbolic tangent and in the factor multiplied with, it is impossible to rewrite it such that we obtain an expression that gives us the value of k. We have seen that we can use fsolve to find a zero value of a function. Hence we rewrite the function slightly such that there is a zero on one side of the equation. 0 = gk tanh(kh) − σ 2 (12.5) We will look for the value of k such that the equation above indeed is equal to zero. However, filling in any guess value for k will likely (unless we guess it correctly) result in a nonzero value. A measure of how closely the equation is satisfied is called a residual. Res = gk tanh(kh) − σ 2 (12.6) Once a (there might be multiple solutions) correct k value is found the residual Res is zero. Algorithms behind fsolve typically compute the residual for a guess of the parameter, and adjust the parameter based on the residual and the gradient of the residual with respect to the parameter. 58 Chapter 12 A way to solve the dispersion relation (or any other function) in python is thus by placing all terms on one side and defining the other side which should be zero to the residual. A script to solve this is shown below. 1 2 import numpy as np from scipy.optimize import fsolve 3 4 5 6 7 g = 9.81 # gravitation acceleration [m/sˆ2] h = 10 # water depth [m] T = 5 # Wave period [s] sigma = 2 * np.pi/T # angular frequency [rad/s] 8 9 10 11 def water_wave_dispersion_relation (k): residual = g * k * np.tanh(k * h)-sigma ** 2 return residual 12 13 k = fsolve( water_wave_dispersion_relation ,1) 14 15 16 17 print(f’k = {k}, check: residual ={ water_wave_dispersion_relation (k)}’) # output: # k = [0.17170284] , check: residual =[2.22044605e -16] Exercise 12.1. Use fsolve to solve the equation sin(x) = 0.5. Is there a unique solution for this equation? Make a figure, where you plot the function, your initial guess and the solution that fsolve finds. Can you explain for what initial guess fsolve finds a different answer? 12.1 Solving systems of equations In the previous section we have seen how we can solve a nonlinear equation. However, frequently you have a system of equations where you have N equations and N variables. In this section we will show give an example to solve a system to two equations and two variables. Suppose that you want to know the intersections of a circle and a line, which are defined by: x2 + y 2 = R2 , y = x, (12.7) By drawing a circle √ and a line we know that√the two intersections of the line and the circle are located at (x, y) = 12 2(R, R) and (x, y) = − 21 2(R, R). We could solve this system of nonlinear equations as well using fsolve, as is done in the following script: 1 2 import numpy as np from scipy.optimize import fsolve 3 4 5 6 7 8 9 def intersect_of_line_and_circle (x): f=np.zeros (2) R = 3 f[0] = x[0] ** 2+x[1] ** 2 -R ** 2 f[1] = x[1]-x[0] return f 10 11 12 r1 = fsolve( intersect_of_line_and_circle ,[-1,-1]) r2 = fsolve( intersect_of_line_and_circle ,[1 ,1]) 13 14 15 16 17 18 print(f’Root1 found at (x={r1 [0]:2.3f},y={r1 [1]:2.3f}), check: {np.linalg.norm( intersect_of_line_and_circle (r1)):2.3e}’) print(f’Root2 found at (x={r2 [0]:2.3f},y={r2 [1]:2.3f}), check: {np.linalg.norm( intersect_of_line_and_circle (r2)):2.3e}’) # output: # Root1 found at (x= -2.121e+00,y= -2.121e+00) , check: 2.842e-14 # Root2 found at (x=2.121e+00,y=2.121e+00) , check: 2.842e-14 Note that we still provide a single argument x to our defined function intersect of line and circle(). However here x actually represents a two dimensional vector, where the first element represents Chapter 12 59 the x and the second element represents y in Equation 12.7. Note that in the function call to fsolve we readily provided two initial guesses that resulted in the two roots of the problem. Without knowing anything about the solution we might not automatically find all roots to the problem. Exercise 12.2. Use fsolve to find the intersections of a circle and a parabola, defined by: x 2 + y 2 = 32 , y = x2 − 4 (12.8) Make a figure, where you plot the circle and parabola. Also plot your initial guess and the solution that fsolve finds. Try different initial guesses, as the figure below shows, there are regions where it is very sensitive to what initial guess you choose. Figure 12.2: A fractal like pattern of regions of initial guesses ending up at specific intersections of a circle and a parabola when using fsolve. For those interested (i.e. not part of this course!), here is a youtube video about fractals when using newton iteration (a method behind fsolve) when finding the roots of polynomials. 60 Chapter 12 Chapter 13 Numerical integration of ordinary differential equations In this chapter you will learn a very basic method which can be used to numerically integrate an ordinary differential equation. Please be aware that there exist numerous numerical methods and schemes that have much better performance than the methods explained in this chapter. However, those methods are beyond the scope of this course. Many systems are described with differential equations. Ordinary differential equations are differential equations differentiating a function with respect to a single variable, for example only time or only space. Partial differential equations describe a function by derivatives with respect to multiple variables, for example one spatial coordinate and time, or all three spatial coordinates and time or multiple spatial coordinates, but not time. Examples of partial differential equations are the heat equation, wave equation, Navier-Stokes equations prescribing fluid motion. Examples of ordinary differential equations are the onedimensional Helmholtz equation describing oscillating systems, or the equations of motion of an object. An example of an ordinary differential equation is for example the position of a car and its velocity: dx =v (13.1) dt Here v is the velocity of a car, and x the position. Suppose that the velocity of the car and the initial position of the car is known. To numerically solve this differential equation we can use the so-called forward Euler integration method. In this method the derivative is approximated by the difference between the next (still unknown) xn+1 and the current xn discrete value of x, divided by the timestep ∆t. dx dt ≈ n xn+1 − xn ∆t (13.2) The discrete equation for timestep n then becomes: xn+1 − xn = vn ∆t (13.3) xn+1 = xn + ∆t vn (13.4) which can be rewritten into: This is an equation that we can use to march forward in time to obtain numerical approximations of the position of the car in time. But we have to be aware that this is an numerical approximation. Generally the smaller the timestep ∆t the better the approximation in Equation 13.2. In the following script we use this expression to solve for the position of the car when some velocity is known. 61 1 2 import numpy as np import matplotlib.pyplot as plt 3 4 5 6 7 8 9 10 T = 10 dt = 0.5 N = int(T/dt)+1 t = np.linspace (0,10,N) a = 2 # [m/sˆ2] acceleration v = a*t # [m/s] velocity x_analytic = 0.5 * a * t ** 2 11 12 13 14 x = np.zeros_like(t) for n in range(len(t) -1): x[n+1]=x[n]+dt * v[n] 15 16 17 18 19 20 21 fig ,ax = plt.subplots () ax.plot(t,x_analytic ,’k’,label=’exact ’) ax.plot(t,x,’--’,label=f’num dt={dt}’) ax.set_xlabel(’t [s]’,fontsize =13) ax.set_ylabel(’x [m]’,fontsize =13) ax.legend () For a constant acceleration we know the analytic solution, hence we can compare the numerical solution with it. The figure below shows that the numerical solution becomes better for smaller timesteps. Figure 13.1: Numerical solution for the position of a car using the Forward Euler method with different ∆t steps, alongside with the exact analytical solution. More accurate integration schemes go beyond the scope of this introductory course. However, for further reading examples of such schemes are Runge-Kutta schemes or higher order backward finite difference schemes for explicit integration. 13.1 Solving systems of ordinary differential equations Often ordinary differential equations have higher order derivatives. For example lets consider the vertical motion of a wave buoy, shown in Figure 13.2. The vertical motion of this buoy can be prescribed by Newtons law of motion, mass times acceleration equals the forces acting on the buoy. mbuoy 62 d2 y = Fgravity + Fbuoyancy + Ffriction dt 2 Chapter 13 (13.5) Figure 13.2: A schematic illustration of a wave buoy. The forces can be expressed in the following manner: mbuoy = ρb HA Fgravity = gρb HA Fbuoyancy = −gρw yA (13.6) dy dy 1 Ffriction = − ρw Cd A 2 dt dt Here ρb is the density of the buoy, ρw is the water density, H and A are the height and crosssectional area of the cylindrical buoy, Cd is a drag coefficient and y is the submerged depth of the buoy. By substitution of the mass and forces into the equation and rewriting we obtain the following equation: d2 y dy dy + Cbuoyancy y − g = 0 (13.7) + Cfriction dt dt dt 2 where we introduced the following constants to simplify the equation, such that we can focus on the derivatives. ρw Cd Cfriction = 2ρb H (13.8) gρw Cbuoycancy = Hρb In the previous section we have seen that for a first derivative we could apply the forward euler method to integrate the unknown function, in this case y. A higher order ordinary differential equation can be written as a system of first order ordinary differential equations. We can achieve this by defining a new variable, in the context of the example this new variable is the time derivative of the position y and hence the vertical velocity of the buoy. When we substitute this definition in our earlier equation we obtain the following system of first order differential equations. dy =v dt (13.9) dv = −Cfriction |v| v − Cbuoyancy y + g dt Since both of the above equations have the form a time derivative to the left of the equal sign and the rest of the terms to its right we can again apply the forward euler method to numerically integrate these equations. In the script shown below the vertical position and velocity are integrated using the forward euler method Chapter 13 63 Listing 13.1: A script that numerically integrates the position and the velocity of the wave buoy. For those interested (but not part of the course) in the commented section two examples of higher order methods are shown, e.g. Heuns method and 4rth order Runge-Kutta. 1 2 import numpy as np import matplotlib.pyplot as plt 3 4 5 6 7 dt = 0.001 T = 10 N = int(T/dt)+1 t = np.linspace (0,T,N) 8 9 10 11 12 13 14 v = np.zeros(N) y = np.zeros(N) v0 = 0 # initial velocity y0 = 0.5 # initial position v[0] = v0 y[0] = y0 15 16 17 18 19 20 21 22 g = 9.81 # gravitational acceleration Cd =0.1 # drag cooficient rho_w = 1000 # water density rho_b = 700 # buoy density H = 1 # Height of buoy Cfric = 0.5 * rho_w * Cd/( rho_b * H) Cbuoy = g * rho_w /(H * rho_b) 23 24 25 26 27 def compute_derivatives (y,v): dydt = v dvdt = -Cfric * abs(v) * v-Cbuoy * min(max(y,0),H)+g return dydt ,dvdt 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 method = ’FE’ # options are: ’FE ’,’HEUN ’,’RK4’ for n in range(N-1): if method == ’FE’: # forward euler: (1st order accurate) dydt ,dvdt = compute_derivatives (y[n],v[n]) y[n+1]=y[n]+dt * dydt v[n+1]=v[n]+dt * dvdt # elif method == ’HEUN ’: # Heun ’s method (2nd order accurate) # k1_y ,k1_v = compute_derivatives (y[n],v[n]) # k2_y ,k2_v = compute_derivatives (y[n]+dt * k1_y ,v[n]+dt * k1_v) # y[n+1]=y[n]+dt * ( k1_y+k2_y)/2 # v[n+1]=v[n]+dt * ( k1_v+k2_v)/2 # elif method == ’RK4 ’: # Runge -Kutta4 (4th order accurate) # k1_y ,k1_v = compute_derivatives (y[n],v[n]) # k2_y ,k2_v = compute_derivatives (y[n]+0.5 * dt * k1_y ,v[n]+0.5 * dt * k1_v) # k3_y ,k3_v = compute_derivatives (y[n]+0.5 * dt * k2_y ,v[n]+0.5 * dt * k2_v) # k4_y ,k4_v = compute_derivatives (y[n]+ dt * k3_y ,v[n]+ dt * k3_v) # y[n+1]=y[n]+dt * ( k1_y +2 * k2_y +2 * k3_y+k4_y)/6 # v[n+1]=v[n]+dt * ( k1_v +2 * k2_v +2 * k3_v+k4_v)/6 47 48 49 50 51 52 53 54 fig = plt.figure () ax1=fig.add_subplot (121) ax1.plot(t,y) ax1.set_title(’depth of buoy [m]’) ax2=fig.add_subplot (122) ax2.plot(t,v) ax2.set_title(’velocity [m/s]’) Exercise 13.1. Using the script above we can play around and investigate the behavior of the wave buoy. Set the friction coeficient Cd to zero, what behavior do you observe? Does it matter what timestep ∆t we use? Note: the forward euler method works reasonably well as long as the timestep is sufficient small. Is the behavior of modeled system physical? If not can you explain what is not physical? Exercise 13.2. Create a script in which you model the trajectory of a ball that is kicked vertically upward with some given initial elevation and velocity. The equation prescribing the position of the football is 64 Chapter 13 given by: d2 y = −g dt 2 (13.10) Exercise 13.3. Likely you have come across pictures of a Lorenz attractor, like the one illustrated in the picture below. Those pictures you can generate yourself by numerically integrating the following equations, and plot any two of the coordinates (or plot x, y and z in three dimensions). dx = σ (y − x) dt dy = x(ρ − z) − y dt dz = xy − βz dt (13.11) Pick some initial conditions for x, y and z yourself, and use the values σ = 10, ρ = 28 and β = 8/3. Figure 13.3: Behavior of the Lorenz attractor. 13.2 Numerical errors Numerical methods are widely used, and a very powerful tool in a wide range of applications. However, numerical models use approximations, and may thus introduce numerical errors. Figure 13.4 shows several numerical solutions of the wave buoy problem in a no-friction scenario. When not taking care one might think that the wave buoy dropped in the water may start to oscillate with exponential increasing amplitude. This can of course not represent reality, as energy would be generated out of nowhere. So it is crucial to have a critical attitude towards numerical models. In the figure the no-friction scenario was considered because an analytical solution is available, and hence we can actually compute the numerical error. However, in almost all engineering practices where numerical models are used an analytical solution is not available. Otherwise the model is not needed anyways. So how can you evaluate weather you can trust numerical model results in practice? Always remember that in a physical system there exists no timestep ∆t, and thus the model results should be independent of the timestep. So when you have a model result, always test whether a smaller (or larger) timestep produces the same results. When this is the case then at least your physical system does not depend on a numerical setting, and thus on the physical parameters. Of course there might still be another mistake in the mathematical description, or implementation of a model. Therefore it is always a good idea to verify model results with available data to validate a model. Chapter 13 65 Figure 13.4: The numerical solution for the wave buoy compared with the analytical solution which is available if friction is set to zero. Three different numerical schemes are used with different timesteps. Higher order schemes result in higher accuracy. 66 Chapter 13 Chapter 14 Exercises combining previous chapters This chapter contains several exercises that will combine the topics of the previous chapters. Traffic related exercises Figure 14.1: Map of Enschede with the two crossings where data is available: Deurningerstraat - Lasondersingel and Kuipersdijk - Van Deinselaan. At the crossing of Deurningerstraat with the Lasondersingel data is available in the directions from West to East and from East to West. For the crossing of the Kuipersdijk with the Van Deinselaan trafic data is available in the directions from South to North and from North to South. The data can be downloaded from Canvas: Traffic in Enschede.zip. 67 Exercise 14.1. Traffic counts in opposing directions on a crossing on the Kuipersdijk and the Van Deinselaan are recorded in the North South direction (and vice versa). Download the Traffic in Enschede.zip file from Canvas. Make a script that reads the data from the files Traffic Kuipersdijk VanDeinselaanNorth to South.csv and Traffic Kuipersdijk VanDeinselaan North to South dates.csv. The first file contains the counts of traffic during 15 minute intervals vertically in the rows (so 24*4=96 rows), the different measurement days are the columns. The top row means measurement day 1 till day 29. In the csv file ending with dates.csv the dates corresponding to the measurement days are stored (1st untill 30th of April (on King’s day, the 27th there are no measurements). Make a plot of the measured traffic counts during a week day and during a weekend day. Do you see a difference between week and weekend days? Exercise 14.2. On April the 10th 2016 there was a soccer match from FC-Twente, and the traffic counts on the crossing of Deurningerstraat and Lasondersingel were recorded. The counts for this crossing are stored in a similar way as in the previous exercise, but are this time stored in the files called Traffic Deurningerstraat Lasondersingel East to West.csv, and the similar filenames with the reverse direction, that can be found on Canvas. Files ending with dates.csv contain the dates of the measurement days. Make a plot of traffic counts during the day versus time. Plot the traffic counts for 4-10 of April. 4th until 8th of April are week days and the 9th and 10th are weekend days where on the 10th the soccer match took place. Did the soccer match affect the traffic on the 10th? Exercise 14.3. We now extend Exercise 14.1 by making a figure with a subplot for each day in the week. In each subplot the average traffic intensity profiles is plotted in both directions for one day of the week. Do you see a difference between working days and weekend days? Are there differences between the 5 working days? Write the average traffic intensity profiles for the 7 days in the week in a csv file. Tip: make a function that can calculate the profile for a given day. Exercise 14.4. Load the data that you created in Exercise 14.3. Which day is daily-average the busiest? Which average week day has the highest peak? Exercise 14.5. Plot the profiles of standard deviation for the same time slots per week versus time for all 7 days in the week. In Exercise 14.3 you have done the same for the mean value. Do you see correlations between the mean value and the standard deviation? When do you see most variation? Exercise 14.6. This is a challenging exercise which it is recommended to work on this together. When modeling traffic flows there are numerous ways to distribute the traffic over the network. This network consists of roads (called edges) and intersections (called nodes). Each trip from one node to another has a cost associated with it. This cost is often equal to the travel time it takes to use the route. Consider the simplified map of Enschede below. 50 cars are leaving the university (at A) and are heading towards Saxion Hogeschool (at E) for a conference. The costs of traveling from one node to another is proportional to the distance between the node. The costs increase with each car that uses the route, since congestion increases the travel time of a route. The base costs are displayed in Table 14.1. The costs of an edge increase by 0.1 for each car that uses it. 68 Chapter 14 Figure 14.2: Simplified map of Enschede (courtesy of Openstreetmap.org) From Table 14.1: The costs of traveling between two nodes. (Note that the high costs of 200 imply that no direct travel is possible between these nodes). A B C D E F G A 0 5 200 10 200 10 200 B 5 0 10 200 200 200 200 C 200 10 0 2 2 200 200 To D 10 200 2 0 200 200 200 E 200 200 2 200 0 200 2 F 10 200 200 200 200 0 10 G 200 200 200 200 2 10 0 There are three possible routes from the University of Twente to Saxion Hogeschool: • A-B-C-E • A-D-C-E • A-F-G-E 1. Import the cost matrix from canvas and read it into your program. 2. Determine the number of cars that use the following routes by sequentially assigning a route to each car. Assume that each car takes the route with the least costs at that moment, not considering the costs that specific car adds to the route. 3. Show the development of the costs of each route in a line plot, with on the x-axis the number of allocated cars up to that point and on the y-axis the costs. Water reservoir exercises Exercise 14.7. The volume inside a reservoir can be described analytically by S = S0 exp (− kt ), where S0 is the initial volume of the reservoir and k the residence time. For the values of S0 = 1000 m3 and k = 25 days, make a plot of the volume S inside the reservoir versus time. When is the volume in the reservoir halved? Exercise 14.8. In the previous exercise you have used an analytical expression for the solution of an emptying S reservoir. The differential equation describing this problem is: dS dt = −Q, where Q = k is the flux of water leaving the reservoir. A way to numerically solve this would be to approximate the n+1 n derivative with a finite difference, e.g. S ∆t−S = −S n /k. The superscript n represents the time index i.e. t = n ∗ ∆t. Program this numerical scheme in a program and initialize S 0 with S n=0 = S0 and use the same k value from the previous exercise. Plot your numerical solution versus the Chapter 14 69 analytical solution that you already plotted in the previous exercise. Is the numerical solution identical to the analytical solution? Does the choice for the numerical time-step ∆t matter? Exercise 14.9. We can extend the previous model by modeling a reservoir with two outlets: dS dt = −Q1 − Q2 . Here Q1 is the main outlet where the majority of the water leaves the reservoir and can be closed off to regulate the volume of water inside the reservoir. When the main outlet is open Q1 = S/k1 , with k1 = 25 days. When it is closed Q1 = 0. The smaller outlet is a fish passage with Q2 = S/k2 , with k2 = 250 days. The main outlet is closed when the water volume is below 500 m3 . Again S0 = 1000 m3 . How long does it take until the volume is below 100 m3 ? Make a plot of the volume in the reservoir versus time. 70 Chapter 14 Chapter 15 Answers to the exercises Note: there is generally not one way to program. The answers given here are possibilities, check if your program performs well according the the exercise requirements. 15.1 Introduction 15.2 Getting started Answer exercises 2.5: The line of code waits for user input in the console. You could use this line if you want the user to read certain lines of code. Similarly this line of code can be used temporarily to pause the program at specific locations if you would like to test a section just before this line. 15.3 Variable types Answer exercises 3.1: 1 2 3 some_list = [5,4,3,2,1,0] print(’the first element of the list is’,some_list [0]) print(’the last element of the list is’,some_list [ -1]) Answer exercises 3.2: 1 2 3 4 list1 = [10 ,20 ,30 ,40 ,50 ,60] list2 = list1 [1:4] print(list1) print(list2) Answer exercises 3.4: 1 2 3 list1 = [12 ,22 ,313 ,10 ,6 ,78 ,69 ,420 ,11 ,88 ,100 ,120 ,300 ,500 ,12 ,3 ,4 ,1 ,55 ,23 ,21 ,65 ,34 ,54] list1.sort () print(list1) Answer exercises 3.5: 1 2 3 4 5 list1 = [12 ,22 ,313 ,10 ,6 ,78 ,69 ,420 ,11 ,88 ,100 ,120 ,300 ,500 ,12 ,3 ,4 ,1 ,55 ,23 ,21 ,65 ,34 ,54] list1.insert (3 ,1000) print(list1) list1.sort () print(list1) Answer exercises 3.7: Modifying elements of a tuple is impossible, as a tuple is immutable. 71 Answer exercise 3.8: 1 2 3 4 grades = { "John":{"Math":5, "Structural Mechanics": 6, "Fluid Mechanics":7}, "Peter":{"Math":6, "Structural Mechanics":9, "Fluid Mechanics":8}, "Jessica":{"Math":8, "Structural Mechanics":4, "Fluid Mechanics":9}} print(grades["Peter"]["Fluid Mechanics"]) 15.4 Numerical operations Answer exercise 4.1: 1 2 3 4 5 6 import numpy R = 1 #[m] <-- fill in the radius here H = 1.2 #[m] <-- fill in the height of the cylinder here A = numpy.pi * R ** 2 #[mˆ2] cross -sectional area of cyclinder V = A * H # [mˆ3] volume print(f’the volume of a cyliner with height {H} [m] and radius {R} [m] is: {V} [mˆ3]’) or a more interactive program using the input() function: 1 2 3 4 5 6 import numpy radius = float(input("Enter the radius of the cylinder in meters: ")) height = float(input("Enter the height of the cylinder in meters: ")) area = numpy.pi * radius ** 2 volume = area * height print(volume , ’mˆ3’) 15.5 Loops and conditional statements Answer exercise 5.2: 1 2 #Square print("1. Square") 3 4 5 # Triangle print("2. Triangle") 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 shape = int(input("Which shape do you choose? ")) if shape == 1: print("You chose a square") side_square = float(input(’Enter the side of the square: ’)) area = side_square * side_square print("Area of the square is: ", area) elif shape == 2: print("You chose a triangle") base = float(input(’Enter the base of the trangle: ’)) height = float(input(’Enter the height of the trangle: ’)) area = (base * height)/2 print("Area of the triangle is:", area) else: print("Incorrect option selected! choose either 1 or 2") Answer exercise 5.5: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 #christmas tree N=15 for n in range(N): r = [] for m in range(N-n-1): r.append(’ ’); for m in range(n * 2+1): r.append(’+’) for m in range(N-n-1): r.append(’ ’) string = ’’ for i in r: string = string + i print(string) 72 Chapter 15 Answer exercise 5.7: 1 2 3 4 5 6 7 8 9 10 11 12 13 # for this list the answer is 4, as there are 4 3’s repeated at the very end of the list numbers = [1,3,2,3,3,4,4,5,5,3,5,6,6,6,4,6,3,3,3,3] maxLength = 0; for i in range(len(numbers)): currentLength = 0; for j in range(i,len(numbers)): if numbers[i] == numbers[j]: currentLength = currentLength + 1; if maxLength < currentLength : maxLength = currentLength ; else: break print(maxLength) Answer exercise 5.8: 1 import random 2 3 4 5 6 7 8 9 10 11 12 answer = random.randint (0, 100) while True: student_guess = int(input(’What is your guess? ’)) if student_guess == answer: print(f’Right! The answer is { student_guess }’) break if student_guess < answer: print(f’Your guess of { student_guess } is too low!’) else: print(f’Your guess of { student_guess } is too high!’) 15.6 Importing external packages Answer exercise 6.2: 1 import numpy as np 2 3 lengths = np.round(np.random.normal (1.75 ,0.2 ,30) ,2) 4 5 6 7 # Print mean height (first column) avg = np.mean(lengths) print("Mean: " + str(avg)) 8 9 10 11 # Print median height. med = np.median(lengths) print("Median: " + str(med)) 12 13 14 15 # Print out the standard deviation on height. stddev = np.std(lengths) print("Standard Deviation: " + str(stddev)) Answer exercise 6.3: 1 import numpy as np 2 3 4 height = [1.70 , 1.60 , 1.80 , 1.94 , 1.64 , 1.55 , 1.43 , 1.67] weight = [64.5 , 59.2 , 70, 88.4 , 68.7 , 60, 87, 55.5] 5 6 7 8 9 # Convert normal lists to numpy arrays and perform element -wise calculation np_height = np.array(height) np_weight = np.array(weight) bmi = np_weight / np_height ** 2 10 11 print(bmi) 15.7 Read and write data Chapter 15 73 Answer exercise 7.2: 1 absolutePath = ’c:\ Windows\System32\mspaint.exe’ Answer exercise 7.4: 1 2 3 4 5 6 7 8 file = open(’Names.txt’, ’w’) try: file.write(’Bob\n’) file.write(’Peter\n’) file.write(’Jessica\n’) file.write(’Susan\n’) finally: file.close () The code below is also correct. However, if an error occurs between the opening of a file and the closing of it, the file is not properly closed. 1 2 3 4 5 6 file = open(’Names.txt’, ’w’) file.write(’Bob\n’) file.write(’Peter\n’) file.write(’Jessica\n’) file.write(’Susan\n’) file.close () Answer exercise 7.10: 1 import pandas as pd 2 3 4 5 6 bridges = {’bridge ’: [’Golden Gate Bridge ’, ’Brooklyn Bridge ’, ’Tower Bridge ’, ’Ponte Vecchio ’, ’Rialto Bridge ’], ’material_used ’: [’Steel ’, ’Concrete ’, ’Stone/Steel ’, ’Stone/Concrete ’,’ Stone/Concrete ’], ’length ’: [2737 , 1834 , 244, 95, 48], ’city ’: [’San Francisco ’, ’New york ’, ’London ’, ’Florence ’, ’Venice ’]} 7 8 9 df = pd.DataFrame(bridges ,columns= [’bridge ’, ’material_used ’, ’length ’, ’city ’]) df.to_csv(’bridges.csv’,index=False) Answer exercise 7.11: 1 2 3 4 5 6 7 8 9 10 bridges = open(’bridges.csv ’,’a’) try: bridge_name = ’Danyang -Kunshan Grand Bridge ’ material = ’Concrete ’ length = ’164800 ’ province = ’Jiangsu ’ newrecord = bridge_name +’,’ + material + ’,’ + length + ’,’ + province + ’\n’ bridges.write(str(newrecord)) finally: bridges.close () Answer exercise 7.12: In this answer the two created folders are called ‘folder1’ and ‘folder2’. The script bellow is stored in the folder ‘folder1’. 1 2 3 4 5 6 7 filename = ’message_for_another_folder .csv’ directory = ’../ folder2/’ file = open(directory+filename ,’w’) try: file.write(’hello folder 2, this is written by a script in folder1 ’) finally: file.close () 15.8 74 Creating figures Chapter 15 Answer exercise 8.3: 1 2 3 4 5 import matplotlib.pyplot as plt import pandas as pd df = pd.read_csv(’world_population_data .csv’,sep=’;’) years = df[’year ’]. values.tolist () # or df.iloc[:, 0]. values.tolist () pop = df[’population ’]. values.tolist () # or df.iloc[:, 1]. values.tolist () 6 7 8 9 10 11 12 plt.plot(years ,pop) plt.xlabel(’Year ’) plt.ylabel(’Population ’) plt.title(’Population Growth ’) plt.yticks ([0,2,4,6,8 ,10],[’0’,’2B’,’4B’,’6B’,’8B’,’10B’]) plt.show () 15.9 Functions Answer exercise 9.1: 1 2 3 4 5 def mysum(numbers): output = 0 for number in numbers: output += number return output 6 7 8 numbers = [10, 20, 30, 40] print(mysum(numbers)) Answer exercise 9.2: 1 2 3 a=’test ’ def some_function (): print(a) 4 5 some_function () Listing 15.1: We are able to print the variable ‘a’ from inside the function 1 2 3 4 a=’test ’ def some_function (): b=2 print(a) 5 6 7 some_function () print(b) Listing 15.2: However, we are not able to print the variable ‘b’ from outside the function 15.10 Numerical integration of ordinary differential equations Answer exercise 13.3: 1 2 3 import numpy as np import matplotlib.pyplot as plt plt.close(’all’) 4 5 6 7 sigma = 10 rho = 28 beta = 8/3 8 9 10 11 12 T = 40 It = 10000 t = np.linspace (0,T,It) dt = t[1]-t[0] 13 14 15 16 x = np.zeros(It) y = np.zeros(It) z = np.zeros(It) Chapter 15 75 17 18 19 x[0]=1 y[0]=1 z[0]=1 20 21 22 23 24 25 def ddt_lorenz_attractor (x,y,z): dxdt = sigma * (y-x) dydt = x * (rho -z)-y dzdt = x * y-beta * z return dxdt , dydt , dzdt 26 27 28 29 30 31 32 def forward_euler_next_timestep (x,y,z,dt): dxdt ,dydt ,dzdt = ddt_lorenz_attractor (x,y,z) x_new = x+dt * dxdt y_new = y+dt * dydt z_new = z+dt * dzdt return x_new ,y_new ,z_new 33 34 35 for it in range(It -1): x[it+1],y[it+1],z[it +1] = forward_euler_next_timestep (x[it],y[it],z[it],dt) 36 37 38 39 40 41 42 fig = plt.figure () ax = fig. add_subplot (111 , projection=’3d’) ax.plot(x,y,z) ax.set_xlabel(’x’) ax.set_ylabel(’y’) ax.set_zlabel(’z’) 15.11 Exercises combining previous chapters Answer exercise 14.1: 1 2 import matplotlib.pyplot as plt import pandas as pd 3 4 5 6 7 8 crossing_name = ’Traffic_Kuipersdijk_VanDeinselaan_North_to_South ’ date_file = crossing_name +’_dates.csv’ trafic_count_file = crossing_name +’.csv’ df_trafic = pd.read_csv( trafic_count_file ) df_date = pd.read_csv(date_file) 9 10 11 12 time = [] for i in range (96): time.append (0.125+0.25 * i) # creating a list with time during the day in hours ( meassurements are per 15 minutes) 13 14 15 16 17 18 19 20 def make_a_readable_day_string (date_index): day_list = [’sunday ’,’monday ’,’tuesday ’,’wednesday ’,’thursday ’,’friday ’,’saturday ’] date_string = day_list[df_date.iloc[date_index ,3] -1] # start with the weekday date_string += ’ ’+str(df_date.iloc[date_index ,0]) # add the year date_string += ’-’+str(df_date.iloc[date_index ,1]) # add the month date_string += ’-’+str(df_date.iloc[date_index ,2]) # add the day return date_string 21 22 23 day1_index = 3 day2_index = 2 24 25 26 day1_string = make_a_readable_day_string (day1_index) day2_string = make_a_readable_day_string (day2_index) 27 28 29 print(day1_string) print(day2_string) 30 31 32 33 34 35 36 37 38 39 fig ,ax = plt.subplots () ax.plot(time ,df_trafic.iloc[:, day1_index ]) ax.plot(time ,df_trafic.iloc[:, day2_index ]) ax.legend ([ day1_string , day2_string ]) ax.set_xlabel(’time [hours]’) ax.set_ylabel(’counts per 15 minutes ’) ax.set_xlim (0 ,24) ax.set_ylim (0) ax.set_title(’Kuipersdijk - Van Deinselaan ’) 76 Chapter 15 40 fig. tight_layout () Answer exercise 14.8: 1 2 3 import numpy as np import matplotlib.pyplot as plt plt.close(’all’) 4 5 6 7 Tmax = 365.0 Sinit = 1.0e3 k = 25 # Maximum simulation time [days] # Initial Storage volumne [mˆ3] # retention time [days] 8 9 dt = 7 # timestep [days] (try different time steps !) 10 11 12 13 14 t = np.arange (0,Tmax ,dt) It=len(t) S_numerical = np.zeros(It) S_numerical [0] = Sinit 15 16 17 for it in range(It -1): S_numerical[it +1]= S_numerical[it] - dt * S_numerical[it]/k 18 19 S_analytical = Sinit * np.exp(-t/k) 20 21 22 23 24 25 26 27 28 fig ,ax =plt.subplots () ax.plot(t, S_numerical ) ax.plot(t, S_analytical ) ax.set_xlim ([0, Tmax ]) ax.legend ([’numerical ’,’analytical ’]) ax.set_xlabel(’t [days]’) ax.set_ylabel(’Reservoir volume , S [mˆ3]’) fig. tight_layout () 77