IT 116: Introduction to Scripting

IT 116: Introduction to Scripting
Class 28

Review

Global Variables
Functions that Return Values
Returning Multiple Values
Files
Working with a File in Python
Objects
Opening a File
Writing Data to a File
Reading Files with a for Loop
Running a Script Using python3
Making a Script Executable
Errors
Syntax Errors
Logic Errors
Runtime Errors
Exception Objects
Catching Exceptions
Exceptions and Data Validation
The Code in the try Clause
The else Clause
Sequences
Lists
Operators That Work on Objects
Concatenating Lists
The Repetition Operator
Using a for Loop with Lists
Indexing
The len Function
Changing Elements Using Indexes
Empty Lists
The in Operator
List Methods
The append Method
The sort Method
The reverse Method
The del Statement
The min and max Functions
Reading a File into a List
Strings Are Sequences
Accessing Characters With Indexing
Using a for Loop with Strings
Strings Are Immutable
String Methods

Microphone

Final Exam

The final exam will be held on Tuesday, December 19th from 3:00 to 6:00 PM..

The exam will be given in this room.

If for some reason you are not able to take the Final at the time it will be offered, you MUST send an email to me before the exam so we can make alternative arrangements.

The final will consist of questions like those on the quizzes, along with questions asking you to write short segments of Python code.

60% of the points on this exam will consist of questions from the Ungraded Class Quizzes.

The remaining 40% will come from 4 questions that ask you to write some code.

To study for the code questions you should be able to

Write a function to create a list from a file
Write a function that changes a list
Write a function that performs a calculation using the elements of a list
Write a function performs a calculation on the characters in a string

A good way to study for the code questions is to review the Class Exercises homework solutions and Class Notes.

The last class on Thursday, December 7th, will be a review session.

You will only be responsible for the material in that review session and the review for the Midterm, which you will find here.

Although the time alloted for the exam is 3 hours, I would expect that most of you would not need that much time.

The final is a closed book exam.

To prevent cheating, certain rules will be enforced during the exam.

Course Evaluation

At the end of each semester we offer you the opportunity to say what you think about this course.

What have I done right?

What have I done wrong?

What can I do better?

You are not asked for you name.

So the submissions are anonymous.

I will not see your responses until after I have submitted grades for this course.

We collect this feedback through Course Evaluations.

I will use what you say to make this course better.

To complete the course evaluation, use the following link .

You have until Friday, December 22nd, to submit the evaluation.

If you do not complete the course evaluation by this date, I will subtract 1 point from the your weighted score, which determines your grade.

Questions

Are there any questions before I begin?

Attendance

Review

Global Variables

Variables created outside of the functions in a script are called global variables
Global variables can be use anywhere inside a program
Even inside functions

Functions that Return Values

Functions that return values can be used in assignment statements
```
>>> result = round(8.765) 
>>> result
9
```
A function returns a value using a return statement
```
return  EXPRESSION
```
A return statement does two things
- It ends the execution of the function
- It sends a value back to the function call

Returning Multiple Values

A Python return statement can return more than one value
To do this use multiple expressions separated by commas
```
return EXPRESION, EXPRESSION [, EXPRESSION, ...]
```

Using this feature I can write a function to get both the first and last name

>>> def get_full_name():
...     first_name = input("Please enter your first name: ")
...     last_name = input("Please enter your last name: ")
...     return first_name, last_name
...

There must be a variable on the left of the =

For each value returned by the function

>>> fname, lname = get_full_name()
Please enter your first name: Glenn
Please enter your last name: Hoffman

This is called multiple assignment

Files

When processing large amount of data we need to use files
A file is simply a linear arrangement of data on some storage medium
That storage medium might be a
- Hard disk
- Flash drive
- CD ROM
- SSD card

Working with a File in Python

A file lives on a some long term storage device
But a program works with the file in RAM
RAM is fast and expensive
External storage is slower and cheap
RAM is limited and files can be very large
So there is usually only a small part of the file in RAM
The interpreter needs to know where to get the rest of the file
The data needed to do this is stored in a file object
This object contains all the bookkeeping data needed to work with the file

Objects

A variable holds only one value
Objects can hold many such values
The values are contained in variables inside the object
Objects also hold functions that interact with the data
The functions that work on this data are called methods
Objects have no name, just a location in RAM
We create an object by using a special function
This function returns a pointer to the object in RAM
We store this value of this pointer in a variable
Then we use this variable to work with the object

Opening a File

To work with a file we have to create a file object
We do this using the built-in function open
The open function does two things
- It creates a file object
- It returns a reference to the object
The reference is stored in a variable
We then use the variable to work with the object
Here is the format for using the open function
```
FILE_VARIABLE = open(FILENAME, MODE)
```
FILE_VARIABLE is the name of the variable that holds the reference
FILENAME is a string specifying the file
MODE determines what we can do with the file

The three most commonly used access modes are

Mode	Description
"r"	Open a file for reading only
"w"	Open a file for writing
"a"	Open a file for writing to the end of the file

A file open for reading cannot be changed
If you open a file for writing or appending that does not exist it will be created
If the file exists it will be overwritten with new text
The a in the last mode stands for "append"
Text is added to the end of the file so nothing is lost

Writing Data to a File

To work with a file in Python, we must first create a file object
This object contains methods that allow us to talk to the file
The functions inside objects are called methods
To call a method you must use dot notation
Dot notation for methods has the following format
```
OBJECT_VARIABLE.METHOD_NAME
```
We first open the file for writing
```
teams_file = open("teams.txt", "w")
```

Now we can send text to the file using write method of this file object

$ cat write_teams.py
# writes the names of the teams in the American League East

teams_file = open("teams.txt", "w")

teams_file.write("Boston Red Sox\n")
teams_file.write("Baltimore Orioles\n")
teams_file.write("Toronto Blue Jays\n")
teams_file.write("Tampa Bay Rays\n")
teams_file.write("New York Yankees\n")


$ python3 write_teams.py 

$ cat teams.txt 
Boston Red Sox
Baltimore Orioles
Toronto Blue Jays
Tampa Bay Rays+
New York Yankees

Notice that I had to put the newline character, \n, at the end of each line
Otherwise all the text would be on one line

Reading Files with a `for` Loop

The easiest way to read a file in Python is to use a for loop
The header of for loop has this format
```
for LOOP_VARIABLE in LIST_OF_SOME_KIND:
```
The last item is some sort of collection
A text file is a collection of lines
This means we can use a file object in a for loop

You can print the file like this

for line in file:
    print(line.rstrip())

Running a Script Using `python3`

Our Python scripts are text files
Written in Python programming language
The computer does not understand this text
So we cannot run Python scripts directly
Only the Python interpreter understands Python scripts
The Python interpreter was written in a language like C
And compiled into an executable file
When we type python3 at the command line we are running this binary file
To run the script we give this command the filename of the Python script
python3 then executes all the Python statements in the file

So if I have the Python script hello_1.py

# prints a friendly message

print("Hello world!")

I can run it on Unix command line like this
```
$ python3 hello_1.py 
Hello world!
```

Making a Script Executable

A script that is made executable
Can be run without first typing python3
To make a file executable we need to do two things
- Give it read and execute permissions
- Add the hashbang line
We can set the permission using the Unix chmod command
```
chmod  755  FILENAME
```
The hashbang line must be the very first line in the script
The first two characters must be #!
This is followed by the absolute pathname of the Python interpreter
The hashbang line on our Unix machines is
```
#! /usr/bin/python3
```

Errors

It is easy to get an error when writing scripts
But to fix errors it is important to recognize the different error types
There are three classes of errors
- Syntax errors
- Logic errors
- Runtime errors

Syntax Errors

Syntax errors occur when you write a statement that violates the rules of the programming language

>>> name = "Glenn"
>>> print(nme)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'nme' is not defined

Syntax errors are the most common errors encountered when writing scripts

Logic Errors

A logic error occurs when the code does not give the correct results

Here is an example

average = num_1 + num_2 / 2
print ("Average:", average)

The numerator should be sum of both num_1 and num_2
But here it is only num_2
You get no error message with a logic error
The code works and unless you check the results you will not know there is a problem
This is why it is always important run tests on the scripts you write

Runtime Errors

Runtime errors occur only when the the program is run
They usually occur when a specific value makes a statement fail
If the program is run again with a different value
No runtime error would occur

A common way to get a runtime error is to open a file that does not exist

$ ./file_open.py 
Filename: xxxxxx
Traceback (most recent call last):
  File "./file_open.py", line 6, in <module>
    file     = open(filename, "r")
FileNotFoundError: [Errno 2] No such file or directory: 'xxxxxx'

Exception Objects

Python has a mechanism that deals with runtime errors
It gathers information about what happened and where
And uses it to create an error message
It does this by creating an exception object
When this happens we say that the interpreter has raised an exception

Catching Exceptions

Python allows you to see exception objects in your code while your script is running
Your code can then do something to prevent it from aborting
And sending you back to the command line

You do this with a try/except statement which has the following form

try:
    STATEMENT
    STATEMENT
    ...
except:
    STATEMENT
    STATEMENT
    ...

If any statement in the try code block causes a runtime error
The interpreter stops executing the rest of the code
Instead it jumps to the except code block and executes those statements
When your code does this, it is said to catch an exception
Whenever you use open to create a file
You should put it inside a try/except statement

Like this

$ cat open_file.py
#! /usr/bin/python3

# demonstrates using a try/except statement 
# to catch exceptions encountered while
# trying to open a file

filename = input("Filename: ")
try:
    file = open(filename, "r")
    for line in file:
        print(line.rstrip())
except:
    print("Could not open file", filename)

$ ./open_file.py
Filename: xxxx
Could not open file xxxx

Notice that the interpreter never ran the for loop

Exceptions and Data Validation

Whenever we get data from the user we need to check it
This is called data validation
We can use input to ask the user for a number
But input returns a string
So we need to use a conversion function
But int and float can cause a runtime error
So we need to call them inside a try/except statement
We should put this inside a while loop
That way the it will keep running until we get valid input

Here is a function that does that

def get_integer():
    while True:
        number = input("Integer: ")
        try:
            number = int(number)
            return number
        except:
            print(number, "cannot be converted into an integer")

The Code in the `try` Clause

You need to put the open statement in the try block of a try/except statement
But the rest of your code needs the file object to function properly
If this code is outside the try/except statement

You will get an error if the file object was not created

filename = input("Filename: ")
try:
    file = open(filename, "r")
except:
    print("Cannot open", filename)

count = 0
total = 0

for line in file:
    count += 1
    date, temp = line.split()
    temp   = int(temp)
    total += temp

average = round(total/count, 2)
print("Average:", average)


$ ./temps_average_3.py 
Filename: xxx
Cannot open xxx
Traceback (most recent call last):
  File "./temps_average_3.py", line 15, in <module>
    for line in file:
NameError: name 'file' is not defined

Because the code depends on the open statement working
You need to put all the code that depends on the file object
Inside the try/exceptstatement

The `else` Clause

A try/except statement can also have an else clause
The statements an else block will be run
If try block code does not cause a runtime error
You should put all code that need the statements in the try clause to work
Inside an else clause

Like this

try:
    file = open(filename, "r")
except:
    print("Cannot open", filename)
else:
    count = 0
    total = 0
    for line in file:
        count += 1
        date, temp = line.split()
        temp   = int(temp)
        total += temp

    average = round(total/count, 2)
    print("Average:", average)

Sequences

Sequences are Python data types that hold many values
The values are store one right after the other in RAM
This picture in memory looks like this

Here are the three sequence data types
- lists
- tuples
- strings
Lists are sequences whose values can be changed
Tuples are sequences that cannot be changed

Lists

The values in a list are called elements
You can change any value in the list
And change their s
You can also add and remove items from the list
You create a list literal by placing a series of values
Inside square brackets

There must be a comma between each value

>>> even_numbers = [2, 4, 6, 8, 10]
>>> even_numbers
[2, 4, 6, 8, 10]]

Here is what this looks like in memory

This is an example of a list literal
Lists can contain a value of any type
- Integers
- Floats
- Strings
- Booleans
Lists can also contain other lists

Operators That Work on Objects

Operators are special characters that perform some action
Every data type has its own set of operators
For numbers, we have the arithmetic operators
- +
- -
- *
- /
- //
- %
- *
Sequences have only two operators
- +
- *

Concatenating Lists

The concatenation operator that works on strings

>>> name = "Glenn" + " " + "Hoffman"
>>>  name
'Glenn Hoffman'

Also works on lists

>>> n1 = [1,2,3]
>>> n2 = [4,5,6]
>>> n1 + n2
[1, 2, 3, 4, 5, 6]

That because both strings and lists are sequence data types

The Repetition Operator

You cannot divide two lists

>>> n1 / n2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for /: 'list' and 'list'

But you can use the * on lists
When you do the first operand is a list
And the second is an integer
The result is a new list

With multiple copies of the elements of the original list

>>> zeros = [0] * 5
>>> zeros
[0, 0, 0, 0, 0]
>>> numbers = [1, 2, 3] * 3
>>> numbers
[1, 2, 3, 1, 2, 3, 1, 2, 3]

The repetition operator also works on strings
```
>>> "Go " * 3
'Go Go Go '
```

Using a `for` Loop with Lists

A for loop will work with a list

>>> numbers = [1, 2, 3, 4, 5]
>>>for n in numbers:
...   print(n)
... 
1
2
3
4
5

When we loop through a list we are said to iterate over that list

We can also use a for loop with strings

>>> team = "Red Sox"
>>> for char in team:
...     print(char)
... 
R
e
d
 
S
o
x
>>>

Indexing

The elements in a list have a definite order
Each element's place in line can be given by a number
This number is called an index
Unfortunately, index numbers do not start at 1
Indexes start at 0
This is called zero-based indexing
This means that the first element in a list has an index of 0
The second has an index of 1 and so on
It also means that the index of the last element
Is one less than the length of the list

To get a particular element we use the [ ] operator
We put this operator right after the name of the list variable

The index goes inside the square brackets

>>> even_numbers[0]
2
>>> even_numbers[1]
4
>>> even_numbers[2]
6

The `len` Function

The built-in function len will return the length of any sequence

>>> even_numbers = [2, 4, 6, 8, 10]
>>> len(even_numbers)
5
>>> len("foo")

We can use len with range to iterate through a list

for i in range(len(even_numbers)):
...   print(even_numbers[i])
... 
2
4
6
8
10

Changing Elements Using Indexes

Lists are said to be mutable
That means they can be changed
This can only be done using indexes
If we have a list of numbers
```
numbers = [1, 4, 6, 8]
```

We can change the first number like this

>>> numbers[0] = 2
>>> numbers
[2, 4, 6, 8]

We can use indexes in a for loop to change every element inside a list

>>> for i in range(len(numbers)):
...   numbers[i] += 1
... 
>>> numbers
[3, 5, 7, 9]

Empty Lists

We create an empty list using a list literal with nothing between the square brackets
```
>>> new_list = []
>>> new_list
[]
```
The empty list serves the same purpose for lists that zero does for numbers

The `in` Operator

Operators are special characters that perform some action
The only operator we have seen so far have been symbols
- +
- -
- *
- /
- //
- **
- %
But operators can also be words
in is an operator that works on objects that are a collection of values
We use it like this
```
VALUE in LIST
```
The in operator returns True if the LIST contains VALUE

Otherwise it returns False

>>> digits
[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
>>> 1 in digits
True
>>> 11 in digits
False

List Methods

Lists are objects
Objects usually have functions that work on the values inside the object
These functions are called methods

Here are some of the more useful list methods

Method	Description
append(item)	Adds item to the end of the list
sort()	Sorts the items in the list so they appear in ascending order (from the lowest value to the highest value)
reverse()	Reverses the order of the items in the list

The append Method

The append method adds a value to the end of a list
It takes one argument, the value you want to add
Is it probably the most frequently used list method
You can use the append method to create a list from scratch
First you create an empty list
```
>>> teams = []
>>> teams
[]
```
Then we can call the append method on this list dot notation using
```
>>> teams.append("Red Sox")
>>> teams
['Red Sox']
```

We can call this method as many times as we like

>>> teams.append("Orioles")
>>> teams.append("Blue Jays")
>>> teams.append("Rays")
>>> teams.append("Yankees")

Each new value is added to the end of the list

>>> teams
['Red Sox', 'Orioles', 'Blue Jays', 'Rays', 'Yankees']

The sort Method

A list is mutable so we can add elements or remove them
But we can also change the order of the elements

This can be done with the sort method which takes no argument

>>> l2 = [9, 1, 0, 2, 8, 6, 7, 4, 5, 3]
>>> l2
[9, 1, 0, 2, 8, 6, 7, 4, 5, 3]
>>>l2.sort()
>>> l2
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

The reverse Method

The reverse method reverses the order of a list

>>> l3 = [5, 4, 3, 2, 1]
>>> l3.reverse()
>>> l3
[1, 2, 3, 4, 5]
>>> l3.reverse()
>>> l3
[5, 4, 3, 2, 1]

If you have a list of unsorted numbers and you want to sort them in descending order you first need to use sort
```
>>> l4 = [2, 5, 9, 1, 8, 6, 3, 7, 4]
>>> l4.sort()
>>> l4
[1, 2, 3, 4, 5, 6, 7, 8, 9]
```

Then use reverse

>>> l4.reverse()
>>> l4
[9, 8, 7, 6, 5, 4, 3, 2, 1]

The `del` Statement

List methods aren't the only things that work on lists
You can delete elements from a list using the del statement
The format of a del statement looks like this
```
del LIST_VARIABLE[INDEX]
```
After the keyword del comes the list variable and the index inside square brackets

>>> l5 = [1, 2, 3, 4, 5, 6]
>>> del l5[2]
>>> l5
[1, 2, 4, 5, 6]

The min and max Functions

Two built-in functions can also be used with lists
They do not change the list but they return information about it
min returns the smallest value in a list
```
>>> l6
[1, 4, 5, 6]
>>> min(l6)
1
```
max returns the largest
```
max(l6)
6
```

Reading a File into a List

Here is the algorithm for creating a list from a file

create an empty list
for each line in the file:
    append the line to the empty list

Using this algorithm we can write the following

file = open("numbs.txt", "r")
total = 0
numbers = []
for line in file:
    num = int(line)
    numbers.append(num)
    total += num

Strings Are Sequences

There are many different data types in Python
Some hold only one value
- Integer
- Float
- Boolean
Other data types hold more than one value
A sequence is a family of data types that contain many different values
Stored one right after the other
Strings are sequences where the elements are individual characters
Sequences are objects and have methods that operate on the data they contain
Each sequence data type has its own set of methods

Accessing Characters With Indexing

The characters in a string have a definite order
So we can use indexing to get a particular character
If we have a string
```
>>> team = "Red Sox"
```

We can use the [ ] operator to get a specific character
Inside the brackets we place an integer
```
>>> team[0]
'R'
```

Using a `for` Loop with Strings

You can also access the individual characters in a string using a for loop
```
>>> for ch in team:
...   print(ch)
... 
R
e
d

S
o
x
```
This allows us to write some very interesting functions that work with strings

We can write a function to count the number of times a certain character occurs in a string

>>> def character_count(string, char):
...    count = 0
...    for ch in string:
...        if ch == char:
...            count += 1
...    return count
... 
>>> character_count("Mississippi", "i")
4
>>> character_count("Mississippi", "s")
4
>>> character_count("Mississippi", "p")
2

We can also use a for loop to reverse a string

>>> def string_reverse(string):
...     new_string = "
...     for ch in string:
...         new_string =  ch + new_string
...     return new_string
... 
>>> string_reverse("Mississippi")
'ippississiM'
>>> string_reverse("radar")
'radar'

Strings Are Immutable

Strings are immutable
You cannot change a string

But you can use one string to create another

>>> string_1 = "Red"
>>> string_1 = string_1 + " Sox"
>>> string_1
'Red Sox'

In the first statement above I created the string "Red"
The string variable string_1 points to this new string
Here is how things look in memory

When I wrote
```
string_1 += " Sox"
```
two things are happening
First, I created a new string consisting of the characters in the first string and " Sox"
Second, the object variable string_1 now points to this new string
Nothing points to the old string "Red"
But that string has not gone away
Here is the new memory situation

String Methods

String have many methods

Some methods create a new string from and old one

Method	Description
lower	Returns a copy of the string with all alphabetic letters converted to lowercase. Any character that is already lowercase, or is not an alphabetic letter, is unchanged.
upper	Returns a copy of the string with all alphabetic letters converted to uppercase. Any character that is already uppercase, or is not an alphabetic letter, is unchanged.
lstrip	Returns a copy of the string with leading characters removed. When run without an argument, it removes leading whitespace characters. When run with an argument, removes those characters.
rstrip	Returns a copy of the string with all trailing characters removed. When run without an argument, it removes tailing whitespace characters. When run with an argument, removes those characters.
strip	Returns a copy of the string with leading and trailing characters removed. When run without an argument, it removes leading and tailing whitespace characters. When run with an argument, removes those characters.

One extremely useful method is split
split creates a list from a string
In order for split to work
It has to know what character to use to break up the string
In other words, what character separates two element values
This character is called a delimiter
If you call split without an argument

It will use whitespace as the delimiter

>>> s = "foo bar bletch"
>>> s.split()
['foo', 'bar', 'bletch']

If any other character or characters serve as the delimiter

They must be given as an argument

>>> date = "2021-05006"
>>> date.split("-")
['2021', '05006']

Review

Final Exam

Course Evaluation

Questions

Attendance

Review

Global Variables

Functions that Return Values

Returning Multiple Values

Files

Working with a File in Python

Objects

Opening a File

Writing Data to a File

Reading Files with a for Loop

Running a Script Using python3

Making a Script Executable

Errors

Syntax Errors

Logic Errors

Runtime Errors

Exception Objects

Catching Exceptions

Exceptions and Data Validation

The Code in the try Clause

The else Clause

Sequences

Lists

Operators That Work on Objects

Concatenating Lists

The Repetition Operator

Using a for Loop with Lists

Indexing

The len Function

Changing Elements Using Indexes

Empty Lists

The in Operator

List Methods

The append Method

The sort Method

The reverse Method

The del Statement

The min and max Functions

Reading a File into a List

Strings Are Sequences

Accessing Characters With Indexing

Using a for Loop with Strings

Strings Are Immutable

String Methods

Reading Files with a `for` Loop

Running a Script Using `python3`

The Code in the `try` Clause

The `else` Clause

Using a `for` Loop with Lists

The `len` Function

The `in` Operator

The `del` Statement

Using a `for` Loop with Strings