IT 116: Introduction to Scripting

IT 116: Introduction to Scripting
Class 24

Microphone

Questions

Are there any questions before I begin?

Homework 10

I have posted homework 10 here.

It is due this coming Sunday at 11:59 PM.

Quiz 9

You will find the answers to Quiz 9 here.

Tips and Examples

Don't Use Indexes to Loop Through a List

The simplest way to loop through a list is to use a for loop

>>> for team in teams:
...     print(team)
... 
Red Sox
Orioles
Blue Jays
Rays
Yankees

Of course you could also do this using indexes

>>> for index in range(len(teams)):
...     print(teams[index])
... 
Red Sox
Orioles
Blue Jays
Rays
Yankees

But that would be stupid
It's more work
Which means a greater chance of making a mistake
You should only use indexing in a for loop ...
when you want to change the elements in a list

Review

Operators That Work on Objects

Operators are special characters that perform some action
Every data type has its own set of operators
For numbers, we have the arithmetic operators
- +
- -
- *
- /
- //
- %
- **
Sequences have only two operators
- +
- *

Concatenating Lists

The concatenation operator that works on strings

>>> name = "Glenn" + " " + "Hoffman"
>>>  name
'Glenn Hoffman'

Also works on lists

>>> l1 = [1,2,3]
>>> l2 = [4,5,6]
>>> l1 + l2
[1, 2, 3, 4, 5, 6]

That's because both strings and lists are sequence data types

The Repetition Operator

You cannot divide two lists

>>> l1 / l2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for /: 'list' and 'list'

But you can use the * on lists
When you do, the first operand is a list ...
and the second is an integer
The result is a new list ...

with multiple copies of the elements of the original list

>>> zeros = [0] * 5
>>> zeros
[0, 0, 0, 0, 0]
>>> numbers = [1, 2, 3] * 3
>>> numbers
[1, 2, 3, 1, 2, 3, 1, 2, 3]

The repetition operator also works on strings
```
>>> "Go " * 3
'Go Go Go '
```

Indexing

The elements in a sequence object have a definite order
Each element's place in the list can be given by a number
This number is called an index
We can get the value of an individual entry by writing the variable that points to the list ...
followed by the index inside square brackets [ ]
```
LIST_VARIABLE[INDEX]
```
Unfortunately, index numbers do not start at 1
Indexes start at 0
This is called zero-based indexing
This means that the first element in a list has an index of 0
And the index of the last element is one less than the length
Lets says we create a list of even numbers
```
even_numbers = [2,4,6,8]
```
The indexes look like this

To get a particular element we use the [ ] operator
We put this operator right after the name of the list variable

The index goes inside the square brackets

>>> even_numbers[0]
2
>>> even_numbers[1]
4
>>> even_numbers[2]
6

Changing Elements Using Indexes

Lists are mutable
That means their elements can be changed
This can only be done using indexes
If we have a list of numbers
```
numbers = [1, 4, 6, 8]
```

We can change the first number like this

>>> numbers[0] = 2
>>> numbers
[2, 4, 6, 8]

We can use indexes in a for loop
But when we do we must call range ...
with the single argument of the length of the list
We can get the length using the len built-in function
To add 1 to every element in a list ...

we can write the following for loop

>>> for i in range(len(numbers)):
...   numbers[i] += 1
... 
>>> numbers
[3, 5, 7, 9]

When we first learned about range I mentioned something that seemed strange
That when range was called with a single number ...
it created a series of integers from 0 ...
to 1 less than the number
range works this way ...
so it can be used on things like sequence objects

Using `range` with Indexes

If you need to change every element in a list ...
you need a for loop where the loop variable ...
gets every possible index value
We can create this list of indexes using range
When we call range with 1 argument ...
it gives you a list of integers starting with 0 ...
and ending with 1 less than the value of the argument
I told you this strage behavior had a reason
The reason is that if we call range with the length of the array as it's argument ...
it will generate all the indexes of that array

Here is an example

>>> nums = [1,2,3,4,5]
>>> for index in range(len(nums)):
...     printfor index in range(len(nums)):
...     print("index =  ", index)
...     print(f"nums[{index}] = {nums[index]}")
... 
index =   0
nums[0] = 1
index =   1
nums[1] = 2
index =   2
nums[2] = 3
index =   3
nums[3] = 4
index =   4
nums[4] = 5

If we want to change all the numbers in a list by adding 1 ...

we need a for loop using range to create the indexes

>>> nums
[1, 2, 3, 4, 5]
>>> for index in range(len(nums)):
...     nums[index] += 1 
... 
>>> nums
[2, 3, 4, 5, 6]

Remember that

nums[index] += 1

means

nums[index] = nums[index] + 1

Slices

You can use one list to create another
A slice is new, smaller, list created from another list ...
using a special form of indexing called a slice

You use the following format

LIST_VARIABLE[FIRST_INDEX:ONE_MORE_THAN_LAST_INDEX]

The first index is the position of the first element you want
The second index is not the index of the last element
It is one more then the index of the last element

If I had a list of day names

>>> days = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]

I could get a list of weekday names using a slice

>>> weekdays = days[1:6]
>>> weekdays
['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday']

Leaving Out Indexes in a Slice

Python allows you to create a slice using only one index
If you leave out the first index ...
the interpreter supplies a value of 0

So 0 is the default value of the first index in a slice

['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']
>>> days[:4]
['Sunday', 'Monday', 'Tuesday', 'Wednesday']
>>> digits = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> digits[:5]
0, 1, 2, 3, 4]

If you leave out the second index ...
the interpreter uses the length of the list
The default value of the second index ...
is the length of the list

This the following slice will end with the last element

>>> days[4:]
['Thursday', 'Friday', 'Saturday']
>>> digits[5:]
[5, 6, 7, 8, 9]

You can even leave out both indexes as long as you keep the :

In this case Python will return a slice that consists of the entire list

>>> days[:]
['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']
>>> digits[:]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

This is a great way to make a copy of a list

>>> daynames = days[:]
>>> daynames
['Sunday', 'Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday']

Skipping Elements in a Slice

You can also skip elements when creating a slice
It works just like the 3rd argument to the range function
To use this feature you add a second colon, :, ...

followed by a number

LIST_VARIABLE[FIRST_INDEX:ONE_MORE_THAN_LAST_INDEX:STEP_VALUE]

The STEP_VALUE works just like the range function
It is what we add to the index of the previous element ...
to get the next one
It is the difference between the index of any two elements in the list

If we wanted to create a slice of all the even digits we could write

>>> digits = [1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
>>> digits[0:9:2]
[0, 2, 4, 6, 8]

Actually, I could leave out the second index
```
digits[0::2]
[0, 2, 4, 6, 8]
```
And the first
```
digits[::2]
[0, 2, 4, 6, 8]
```

The `in` Operator

Operators are special characters that perform some action
The only operator we have seen so far have been symbols
- +
- -
- *
- /
- //
- **
- %
But operators can also be words
in is an operator that works on objects that are a collection of values
We use it like this
```
VALUE in LIST
```
The in operator returns True if the LIST contains VALUE

Otherwise it returns False

>>> digits
[1, 2, 3, 4, 5, 6, 7, 8, 9, 0]
>>> 1 in digits
True
>>> 11 in digits
False

List Methods

Lists are objects
Objects usually have functions that work on the values inside the object
These functions are called methods

Here are some of the more useful list methods

Method	Description
append(item)	Adds item to the end of the list
pop(index)	Removes the element at a given index from the list and returns the value. If called with no argument, it returns the last element and deletes it from the list.
sort()	Sorts the items in the list so they appear in ascending order (from the lowest value to the highest value)
reverse()	Reverses the order of the items in the list

The append Method

The append method adds a value to the end of a list
It takes one argument ...
the value you want to add
Is it probably the most frequently used list method
You can use the append method to create a list from scratch
First you create an empty list
```
>>> teams = []
>>> teams
[]
```
Then we can call the append method on this list using dot notation
```
>>> teams.append("Red Sox")
>>> teams
['Red Sox']
```

We can call this method as many times as we like

>>> teams.append("Orioles")
>>> teams.append("Blue Jays")
>>> teams.append("Rays")
>>> teams.append("Yankees")

Each new value is added to the end of the list

>>> teams
['Red Sox', 'Orioles', 'Blue Jays', 'Rays', 'Yankees']

The pop Method

The pop method does two things
- Removes an element from a list
- Returns the value of that element

Its argument is the index of the value to be removed

>>> numbs = [1, 2, 3, 4, 5]
>>> n = numbs.pop(0)
>>> n
1
>>> numbs
[2, 3, 4, 5]

If you leave out the argument ...

pop removes the last element

>>> n = numbs.pop()
>>> n
5
>>> numbs
[2, 3, 4]

The sort Method

A list is mutable
So we can add elements
Or remove them
But we can also change the order of the elements
This can be done with the sort method ...

which takes no argument

>>> l2 = [9, 1, 0, 2, 8, 6, 7, 4, 5, 3]
>>> l2
[9, 1, 0, 2, 8, 6, 7, 4, 5, 3]
>>>l2.sort()
>>> l2
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

The reverse Method

The reverse method reverses the order of a list

>>> l3 = [5, 4, 3, 2, 1]
>>> l3.reverse()
>>> l3
[1, 2, 3, 4, 5]
>>> l3.reverse()
>>> l3
[5, 4, 3, 2, 1]

If you want to sort a list in descending order

You first need to use sort

>>> l4 = [2, 5, 9, 1, 8, 6, 3, 7, 4]
>>> l4.sort()
>>> l4
[1, 2, 3, 4, 5, 6, 7, 8, 9]

Then use reverse

>>> l4.reverse()
>>> l4
[9, 8, 7, 6, 5, 4, 3, 2, 1]

The `del` Statement

You can delete elements from a list using the del statement
The format of a del statement looks like this
```
del LIST_VARIABLE[INDEX]
```
After the keyword del comes the list variable ...
and an index inside square brackets

>>> l5 = [0, 1, 2, 3, 4, 5]
>>> del l5[2]
>>> l5
[0, 1, 3, 4, 5]

The min and max Functions

The two built-in functions min and max ...
can also be used with lists
min returns the smallest value in a list
```
>>> l6
[1, 4, 5, 6]
>>> min(l6)
1
```
max returns the largest
```
max(l6)
6
```
This only works if the elements are all of the same type ...
and have a natural ordering
Like numbers and strings

Attendance

New Material

Time for Computer Operations

Computers are very fast
Much faster than we are
But different computer operations take different speeds
There are three time scales when dealing with data
- Getting data from a user
- Getting data from a file
- Getting data from RAM
Think about using a program like Word
Even if you were the fastest typist in the world ...
you would be glacially slow compared to a computer
This is why you want to minimize user input
Getting data from a disk is better
It is much faster than waiting for characters from the keyboard
But there is still a time penalty
With a hard disk you have to wait for the sector to spin under the head
A file is usually located on many different sectors
So the time to access each sector can add up
Accessing data from RAM is the fastest
There is no human typist or mechanical device ...
to slow things down

Reading a File into a List

In the past, when we read in numbers from a file, ...
say to compute the average, ...

we used a loop like this

file = open("numbs.txt", "r")
total = 0
count = 0
for line in file:
    number = int(line)
    total += number
    count += 1

average = round(total/count)

To count the entries in the file above the average value ...

we needed to loop through the file a 2nd time

file = open("numbs.txt", "r")
above_average = 0
for line in file:
    number = int(line)
    if number > average:
        above_average += 1

We had to loop through the file twice
And create a new file object each time
This is very inefficient
And it is slow as I explained above
It would be much better to read the file into a list
Once this is done all work would be done on the list
The list lives in RAM so data access is quick

Here is how we can create a list from a file

create an empty list
for each line in the file:
    append the line to the empty list

Using this algorithm we can write the following

file = open("numbs.txt", "r")
numbers = []
for line in file:
    numbers.append(int(line))

total = 0
for num in numbers:
    total += num

average = round(total/len(numbers))

above_average = 0
for num in numbers:
    if num > average:
        above_average += 1

print(above_average)

Notice that I did not have to count the number of entries ...
to computed the average
I can use the length of the list ...
which I can get by calling len on the list variable
Actually this code is not efficient ...
it uses 3 for loops
1. Read in the file to create the list
2. Loop through the list to create the total
3. Loop through the list to get the entries above average
We can combined the first two loops

Into a single for loop

file = open("numbs.txt", "r")
total = 0
numbers = []
for line in file:
    num = int(line)
    numbers.append(num)
    total += num

average = round(total/len(numbers))

above_average = 0
for num in numbers:
    if num > average:
        above_average += 1

print(above_average)

Advantages of Using Lists with Files

Reading a file into a list has a number of advantages
We do not have to count the number of entries in the file
We can get this value using len on the list object
Remember the work required to get the maximum value from a file?

We used the following algorithm

set a variable to a very low value
for each value in the file:
    if the value is greater than the variable:
        set the variable to this new value

Once we have read the values from a file into a list
We can use max to get the largest value
And use min to get the smallest
We can sort the file values in ascending order using sort
And in descending order using sort and reverse
You should always use a for loop to create a list ...
when working with a file

Problems with Copying Objects

To create a new variable holding a number ...
we can use an assignment statement
```
>>> n1 = 5
```
You can then use this variable ...
to create another variable
```
>>> n2 = n1
>>> n2
5
```
If you then change n1
The value of n2 remains unchanged
```
>>> n1 = 6
>>> n1
6
>>> n2
5
```
This is not true for objects ...
such as lists
Let's say we create a list object ...

and store its location in the variable l1

>>> l1 = [1,2,3,4,5,6,7]
>>> l1
[1, 2, 3, 4, 5, 6, 7]

What happens if we copy this list variable?
You can create a new variable l2 ...

by assigning it the value of l1

>>> l2 = l1
>>> l2
[1, 2, 3, 4, 5, 6, 7]

But when you change something in l1 ...

the same change appears in l2

>>> l1[6] = 8
>>> l1
[1, 2, 3, 4, 5, 6, 8]
>>> l2
[1, 2, 3, 4, 5, 6, 8]

Why?
To understand we need to look at what is happening in memory
When we first created l1 we have this

When we copy l1 to l2 with the following code
```
>>> l2 = l1
```
This is what we have in RAM

l1 and l2 point to the same object
When we make a change using l1
It's the same as using l2 to make the change
```
>>>  l1[6] = 8
```

Using an assignment statement to create a new list does not work
It merely creates a new variable pointing to the same list
This is true for all objects
Creating two variables that point to the same object is a very bad idea
Because you might think you were working with two different list ...
when you were rally working with one

Copying Lists

To copy a list you need to create a new list
This new list must have the same elements as the original list ...
and the elements must be in the same order

To copy a list we can use concatenation and the empty list

>>> l1
[1, 2, 3, 4, 5, 6, 7]
>>> l2 = [] + l1 
>>> l2
[1, 2, 3, 4, 5, 6, 7]

Here is the picture in memory

Now when you change one list the other remains unchanged

>>> l1[6] = 8
>>> l1
[1, 2, 3, 4, 5, 6, 8]
>>> l2
[1, 2, 3, 4, 5, 6, 7]

In memory, it looks like this

You can also copy a list using a slice

>>> l1 = [1,2,3,4,5,6,7]
>>> l2 = l1[0:len(l1)]
>>> l1[6] = 9
>>> l1
[1, 2, 3, 4, 5, 6, 9]
>>>  l2
[1, 2, 3, 4, 5, 6, 7]

There is a shorter way to do the same thing
When you omit the first index in a slice ...
the interpreter assumes you mean 0
And when you omit the second index ...
it assumes you mean the length of the list

So we can write

>>> l1 = [1,2,3,4,5,6,7]
>>> l2 = l1[:]
>>> l1.append(8) = 10
>>> l1
[1, 2, 3, 4, 5, 6, 7, 8]
>>> l2
[1, 2, 3, 4, 5, 6, 7]

Functions That Return Lists

Functions can return objects as well as values
So we can create a function that reads in a file of numbers ...
with one number to a line ...
and returns a list

Here is the algorithm

use the filename to create a file object
create an empty list
for each line in the file:
    convert the line into a number
    add the number to the empty list
return a variable pointing to the list

This gives us the following code

#! /usr/bin/python3
# reads a text file containing integers
# and prints it

# reads a text file of integers
# and stores them in a list which is returned
def read_integers_into_list(filename):
    file = open(filename, "r")
    new_list = []
    for line in file:
        number = int(line)	
        new_list.append(number)
    return new_list

number_list = read_integers_into_list("temperatures.txt")
print("List:", number_list)

When we run the script we get

$ ./integers_read.py 
List: [76, 67, 74, 76, 84, 69, 67, 76, 66, 71]

Functions That Return Objects

The function above returns a pointer to a list it creates
But where is the list?
When a function runs it gets its own chunk of memory
All the function's variables live inside this memory
The variables disappear when the function ends
This brings up two questions
- Why doesn't the list disappear when the function ends?
- What's the point of returning the list variable?
The list remains after the function ends ...
because it was not created in the function's memory space
Objects are created in the memory space for the script
Not the memory space for the function
Script memory space does not change when the function ends
When a function returns a variable it returns the value of that variable
The value of new_list is the memory address of the list
And that memory address is outside the function's memory space
The situation in RAM looks like this

Functions That Work with Lists

Functions can work on the values contained in a list
When we use such a function we give it a list variable as an argument
The function uses the memory address of the list to access the list

Here is an algorithm for a function that returns the average of a list of numbers

set total to zero
loop through the list using the list variable:
    add each number to total
return the total divided by the length of the list

Here is the function

def average_list(list):
    total = 0
    for number in list:
        total += number
    return total/len(list)

Similarly, we can create a function returning the number of entries ...
above a certain value
This function will need two parameters
- A pointer to the list
- A value

The function looks like this

def entries_above(list, value):
    number_above = 0
    for number in list:
        if number > value:
            number_above += 1
    return number_above

Using these functions we can create the following script

#! /usr/bin/python3
# this script reads in daily temperatures from a file and 
# calculates the numbers of days with above average temperatures

# reads integers into a list
def read_integers_into_list(filename):
    file = open(filename, "r")
    new_list = []
    for line in file:
        number = int(line)
        new_list.append(number)
    return new_list

# returns the average of list of numbers
def average_list(list):
    total = 0
    for number in list:
        total += number
    return total/len(list)

# returns the number of entries in a list
# above a certain value
def entries_above(list, value):
    number_above = 0
    for number in list:
        if number > value:
            number_above += 1
    return number_above

temps = read_integers_into_list("temperatures.txt")
print("Temperature list:", temps)
average = average_list(temps)
print("Average:", average)
print("Days above average:", entries_above(temps, average))

Running this script we get

$ ./above_average_2.py 
Temperature list: [76, 67, 74, 76, 84, 69, 67, 76, 66, 71]
Average: 72.6
Days above average: 5

Functions That Change Lists

Lists are mutable
They can be changed
And we can write a function to make those changes

Here is a function that doubles all the elements in a list of numbers

>>> def double_list(list):
...    for index in range(len(list)):
...       list[index] =  2 * list[index]
... 
>>> numbers = [1,2,3,4,5]
>>> double_list(numbers)
>>> numbers
[2, 4, 6, 8, 10]

Notice that the function did not return a value
It doesn't have to
The calling code already has a variable pointing the the list

Intellectual Property