CS 624: Analysis of Algorithms

Carl Offner

Spring 2017

office: 130
email: offner "at" cs.umb.edu

In addition to this web page, I will put my lectures and other materials in the directory ~offner/cs624 and subdirectories under it. Feel free to poke around there. Anything there (or under there) is for you to take. (Note that these are not web pages—they are ordinary files, and you have to log in to your cs.umb.edu account to access them.)

Assignments:

[[ Assignments will be posted here. ]]

Prerequisite:
• CS 320L: Applied Discrete Mathematics

This prerequisite is important. If you have not had it, and done well in it, you should not be enrolled in this course.

Further, while CS 320L is the official prerequisite, there is some other material that you really need to know in order to understand what we are doing in this course. Undoubtedly you have studied this before, but I just want to make it explicit here. I will not review this material in class—you are responsible for understanding it. The homework and also the exams will include problems using this material:

• Some serious single-variable calculus: derivatives, the chain rule, integrals, the "Fundamental Theorem of the Calculus", maxima and minima, increasing and decreasing functions, convex functions, step functions. (Do you know what a step function is? If not, you need to find out right away. Is a step function really a function? And how do you integrate a step function?)

And of course you should never get confused between the two functions

• `f(x) = xa` (for instance, ```f(x) = x3```) and
• `g(x) = ax` (for instance, ```g(x) = 3x```)

Which one grows faster? And is the difference in growth significant? (And I'm not putting these two questions here to practice my typing. Will you be able to answer these questions if I ask them to you on the first day of class?)

Do you know the formula for the sum of a geometric series? Can you derive it?

• High school algebra: Can you factor polynomials? Can you add algebraic fractions? Can you manipulate and simplify expressions involving square roots? Do you know the quadratic formula? Can you derive it?
• Some basic probability theory: combinatorics applied to probability; expectation; conditional probability; independence. Appendices C.1-C.4 of the text go over this material—more than we will really need, but you should at least look it over and you should be familiar with a fair amount of it. You should be able to do at least some of the exercises there yourself. If you have questions about them, send me email, and I'll be glad to help. But I'm not going to cover this material in class—you are responsible for it.

Class meetings: M/W 5:30-6:45 PM in room M-3-617 (that's the McCormack building)

Office hours: Before class: M/W 4:30-5:15 PM

Text:

Introduction to Algorithms, Third Edition
by Cormen, Leiserson, Rivest, and Stein
MIT Press, 2009

Syllabus:

I'm sure everyone in this class has taken one or more courses in which you have learned some of the really great algorithms.

This is not one of those courses. This is not a course in "really great algorithms", or in writing programs, or even in writing pseudo-code.

This is for all practial purposes a mathematics course. It is a course on the analysis of algorithms. We are going to prove theorems in this course. In fact, that's just about all we are going to do. And when I say we, I really mean you. The homework will consist of theorems that you will have to prove. The exams will consist of theorems that you will will have to prove. That's what we're going to do.

At a minimum, I plan to cover the following material this term:

1. Introductory material. What do we mean by analysis of algorithms? Why do we care? Some basic examples.
2. Mathematical background and some useful mathematical techniques. Orders of growth of functions. Solving and estimating solutions of recursions. Generating functions.
3. The analysis of Heapsort and Quicksort. Best-case, worst-case, and average-case behavior.
4. Lower bounds on sorting algorithms; bucket sort.
5. Order statistics.
6. The theory of binary search trees.
7. Dynamic programming.
8. Amortized analysis.
9. Graph algorithms; in particular:
• a detailed mathematical analysis of depth-first search, together with some of its remarkable applications;
• the max-flow min-cut theorem and its consequences.
10. The theory of NP-completeness.

There are many other topics that I'd really like to cover as well. This is just a fascinating field. We'll see…

Course Work:

There will be one homework assignment each week. I will not accept assignments handed in late.

There will be two in-class "mid-term" exams, and one final exam.

All the exams will be "closed-book" exams. No books, no notes, no smartphones or other electronic equipment, and so on. Just bring something to write with.

I will weight the work in the course roughly as follows when making up final grades:

• All the homeworks, taken together: 20%
• All the mid-term exams, taken together: 40%
• Final exam: 40%

I will not give out "review sheets" for the exams. It should go without saying, however, that the very best way to prepare for the exams is to

• do the homework, and
• read the handouts carefully. If there is anything you don't understand in them, you need to ask me about it. And the earlier you ask, the better.

Writing Proofs:

This course does not involve any programming per se, although without a doubt your experiences in building serious programs will help you understand many of the issues we deal with.

In fact, as I already stated above, the course could well be regarded as a mathematics course. I will be proving things in class, and you will be expected to write formal proofs in your assignments and on exams. If this is something you are not ready to come to grips with, you should wait until you are ready to take this class.

There is a style to writing proofs. In fact, there are several styles. Your goal should be to write proofs that are easy to read and that illuminate the reason why something is true. This is not easy to do. Good writing of any kind is difficult, and we will work hard at it all term.

Probably the single most important proof technique in this area is mathematical induction. I will assume not only that you are familiar with this, but are comfortable using it. I will not teach it, but I will expect to see inductive proofs written out clearly and well.

I also expect you to be familiar with standard mathematical notations such as summations (and of course, integrals). Some of this is reviewed in the appendices of the text. I will not review this in class. I expect you to know it, and we will be using it a lot.

One thing is very important to bear in mind: like any other kind of writing, a proof must be understandable. If I can't understand what you are writing, then it can't be correct. And I'm not going to try to guess what you had in mind. (One thing I have noticed is that some students seem to believe there is only one way to do something, and only one way to explain it, and so of course I must understand what they mean. But that's not true. There are almost always many ways to do things, and many ways to explain them. If I write on your paper "I don't understand this", I'm not being irritable—I really don't understand it.) If you are uncertain how to express something, please just talk to me or send me email, and I'll try to help. But don't even bother writing something that can't be understood.

Think of this like you would when writing a computer program. If the compiler or interpreter can't understand it, it's wrong. And you have to be really careful with your use of language and terms. For instance, one thing beginning programmers often get mixed up about is the correct use of "or" and "and". The meaning in computer languages is not the same as in English (or any other natural language, so far as I know). The same is true in proofs. We use language in very precise and fixed ways in proofs, and we do this so that what we write can be understood in its exact meaning by anyone at any time, anywhere in the world.

Just for a simple example: the sentence "All horses are not white." does not mean the same as "Not all horses are white." At least, not when writing proofs. In ordinary English, these two sentences are generally regarded as equivalent. But for our purposes they are not. You have to be careful about this.

Here's another example: The symbol "==>" means "implies". It does not mean "then". So you can write

```   if x = 3 then x + 5 = 8
```
or you can write
```   x = 3 ==> x + 5 = 8
```
But you cannot write
```   if x = 3 ==> x + 5 = 8   !!! WRONG !!!
```
Here's another thing to watch out for: the symbol "=" means "equals". It doesn't mean anything else. (Well, there is one small exception, involving "big-O" notation, which we will cover in the second class.) So could could write something like this:
```    2x - 1 = 5
so
x = 3
```
But you cannot write
```    2x - 1 = 5
=     x = 3       !!! WRONG !!!
```
Please try to be careful about things like this. And please don't be offended if I correct mistakes like this in what you write.

Collaborating with Others

I encourage you to talk with other students—in fact, with anyone—about the topics we are covering. And I also enourage you to send me email with questions; I'll be happy to get back to you as quickly as I can. However, the writeups you hand in must be entirely your own. You may not copy proofs, or algorithms, or parts of proofs or algorithms, or anything else, from anyone else's paper, or from the web, or from books, or from anywhere else, even if you give credit to that person or source. And you must be able to explain to me what you have written.

Please do take this seriously. I'm very happy to try to help you out in person or by email. I really do want every one of you to succeed, and I'll try hard to make that happen. But I have no tolerance at all for plagiarism. If I find evidence of it, you will receive an F for the course. And I don't give "second chances".

On-line resources:

Two web sites that you might profitably look at are:

On both these web sites you can find lecture notes and videos of course lectures.

Some University Policies:

• Student Conduct and Plagiarism

Students are required to adhere to the polices expressed at the University web site Academic Policies and Rights.

• Accommodations for Students with Disabilities

Section 504 of the Americans with Disabilities Act of 1990 offers guidelines for curriculum modifications and adaptations for students with documented disabilities. If applicable, students may obtain adaptation recommendations from the Ross Center for Disability Services, (Campus Center, UL Room 211; 617-287-7430). The student must present these recommendations and discuss them with each professor within a reasonable period, preferably by the end of Drop/Add period.