CS 624: Analysis of Algorithms
office: 179 (I share this office with Prof. Bolker.)
email: offner "at" cs.umb.edu
In addition to this web page, I will put my lectures and other materials in the
directory ~offner/cs624 and subdirectories under it. Feel free to poke around
there. Anything there (or under there) is for you to take. (Note that these
are not web pages—they are ordinary files, and you have to log in to your
cs.umb.edu account to access them.)
- Week 1:
- Monday: Chapters 1-2 of the text.
- Wednesday: Chapters 3-4 of the text (read 4.1 quickly; you can skip
4.2 and 4.6).
due February 6: Introduction, some preliminary examples of analysis of
algorithms, order of growth of functions, recursions and growth of their
solutions, generating functions.
- Week 2:
- Monday: Chapter 6 of the text (Heapsort).
- Before Wedneday:
- At least the first 5 pages of my expository paper "Some Early
Analytic Number Theory", which you can find on my web page. This is not
hard reading, but it is essential for what we are going to be doing this
week (on Wednesday) and next week.
- The handout "Harmonic Numbers". I will give this to you in class,
but it is also available in the usual place.
- Wednesday: Chapter 7 of the text (Quicksort).
due February 17: Heaps, Heapsort, Priority Queues, Quicksort
Please note: February 17 is Presidents Day, and there is no class that
day. So please make sure to email me your papers by 7 PM on February 17,
and that way I will be able to return them in class on Wednesday. This is
important; I will not accept late papers.
- Week 3:
- Chapter 8 of the text (Sorting in Linear Time).
- The class notes, of course and also the handout "Stirling's Formula",
which I will hand out, but it's also available in the usual place.
due Monday, February 24: Lower bound for comparison sorting; some
- Week 4:
- Chapter 9 of the text (Medians and Order Statistics).
- Chapter 12 of the text (Binary Search Trees).
- The class notes, of course, and the notes on "Convex Functions", which
I handed out a week or so ago and which we discussed in class at the time.
You'll need this for Wednesday's lecture.
due Monday, March 3: Problems on trees and binary search trees.
Due to sloppiness on my part, the original Problem 2 on the assignment
(which was Exercise 1.2 in the lecture notes for Lecture 7) was just
incorrect. Please don't do it. It doesn't affect anything else.
I have deleted that question from the assignment, and also from the lecture
notes. The assignment now consists of 7 problems.
- Week 5:
- Chapter 15 of the text (Dynamic Programming), sections 15.3, 15.4, and
- Chapter 17 of the text (Amortized Analysis). You can skip the
discussion of Multipop (for stacks) and Section 17.4.2 (on deleting
elements from a dynamic table).
due Monday, March 10: Problems on dynamic programming.
- Week 6:
- Monday: Review for the first mid-term exam. Come with questions.
- Wednesday: First mid-term exam. It's "closed-book". No books, no
papers, no notes, no cell phones, ... . Just bring something to write
- Week 7:
- Monday: We'll finish up our treatment of amortized analysis by
covering the Sleator-Tarjan theorem on self-organizing lists.
Everything will be in the handout.
- Wednesday: We start a unit on graph algorithms. Today we'll cover
breadth-first search. This corresponds to Chapter 22, Sections 1
Assignment due Monday, March 31: Problems on amortized analysis
and dynamic programming.
- Week 8:
- Sections 22.3 through 22.5 in the text.
The first chapter of my expository article "Notes on Graph
Algorithms Used in Optimizing Compilers", which you can find on my web
Please be sure to print out a copy for yourself in
preparation for this week's classes, as I won't be handing it out, but
I will be going through it pretty closely. (If you don't have it with
you in class, you will be really lost.)
This covers the same material as the text, so you can just skim the
text if you want.
due Monday, April 7: breadth-first and depth-first walks and finding a
vertex cover for a tree.
- Week 9:
The Lecture 14 handout. This is on flows in networks, and the
max-flow min-cut theorem and some applications.
It covers pretty much what is in Chapter 26 of our text. However,
I strongly recommend that you do not read Chapter 26. The
reason is that I am using a different convention for representing such
networks than the book does, and if you try to read both the book and
my notes, you will really get confused.
due Monday, April 14: flows in networks.
- Week 10:
- Monday: Finish up the Lecture 14 handout.
- Wednesday: Review for the second mid-term exam.
- Assignment due Wednesday, April 23: just do the last two problems
from the previous assignment, which I postponed till this one.
- Week 11:
- Monday: No class -- university holiday.
- Wednesday: Second mid-term exam.
- Week 12:
- Reading: We start the topic of NP-completeness. We will cover up
through section 3.4 of the notes.
- Assignment due Monday, May 5: All the problems in the sections of the
notes we have covered, with one exception: Don't do Exercise 3.7.
- Week 13:
- Reading: We continue the topic of NP-completeness, up through the end
of Section 3 of the notes (page 29).
Assignment due Monday, May 12: Proving some problems are NP-complete
- Week 14:
- Monday: We finish our treatment of NP-completeness, up through the
end of the notes.
Wednesday: Course evaluation and review for the final. (And some
final comments that I think you will find interesting.) Come with
And by the way, "What's going to be on the final?" is not a question.
The final will cover the entire course. As usual, there will be more
questions than you can possibly do. So you don't have to worry if there
are some you get stuck on. Just do other ones.
- CS 320L: Applied Discrete Mathematics
This prerequisite is important. If you have not had it, and done well in it,
you should not be enrolled in this course.
Further, while CS 320L is the official prerequisite, there is some other
material that you really need to know in order to understand what we are doing
in this course. Undoubtedly you have studied this before, but I just want to
make it explicit here. I will not review this material in class—you
are responsible for understanding it.:
M/W 5:30-6:45 PM in room M-2-207 (that's the McCormack building)
Before class: M/W 4:30-5:15 PM
Some serious single-variable calculus: derivatives, the chain rule,
integrals, the "Fundamental Theorem of the Calculus", maxima and minima,
increasing and decreasing functions, convex functions, step functions.
(Do you even know what a step function is? If not, you need to find out
right away. Is a step function really a function? And how do you integrate
a step function?)
And of course you should never get confused between the two
f(x) = xa (for instance,
g(x) = ax (for instance,
Which one grows faster? And is the difference in growth significant?
(And I'm not putting these two questions here to practice my typing. Will
you be able to answer these questions if I ask them to you on the first day
- Some basic probability theory: combinatorics applied to probability,
expectation, conditional probability, independence. Appendices C.1-C.4 of the
text go over this material and include a number of good exercises many of
which you should be able to do yourself. And you should work on them before
the first class. If you have questions about them, send me email, and I'll be
glad to help. But I'm not going to cover this material in class—you are
responsible for it.
Introduction to Algorithms, Third Edition
by Cormen, Leiserson, Rivest, and Stein
MIT Press, 2009
I'm sure everyone in this class has taken one or more courses in which you
have learned some of the really great algorithms.This is not one of those courses. This is not a course in "really great
algorithms", or in writing programs, or even in writing pseudo-code.
This is for all practial purposes a mathematics course. It is a course on
the analysis of algorithms. We are going to prove theorems in this
course. In fact, that's just about all we are going to do. And when I say
we, I really mean you. The homework will consist of theorems
that you will have to prove. The exams will consist of theorems that you will
will have to prove. That's what we're going to do.
At a minimum, I plan to cover the following material this term:
There are many other topics that I'd really like to cover as well. This is just
a fascinating field. We'll see…
- Introductory material. What do we mean by analysis of algorithms? Why do
we care? Some basic examples.
- Mathematical background and some useful mathematical techniques. Orders
of growth of functions. Solving and estimating solutions of recursions.
- The analysis of Heapsort and Quicksort. Best-case, worst-case, and
- Lower bounds on sorting algorithms; bucket sort.
- Order statistics.
- The theory of binary search trees.
- Dynamic programming—how do we know when it is applicable?
- Amortized analysis.
- Graph algorithms; in particular:
- a detailed mathematical analysis of depth-first search, together
with some of its remarkable applications;
- another topic—possibly the max-flow min-cut theorem and its
- The theory of NP-completeness.
There will be one homework assignment each week. I will not accept
assignments handed in late.
There will be two in-class "mid-term" exams, and one final exam.
All the exams will be "closed-book" exams. No books, no notes, no
smartphones or other electronic equipment, and so on. Just bring something to
I will weight the work in the course as follows
when making up final grades:
- All the homeworks, taken together: 20%
- First mid-term exam: 20%
- Second mid-term exam: 20%
- Final exam: 40%
I will not give out "review sheets" for the exams. It
should go without saying, however, that the very best way to prepare for the
exams is to
- do the homework, and
- read the handouts carefully. If there is anything you don't understand
in them, you need to ask me about it. And the earlier you ask, the
This course does not involve any programming per se,
although without a doubt your experiences in building serious programs will help
you understand many of the issues we deal with.
In fact, as I already stated above, the course could well be regarded as a
mathematics course. I will be proving things in class, and you will be expected
to write formal proofs in your assignments and on exams. If this is something
you are not ready to come to grips with, you should wait until you are ready to
take this class.
There is a style to writing proofs. In fact, there are several styles. Your
goal should be to write proofs that are easy to read and that illuminate the
reason why something is true. This is not easy to do. Good writing of any kind
is difficult, and we will work hard at it all term.
Probably the single most important proof technique in this area is
mathematical induction. I will assume not only that you are familiar with this,
but are comfortable using it. I will not teach it, but I will expect to see
inductive proofs written out clearly and well.
I also expect you to be familiar with standard mathematical notations such as
summations (and of course, integrals). Some of this is reviewed in the
appendices of the text. I will not review this in class. I expect you to know
it, and we will be using it a lot.
One thing is very important to bear in mind: like any other kind of writing,
a proof must be understandable. If I can't understand what you are
writing, then it can't be correct. And I'm not going to try to guess what you
had in mind. If you are uncertain how to express something, please just talk to
me or send me email, and I'll try to help. But don't even bother writing
something that can't be understood.
Think of this like you would when writing a computer program. If the compiler
or interpreter can't understand it, it's wrong. And you have to be really
careful with your use of language and terms. For instance, one thing beginning
programmers often get mixed up about is the correct use of "or" and "and". The
meaning in computer languages is not the same as in English (or any other
natural language, so far as I know). The same is true in proofs. We use
language in very precise and fixed ways in proofs, and we do this so that what
we write can be understood in its exact meaning by anyone at any time, anywhere
in the world.
Just for a simple example: the sentence "All horses are not white." does
not mean the same as "Not all horses are white." At least, not when
writing proofs. In ordinary English, these two sentences are generally regarded
as equivalent. But for our purposes they are not. You have to be careful about
Here's another example: The symbol "==>" means "implies". It does not
mean "then". So you can write
if x = 3 then x + 5 = 8
or you can write
x = 3 ==> x + 5 = 8
But you cannot write
if x = 3 ==> x + 5 = 8 !!! WRONG !!!
Please try to be careful about things like this. And please don't be offended
if I correct mistakes like this in what you write.
Collaborating with Others
I encourage you to talk with other students—in fact, with
anyone—about the topics we are covering. And I also enourage you to send
me email with questions; I'll be happy to get back to you as quickly as I can.
However, the writeups you hand in must be entirely your own. You may not copy
proofs, or algorithms, or parts of proofs or algorithms, or anything else, from
anyone else's paper, or from the web, or from books, or from anywhere else. And
you must be able to explain to me what you have written.
Please do take this seriously. I'm very happy to try to help you out in person
or by email. I really do want every one of you to succeed, and I'll try hard to
make that happen. But I have no tolerance at all for plagiarism. If I find
evidence of it, you will receive an F for the course. And I don't give "second
Two web sites that you might profitably look at are:
On both these web sites you can find lecture notes and videos of course
- The course as it was taught at MIT in 2005 by Charles Leiserson (one of the
authors of our text) and Erik Demaine:
- The course as it has been taught at Stony Brook by Steven Skiena: http://www.algorist.com. Skiena is the
author of another text—The Algorithm Design Manual—that is
just full of wonderful and insightful case studies and "war stories" about the
design and practical application of algorithms. It is well worth reading, and in the
past has been used as an auxiliary text in this class. I didn't want to make
it a required text, but you won't be wasting either your time or your money by
Some University Policies: