CS 624: Analysis of Algorithms

CS 624: Analysis of Algorithms

Carl Offner

Spring 2011

office: 88
email: offner "at" cs.umb.edu
home phone: (978) 443-4697

All the on line material for this course will be found on the web at http://www.cs.umb.edu/~offner/cs624.

Assignments:

Week 1:
- Reading:
  - Monday: Chapters 1-2 of the text (Cormen et al.); Chapter 1 of Skiena.
  - Wednesday: Chapters 3-4 of the text (read 4.1 quickly; you can skip 4.2); Chapter 2 of Skiena.
- Assignment due January 31: Introduction, some preliminary examples of analysis of algorithms, order of growth of functions, recursions and growth of their solutions, generating functions.
Week 2:
- Reading:
  - Monday: Chapter 6 of the text (Heapsort).
  - Wednesday: Chapter 7 of the text (Quicksort).
  - Chapter 4 of Skiena. This chapter is really priceless.
  - At least the first 5 pages of my expository paper "Some Early Analytic Number Theory", which you can find on my web page. This is not hard reading, but it is essential for what we are going to be doing this week and next week.
  - In conjunction with that reading, read the handout on Harmonic Numbers that I passed out in class. We will need that for our work in Lecture 4.
- Assignment due February 7: Heaps, Heapsort, Priority Queues, and Quicksort.
Week 3:
- Reading:
  - Monday: Chapter 7 of the text (Quicksort).
  - Wednesday: Chapter 8 of the text (More about sorting.)
  - The handout on Stirling's Formula.
- Assignment due February 14.
Week 4:
- Reading:
  - Monday: Chapter 9 of the text (Medians and Order Statistics).
  - Wednesday: Chapter 12 of the text (Binary Search Trees.)
    You don't really have to read Section 12.4, because I cover it (in a different manner) in the handout, and I think you'll probably find the way I did it easier to understand. But certainly read Section 12.4 if you want to!
  - The handout on Convex Functions.
- Assignment due February 23. (Monday, February 21 is the Presidents Day holiday.)
Week 5:
- Reading (in addition to what is handed out in class):
  - Chapter 15 (dynamic programming), sections 15.3, 15.4, and 15.5.
  - Skiena Chapter 8 start browsing the chapter. You certainly don't have to read all of it, but it's very instructive and illuminating. Look at the War Stories in particular.
- Assignment due February 28: Dynamic programming.
Week 6:
- Reading (in addition to what is handed out in class):
  - Chapter 15 (dynamic programming), section 15.5.
  - Chapter 16 (greedy algorithms), sections 16.2, 16.3
  - Skiena Chapter 8 keep browsing the chapter. You certainly don't have to read all of it, but it's very instructive and illuminating. Look at the War Stories in particular.
- Assignment due March 7: Dynamic programming and greedy algorithms.
Week 7:
- Reading (in addition to what is handed out in class):
  - Chapter 10, Section 1 (stacks and queues) You'll need this for the homework assignment, and I'm sure you'll find it very simple. I won't cover this in class because I'm sure you already know this. But you should read it over just to make sure you are familiar with the algorithms, in particular the algorithm for implementing a queue in terms of an array of fixed size.
  - Chapter 17 (amortized analysis). You can skip the discussion of Multipop (for stacks) and Section 17.4.2 (on deleting elements from a dynamic table).
- Assignment due March 21: Amortized analysis.
  It should go without saying at this point, but let me say it once again: when you write an algorithm, you have to give some convincing reason why it works. It doesn't necessarily have to be a formal proof (unless I ask for one), but you can't just say, "here's the algorithm".
  
  Similarly, if you give an estimate of the cost of an algorithm or an operation, you have to justify it.
Week 8:
- Monday: Review for the mid-term.
- Wednesday. First mid-term exam. As announced on the syllabus, it's "closed-book".
Week 9:
- Reading (in addition to the class handout on Wednesday):
  - Monday: We go over the mid-term.
  - Wednesday. Read Chapter 22 up through section 22.2.
- Assignment due April 4: Breadth-first search.
Week 10:
- Reading:
  - Sections 22.3 and 22.4 in the text.
  - Notes from class. This is the first chapter of my expository article "Notes on Graph Algorithms Used in Optimizing Compilers", which you can find on my web site. This covers the same material as the text, so you can just skim the text if you want.
- Assignment due April 11: Depth-first search and some of its applications.
Week 11:
- Reading:
  - Notes from class: finish up through the algorithm for strongly-connected components.
  - Chapter 23. You can skip Kruskal's algorithm.
  - Notes from Wednesday's class on Chapter 23.
- Assignment due April 20: Depth-first search and some of its applications (continued); Minimal spanning trees.
Week 12:
There is no class on Monday. We will devote Wednesday to a review for the second mid-term exam, which will be on the following Monday, April 25.

Come with questions!
Week 13:
- Monday: Second Mid-Term Exam.
- Wednesday: We start the theory of NP-completeness.
- Assignment (due May 2): The main thing I want you to do is to read carefully the handout through page 4, and to read the beginning of Chapter 34 (through the end of Section 34.1). Don't worry about languages. We'll talk about them later.
  
  In addition, I want you to do two problems:
  - Exercise 1.1 (page 2) in the handout.
  - Exercise 1.3 (page 4) in the handout.
Week 14
- Reading: We continue working through the theory of NP-completeness. We'll cover through Section 3.6 in the Lecture 16 handout.
- Assignment (due May 9): All problems are in the Lecture 16 handout. As always, I'm looking for clear and careful explanations.
  - Problem 2.1 (page 7)
  - Problem 3.2 (page 9)
  - Problem 3.6 (page 13)
  - Problem 3.7 (page 13)
  - Problem 3.12 (page 19)
Week 15
We finish working through the Lecture 16 handout. Here's what we'll do:
- Monday: Sections 3.7 (Some more simple reductions) and 3.8 (the proof that SAT is NP-complete).
- Wednesday:
  - Course evaluation.
  - Some final remarks.
  - Review for the final exam. Come with questions!

Prerequisite:

CS L320: Applied Discrete Mathematics

This prerequisite is important. If you have not had it, and done well in it, you should not be enrolled in this course.

Class meetings: M/W 7:00-8:15 PM in room M-1-201

Office hours: Before class: M/W 6:00-6:45 PM

Texts:

The first text is required:
Introduction to Algorithms, Third Edition
by Cormen, Leiserson, Rivest, and Stein
MIT Press, 2009

The second text is not strictly speaking required, but I highly recommend it. It has a lot of good "war stories" in it, a lot of really excellent problems, and gives a good feeling for the subject. I will be assigning problems and readings from this book.
The Algorithm Design Manual, Second Edition
by Steven S. Skiena
Springer-Verlag, 2008

Syllabus:

At a minimum, I plan to cover the following material this term:

Introductory material. What do we mean by analysis of algorithms? Why do we care? Some simple examples.
Mathematical background and some useful mathematical techniques. Orders of growth of functions. Solving and estimating solutions of recursions. Generating functions. Probabilistic notions and techniques.
Heapsort and Quicksort.
Binary search trees.
Dynamic programming.
Greedy algorithms.
Amortized analysis.
Graph algorithms; in particular, depth-first search and some of its remarkable applications.
The theory of NP-completeness.

There are many other topics that I'd really like to cover as well. This is just a fascinating field. We'll see…

Course Work:

There will be one homework assignment each week. I will not accept assignments handed in late.

There will be two in-class "mid-term" exams, and one final exam.

All the exams will be "closed-book" exams.

Grading:

I will weight the work in the course as follows when making up final grades:

All the homeworks, taken together: 20%
First mid-term exam: 20%
Second mid-term exam: 20%
Final exam: 40%

I most likely will not give out "review sheets" for the exams. It should go without saying, however, that the very best way to prepare for the exams is to do the homework.

Writing Proofs:

This course does not involve any programming per se, although without a doubt your experiences in building serious programs will help you understand many of the issues we deal with.

In fact, the course could well be regarded as a mathematics course. I will be proving things in class, and you will be expected to write formal proofs in your assignments and on exams. If this is something you are not ready to come to grips with, you should wait until you are ready to take this class.

There is a style to writing proofs. In fact, there are several styles. Your goal should be to write proofs that are easy to read and that illuminate the reason why something is true. This is not easy to do. Good writing of any kind is difficult, and we will work hard at it all term.

Probably the single most important proof technique in this area is mathematical induction. I will assume not only that you are familiar with this, but are comfortable using it. I will not teach it, but I will expect to see inductive proofs written out clearly and well.

I also expect you to be familiar with standard mathematical notations such as summations (and of course, integrals). Some of this is reviewed in the appendices of the text. I will not review this in class. I expect you to know it, and we will be using it a lot.

One thing is very important to bear in mind: like any other kind of writing, a proof must be understandable. If I can't understand what you are writing, then it can't be correct. And I'm not going to try to guess what you had in mind. If you are uncertain how to express something, please just talk to me or send me email, and I'll try to help. But don't even bother writing something that can't be understood.

Think of this like you would when writing a computer program. If the compiler or interpreter can't understand it, it's wrong. And you have to be really careful with your use of language and terms. For instance, one thing beginning programmers often get mixed up about is the correct use of "or" and "and". The meaning in computer languages is not the same as in English (or any other natural language, so far as I know). The same is true in proofs. We use language in very precise and fixed ways in proofs, and we do this so that what we write can be understood in its exact meaning by anyone at any time, anywhere in the world.

Just for a simple example: the sentence "All horses are not white." does not mean the same as "Not all horses are white." At least, not when writing proofs. In ordinary English, these two sentences are generally regarded as equivalent. But for our purposes they are not. You have to be careful about this.

Here's another example: The symbol "==>" means "implies". It does not mean "then". So you can write

   if x = 3 then x + 5 = 8

or you can write

   x = 3 ==> x + 5 = 8

But you cannot write

   if x = 3 ==> x + 5 = 8   !!! WRONG !!!

Please try to be careful about things like this. And please don't be offended if I correct mistakes like this in what you write.

Collaborating with Others

I encourage you to talk with other students—in fact, with anyone—about the topics we are covering. And I also enourage you to send me email with questions; I'll be happy to get back to you as quickly as I can. However, the writeups you hand in must be entirely your own. You may not copy proofs, or algorithms, or parts of proofs or algorithms, or anything else, from anyone else's paper, or from the web, or from books, or from anywhere else. And you must be able to explain to me what you have written.

Please do take this seriously. I'm very happy to try to help you out in person or by email. I really do want every one of you to succeed, and I'll try hard to make that happen. But I have no tolerance at all for plagiarism. If I find evidence of it, you will receive an F for the course. And I don't give "second chances".

On-line resources:

Two web sites that you might profitably look at are:

The course as it was taught at MIT in 2005 by Charles Leiserson (one of the authors of our main text) and Erik Demaine: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-046JFall-2005/CourseHome/index.htm
The course as it has been taught at Stony Brook by Steven Skiena (the author of our other text): http://www.algorist.com

On both these web sites you can find lecture notes and videos of course lectures.

Some University Policies:

Student Conduct and Plagiarism
Students are required to adhere to the University Policy on Academic Standards and Cheating, to the University Statement on Plagiarism and the Documentation of Written Work, and to the Code of Student Conduct as delineated in the University Catalog and Student Handbook. The Code is available online at: http://www.umb.edu/student_services/student_rights/code_conduct.html.
Accommodations for Students with Disabilities
Section 504 of the Americans with Disabilities Act of 1990 offers guidelines for curriculum modifications and adaptations for students with documented disabilities. If applicable, students may obtain adaptation recommendations from the Ross Center for Disability Services, M-1-401, (617-287-7430). The student must present these recommendations and discuss them with each professor within a reasonable period, preferably by the end of Drop/Add period.