CS451/651 Compilers
Bill Campbell
Spring 2008
Programming Exercise 2: Scanning  Java Tokens

Here you will have to modify the Lexer so as to be able to recognize almost all Java tokens.  The purpose of this part is to get you familiar with lexical analysis and is the first step toward implementing additional Java functionality.

Modify the Lexer to scan the following tokens.

 

1. Scan (and ignore) multi-line comments.
2. All Java operators.
3. All Java keywords.
4. Double precision literals (returned as DOUBLE_LITERAL).
5. Other literals in Java, eg FLOAT_LITERAL, LONG_LITERAL, etc.
6. Other representations of integrals (hexadecimal, octal, etc)
7. Other means for representing characters in Strings and chars. NB: Not all of this may be practical to implement – full Unicode is hairy.  In your narrative, discuss what you could implement and discuss what you chose not to and why..

 

Be sure that your narrative clearly describes what you have done.

This assignment is due Monday, February 25 at 5:30 p.m. sharp. You should leave a working copy in the subdirectory p2/ of your cs651 working directory. You should also bring along hard copy of your source, and the results of all of your tests and your narrative. Late papers (and late arrival to class!) will be marked down.

Your narrative will be very important in this assignment, since there are things left unsaid. I strongly suggest that you start out on this assignment immediately; there is lots of programming to do, there is lots of testing to do, and probably there will be lots of repairing to do. And, of course there is lots of writing to do. Have fun! The grade sheet is here.