San Jose State University : Site Name

Navigation

Main Content

Goals of the course, JPL

Ronald Mak

Department of Computer Science
Department of Applied Data Science
Fall Semester 2023

Office hours: W 6:00 – 7:00 PM, Clark Hall CL 325
Office location: Clark Hall CL 325 (mostly working from home)
E-mail: ron.mak@sjsu.edu

CS/SE 153 Concepts of Compiler Design


Section 1: MW 4:30 - 5:45 PM room MQH 422


Assignments

# Date Due Assignment
1 Aug 21 Aug 28 Compile and execute a Pascal program

Assignment files: Presidents.pas    presidents.txt

Example Pascal program: EmployeeListing.pas
Example program input file: employees.txt
Example program output: employees.out.txt
2 Aug 28 Sep 11 Pascal Scanner

Simple Pascal: Simple.zip
Test input files: SquareRootTable.pas    ScannerTest.txt
Sample output: ScannerTest.out.txt
Sample solution: Asgn02Solution.zip
3 Sep 6 Sep 22 Simple Pascal interpreter

Test input files and expected output: TestWhile.txt    TestWhile.out.txt    TestIf.txt    TestIf.out.txt    TestFor.txt    TestFor.out.txt    TestCase.txt    TestCase.out.txt
4 Sep 18 Oct 6 Pcl interpreter using ANTLR

Executor skeleton: Asgn04Skeleton.zip
Test input files: TestWhile.txt    TestIf.txt    TestFor.txt    TestCase.txt    HelloWorld.txt    Temperature.txt    SquareRootTable.txt
Sample solution: Pcl4.zip
5 Oct 11 Oct 23 Pascal to Java Converter

Partial converter code: Asgn05.zip
6 Oct 23 Nov 6 Pascal Compiler

Partial compiler code: Asgn06.zip
7 Nov 1 Nov 13 Project Grammar

Lectures

Week Date Content
1 Aug 21 Slides: Goals; course learning outcomes; project teams; grading; postmortem assessment report; overview of the compilation process; compiler as translator; other forms of translation; translation demos; Pascal tutorials

Example Pascal program: SquareRootTable.pas
Example object code (Jasmin): SquareRootTable.j
Example language conversion (Pascal to Java): SquareRootTable.java
Aug 23 Slides: It's all about translation; conceptual design; major parts of compilers, converters, and interpreters; compilers vs. interpreters; three Java packages; syntax diagrams; how to scan for tokens; basic scanning algorithm; test the scanner; current character vs. next character; consume characters and tokens

Sample code: Simple.zip
2 Aug 28 Slides: Assignment #2; Pascal statement syntax diagrams; expressions; operator precedence; control statements; REPEAT statement syntax; parse tree design and operations; building a parse tree; test the parser; parse tree nodes; printing parse trees
Aug 30 Slides: Building the parse tree with parser methods; handling syntax errors; the symbol table and its entries; a hack; a simple interpreter; what is "run time"; using the parse tree and symbol table at run time; another hack; parse tree visit methods

Sample erroneous code: HelloErrors-1.txt    HelloErrors-2.txt
3 Sep 4 Holiday
Sep 6 Slides: Simple interpreter; visiting parse nodes during run time; REPEAT statement; LOOP subtree; evaluating expressions; Assignment #3; WHILE statement; IF statement and the dangling ELSE; FOR statement; CASE statement
4 Sep 11 Slides: Assignment #3; maximum munch; an executor improvement; syntax and semantics; options for error recovery; top-down recursive descent parsing; accomplishments so far; temporary hacks; a DFA scanner

DFA scanner: SimpleDFAScanner.java    SimpleDFAInput.txt
Sep 13 Slides: BNF; grammars and languages; derivations and productions; EBNF; compiler-compilers; ANTLR 4; ANTLR lexer; ANTLR grammar file; Java main programs; on the command line; Eclipse ANTLR plug-in

ANTLR example: Expr.g4    ExprMain.java    input.txt    error.txt
5 Sep 18 Slides: What does ANTLR do for us?; ANTLR workflow; ANTLR parse trees; syntax error handling; resolving ambiguities; visitor interface; interface ExprVisitor; base visitor class ExprBaseVisitor; labeled production rules; interface ExprLabeledVisitor; base class ExprLabeledBaseVisitor; class Executor

ANTLR example: ExprLabeled.zip
Sep 20 Slides: Pcl4 grammar; package structure; Pcl4 visitor interface; Pcl4 base visitor class; class Executor; Assignment #4; declarations; constant definitions; simple type definitions; array type definitions; record type definitions; variable declarations; declarations and the symbol table; scope and the symbol table stack
6 Sep 25 Slides: Scope and the symbol table stack; nested scopes; scope of record fields; class Symtab; class SymtabStack; class SymtabEntry; Pascal declarations; grammar file Pcl5.g4; declarations and the symbol table; type specification attributes; class Typespec

ANTLR example: Pcl5.zip
Sep 27 Slides: Syntax and semantics; multipass compilers; software engineering; type declaration structures; multidimensional array; cross-reference listing; predefined scalar types; type checking; assignment and comparison compatible; type checking expressions; class TypeChecker; visit methods for type definitions; new fields for parse tree nodes; Pcl6

ANTLR example: Pcl6.zip
7 Oct 2 Slides: Pascal.g4; pass 3 uses node info from pass 2; runtime memory management; symbol table stack vs. runtime stack; stack frame; access to nonlocal variables; runtime display; recursive calls; allocate a stack frame; runtime error checking; Pascal interpreter

ANTLR example: PascalInterpreter.zip
Oct 4 Slides: Grammar for a variable; type check a variable; grammar for a procedure or function definition; symbol table entry for a procedure or function; grammar for procedure and function calls; type check a call; source-level debugger; debugger command language; review for the midterm

ANTLR example: PascalDebugger.zip

Midterm review questions
8 Oct 9 Midterm
Oct 11 slides Interpreter, converter, compiler; converting Pascal to Java; target machines for compilation; Java Virtual Machine (JVM) architecture; JVM runtime stack and stack frame; Jasmin assembler; Jasmin assembly instructions; shortcut instructions
9 Oct 16 Slides: Example conversion of Pascal to Java; local variables, load and store, arithmetic, other instructions; code templates; compilation strategy; Jasmin datatype descriptors; program fields; code template for the main method; loading and storing a program variable's value; code for procedures and functions; compiling local variables; generating code for expressions; comparing integer values and other datatypes
Oct 18 Slides: Jasmin code for assignment statements; load and store tips; What Would James Gosling Do?; Jasper; relational expressions; code templates for IF statements and looping statements; code for Newton's square root function; code for FOR and CASE statements; code for procedures and functions and calls; code for the main program
10 Oct 23 Slides: Calling methods on objects; invokestatic vs. invokevirtual; code for println() and printf(); final compiler project
Oct 25 Slides: Resource acquisition is initialization (RAII); Pascal arrays; code generation for arrays and subscripts; allocate memory for scalar arrays; access an element of a 2-D array; subscripted variables; allocate memory for non-scalar arrays; 1-D array of strings; 2-D array of strings; code for records and fields; passing parameters; runtime libraries
11 Oct 30 Slides: backend compiler architecture
Nov 1 Slides: Code generation; instruction selection; register allocation; data flow analysis; instruction scheduling; code optimization: safety, profitability, constant folding, constant propagation, strength reduction, dead code elimination, loop unrolling, common subexpression elimination; debugging and optimizing compilers; compiling object-oriented languages

Code optimization example: Unoptimized.pas   Unoptimized.j   Optimized.j
12 Nov 6 Slides: Passing parameters: Java and C++; Java hack; runtime libraries; library example; code generation for WHILE, IF, FOR, CASE, and procedure call

Pass-by-reference examples: ScalarRef.cpp   ObjectRef.cpp   ObjectRef.java   IWrap.java   ReferenceHack1.java   ReferenceHack2.java
Runtime library example: Multiplier.java   LibraryTest.java   LibraryTest.j
Nov 8 Slides: Static and dynamic scoping; runtime memory management; heap management; garbage collection algorithms: reference counting, mark and sweep, stop and copy, generational; aggressive heap management; change Java's heap size; garbage collection research
13 Nov 13 Slides: On writing compilers; compilation challenges; speed comparisons; stack machine vs. register machine; CISC vs. RISC; a GUI-based debugger; integrated development environment (IDE); interprocess communication
Nov 15 Slides: Regular grammar for tokens; encoding a DFA transition matrix; context-free and context-sensitive grammars; top-down and bottom-up parsers; shift-reduce parsing; yacc and lex

Example grammar: calc.l   calc.y
14 Nov 20 Slides: Syntax-directed translation; attribute grammars; LL(1) and LR(0) parsers; FIRST and FOLLOW sets; three-address code; data-flow analysis; JavaCC
15 Nov 27 Slides: Case study
Nov 27 Compiler lab

Example interpreter/converter/compiler: Pascal.zip
For about 85% of classic Pascal. Unfortunately, the compiler still has the off-by-one bug that TestProcedure.pas exposes. The final emitted ".limit stack 7" directive is one too small.

Goals of the course

This course will concentrate on practical aspects of compiler construction, programming language design, and engineering a large, complex software application.

This is a challenging course that will demand much of your time and effort throughout the semester.

Course Learning Outcomes (CLO)

Prerequisites

Department policy is to enforce
all course prerequisites strictly

CS 47 Introduction to Computer Systems
or CMPE 102 Assembly Language Programming
grade C- or better
CS 146 Data Structures and Algorithms grade C- or better
CS 154 Formal Languages and Computability grade C- or better

Required book

The Definitive ANTLR 4 Reference, 2nd edition
Terence Parr
Pragmatic Bookshelf, 2012
ISBN: 978-1934356999
http://www.antlr.org/

Recommended book

Writing Compilers and Interpreters, 3rd edition
Ronald Mak
Wiley Publishing, Inc., 2009
ISBN: 978-0-470-17707-5
Source files

Online Pascal tutorials

We will use Pascal as the example source language.
Pascal Tutorial  looks very good. It has an online Pascal compiler.
Learn Pascal  also looks good, although it doesn't appear to cover set types.

Online Pascal development sites

You can edit, compile, and run Pascal programs online.
http://rextester.com/l/pascal_online_compiler
https://www.tutorialspoint.com/compile_pascal_online.php
https://www.jdoodle.com/execute-pascal-online

Other useful tutorials.

Install the ANTLR 4 plug-in for Eclipse, etc.