CS256 Fall 2017 Sec1Home Page/Syllabus

Topics in Artificial Intelligence

Instructor: Chris Pollett
Office: MH 214
Phone Number: (408) 924 5145
Email: chris@pollett.org
Office Hours: MW 5:45pm-7pm
Class Meets:
Sec1 MW 4:30pm-5:45pm in SCI 311

Prerequisites

To take this class you must have taken: CS156 with a grade of B- or better.

Texts and Links

Required Texts: Deep Learning. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. MIT Press. 2016.
Online References and Other Links: Official Python Website.
Python Implementation of Code from the Book.

Description

Artificial Neural Networks provide a general model for computers to learn how to perform a variety of tasks. With the advent of both cheap hardware, and large scale data sets, it has become increasingly possible to train computers to perform tasks that previously could only be carried out by humans, if at all. In this class, we will take both a pragmatic and theoretical approach to how to design and build artificial neural networks. We will start with an overview of machine learning and the math needed for machine learning. We will discuss single neuron/layer networks (perceptron, SVM, etc) and how to train them. We will then look at multi-layer networks and variations on the back-propagation training algorithm. We will consider different kinds of neural networks layer types, such as convolutional neural network layer, recurrent neural network layers, etc and when and why we would use these different kinds of layers. To make our models generalize we will consider different a variety of regularization techniques. As the class goes on, we will consider how to design neural networks for different applications such as vision tasks like object classification and detection, language processing, planning tasks, and policy selection. We will also learn how to tune our networks and determine how data we need to be able to effectively train them. Code for this class will be in Python and we will use a library like Caffe (useful because of the Intel Movidius Compute Stick) or Tensorflow for our more sophisticated networks.

Course Learning Outcomes (CLOs)

By the end of this course, a student should be able to:

CLO1 -- Be able to code without a library a single perceptron training algorithm.

CLO2 -- Be able to predict the effect of different activation functions on the ability of a network to learn.

CLO3 -- Be able to explain how different neural network training algorithms work.

CLO4 -- Be able to select neural network layers type to build a network suitable for various learning tasks such as object classification, object detection, language processing, planning, policy selection, etc.

CLO5 -- Be able to select an appropriate regularization technique for a given learning task.

CLO6 -- Be able to code and train with a library such as Caffe, Theano, Tensorflow a multi-layer neural network.

CLO7 -- Be able to measure the performance of a model, determine if more data in needed, as well as how to tune the model.

Below is a tentative time table for when we'll do things this quarter:

Week 1: Aug 23 Syllabus (I will be at CAIP 2017)
Week 2: Aug 28, Aug 30 Read Ch 1 Intro, Skim Ch 2 to 5 (over next few weeks), Background Probability
Week 3: Sep 4 (No Class, Labor Day), Sep 6 Background linear algebra
Week 4: Sep 11, Sep 13 Perceptrons, PAC Learning, Start Python
Week 5: Sep 18, Sep 20 Python, Ch 5
Week 6: Sep 25, Sep 27 Perceptron Networks and p-time, SVMs
Week 7: Oct 2, Oct 4 Review, Midterm 1
Week 8: Oct 9, Oct 11 Numpy, Pillow, Neural Net Experiments, Ch 6 Feedforward Networks
Week 9: Oct 16, Oct 18 More Ch 6 Cost Functions, Output Layers, Minimization Methods, Hidden Units, Stochastic Gradient Descent
Week 10: Oct 23, Oct 25 Finish Ch 6, Backpropagation, Tensorflow, Ch 7 Regularization
Week 11: Oct 30, Nov 1 (Midterm) Review, Midterm 2
Week 12: Nov 6, Nov 8 Finish Ch 7, Ch 8 Learning Algorithms as an Optimization Problem
Week 13: Nov 13, Nov 15 Finish Ch 8, Ch 9 Convolutional Networks
Week 14: Nov 20, Nov 22 (No Class) Ch 10 Recurrent and Recursive Networks
Week 15: Nov 27, Nov 29 Ch 11 Tuning and Debugging Networks, Ch 12 Neural Net Applications
Week 16: Dec 4, Dec 6 Ch 13 Linear Factor Models, Ch 14 Autoencoders
Week 17: Dec 11 (Hw5 due) Review
The final will be 2:45pm-5:00pm, Tuesday, December 19

Grading

Homeworks and Quizzes 50%
Midterm 1 15%
Midterm 2 15%
Final 20%
Total100%

Grades will be calculated in the following manner: The person or persons with the highest aggregate score will receive an A+. A score of 55 will be the cut-off for a B-. The region between this high and low score will be divided into five equal-sized regions. From the top region to the low region, a score falling within a region receives the grade: A, A-, B+, B, B-. If the boundary between an A and an A- is 85, then the score 85 counts as an A-. Scores below 55 but above 50 receive the grade D. Those below 50 receive the grade F.

If you get an A- or better in this class and want me to write you a letter of recommendation, I will generally be willing provided you ask me within two years of taking my course. Be advised that I write better letters if I know you to some degree.

Homework and Project Info

This semester we will have five homeworks, weekly quizzes, and weekly in-class exercises.

Every Monday this semester, except the first day of class, the Midterm Review Day, and holidays, there will be a quiz on the previous week's material. The answer to the quiz will either be multiple choice, true-false, or a simple numeric answer that does not require a calculator. Each quiz is worth a maximum of 1pt with no partial credit being given. Out of the total of twelve quizzes this semester, I will keep your ten best scores.

On Wednesday's, we will spend 15-20 minutes of class on an in-class exercise. You will be asked to post your solution to these exercises to the class discussion board. I will set you up with an account before the first exercise. Posting your solution is worth 1 "pre-point" towards your grade. A "pre-point" can be used to get one missed point back on a midterm or final, up to half of that test's total score. For example, if you scored 0 on a midterm and have 10 pre-points, you can use 7.5 of your 10 prepoints, so that your midterm score is a 7.5. On the other hand, if you score 13/15 on the midterm, you can use at most 1 pre-point since half of what you missed (2pts) on the midterm is 1pt. Your prepoints for the whole semester will be distributed among all three tests (2 midterms and final), so as to maximize your score.

Links to the current list of homeworks and quizzes can be found on the left hand frame of the class homepage. After an assignment has been returned, a link to its solution (based on the best student solutions) will be placed off the assignment page. Material from assignments may appear on midterms and finals. For homeworks you are encouraged to work in groups of up to three people. Only one person out of this group needs to submit the homework assignment; however, the members of the group need to be clearly identified in all submitted files.

Homeworks for this class will be submitted and returned completely electronically. To submit an assignment click on the submit homework link for your section on the left hand side of the homepage and filling out the on-line form. Hardcopies or e-mail versions of your assignments will be rejected and not receive credit. Homeworks will always be due by the start of class on the day their due. Late homeworks will not be accepted and missed quizzes cannot be made up; however, your lowest score amongst the five homeworks and your quiz total will be dropped.

For this class, I expect each student to have available a laptop with recent version of Python installed (either 2.7.x or 3.x branch). Your laptop will be used whenever you want to show me something in my office concerning one of your projects.

When doing the programming part of an assignment please make sure to adhere to the specification given as closely as possible. Names of files should be as given, etc. Failure to follow the specification may result in your homework not being graded and you receiving a zero for your work.

Exams

The midterms will be during class time on: Oct 4 and Nov 1.

The final will be: 2:45pm-5:00pm, Tuesday, December 19.

All exams are closed book, closed notes and in this classroom. You will be allowed only the test and your pen or pencil on your desk during these exams. Beeper or cell-phone interruptions will result in immediate excusal from the test. The final will cover material from the whole quarter although there will be an emphasis on material after the last midterm. No make ups will be given. The final exam may be scaled to replace a midterm grade if it was missed under provably legitimate circumstances. These exams will test whether or not you have mastered the material both presented in class or assigned as homework during the quarter. My exams usually consist of a series of essay style questions. I try to avoid making tricky problems. The week before each exam I will give out a list of problems representative of the level of difficulty of problems the student will be expected to answer on the exam. Any disputes concerning grades on exams should be directed to me, Professor Pollett.

Regrades

If you believe an error was made in the grading of your program or exam, you may request in person a regrade from me, Professor Pollett, during my office hours. I do not accept e-mail requests for regrades. A request for a regrade must be made no more than a week after the homework or a midterm is returned. If you cannot find me before the end of the semester and you would like to request a regrade of your final, you may see me in person at the start of the immediately following semester.

University Policies and Procedures

Per University Policy S16-9, university-wide policy information relevant to all courses, such as academic integrity, accommodations, etc. will be available on Office of Graduate and Undergraduate Programs' Syllabus Information web page at http://www.sjsu.edu/gup/syllabusinfo/. Below are some brief comments on some of these policies as they pertain to this class.

Academic Integrity

For this class, you should obviously not cheat on tests. For homeworks, you should not discuss or share code or problem solutions between groups! At a minimum a 0 on the assignment or test will be given. A student caught using resources like Rent-a-coder will receive an F for the course. Faculty members are required to report all infractions to the Office of Student Conduct and Ethical Development.

Accommodations

If you need a classroom accommodation for this class, and have registered with the Accessible Education Center, please come see me earlier rather than later in the semester to give me a heads up on how to be of assistance.