More Regularization




CS256

Chris Pollett

Nov 6, 2017

Outline

Introduction

Regularization and Under-Constrained Problems

Data Augmentation

Noise Robustness

Semi-Supervised Learning

Semi-Supervised Learning - Generative and Discriminative Models

Quiz

Which of the following is true?

  1. The update back propagation rule always updates the `i`th weight of the `j`th neuron of the `k`th layer by the same amount as the `i`th weight of the `j`th neuron of the `k+1`st layer.
  2. symbol-to-number differentiation is how Theano and Tensorflow evaluate partial derivatives.
  3. `L_2`-parameter regularization is a kind of parameter norm penalty.

Multitask Learning

Early Stopping

Early Stopping - Using all the data

Parameter Tying and Sharing