Neural Net Experiments, Feedforward Networks




CS256

Chris Pollett

Oct 11, 2017

Outline

Neural Net Experiments

What should a Neural Network Experiment contain?

  1. A statement of a hypothesis which could be proven true or false.
    • The hypothesis should be settling something interesting to know about your neural net. For example, how well it generalizes.
    • Your hypothesis should be robust under tweakings of hyperparameters. I.e., if you can change the value of the answer of your hypothesis too easily by tweaking hyperparameters, then it suggests your results don't generalize.
  2. A description of your neural net and training set ups for the experiment conducted. This should be in sufficient details so that someone else could try to replicate your experiment.
  3. The results of your experiments. I.e., what was the output of your experiments.
  4. A conclusion from your results with regard to your initial hypothesis.

Things to Measure with Experiments 1 - Accuracy , Confusion Matrix

What kind of hypothesis we have is dependent on the things we can actually measure with an NN experiment. On the next few slides I give some examples of the kinds of things we can measure. After this, I give some example hypothesis which could be tested using these kind of experiments.

Things to Measure with Experiments 2 - ROC Curve

Scatter Plot Example ROC Curve Example

Things to Measure with Experiments 3 - Performance

Example Hypotheses

Representing Our Results - Graphing and Charting

matplotlib - Basic Graphing

matplotlib - Scatter Plots, Bar, Histograms, Pie Charts

matplotlib - Saving Images

In-Class Exercise

Maximizing Your Test Data - Cross Validation

Estimators

Estimator Bias

The Gaussian Distribution and an Example Biased Estimator

Cross-Validation and Bias

Recurrent, MLP, and Feedforward Networks

Computing a function from a feedforward MLP