Artificial Neural Network

(Click to Download Slides)

Lecture 1: What are neural networks?
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: How neural networks learn from experience ( .pdf)

Lecture 2: Two simple learning algorithms
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Connectionist Learning Procedures, pp 185-190; 193-198. ( .pdf)

Lecture 3: Learning in multilayer networks
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 4: Learning to model relationships and word sequences
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: A neural probabilistic language model. (.ps file)

Lecture 5: Applying backpropagation to shape recognition
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 6: Learning in recurrent networks
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Learning internal representations by error propagation, pp 354-362. ( .pdf)

Lecture 7: Overfitting and ways to fix it
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: short web reading on generalization
Reading: short web reading on overfitting
Reading: short web reading on early stopping
Reading: short web reading on weight decay

Lecture 8: The Bayesian way to fit models
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 9: More on Bayesian model fitting
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 10: Speeding up learning
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 11: Learning without a teacher: Autoencoders and PCA
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 12: Clustering: The EM algorithm for fitting mixtures of Gaussians
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 13: Mixtures of experts
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Adaptive mixtures of local experts .pdf 

Lecture 14: Hidden Markov Models
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. pdf

Lecture 15 (called 17): Learning Hidden Markov Models using EM
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Alan Poritz, Hidden Markov Models: A guided tour., ICASSP 1988. .pdf

Lecture 16: Distributed representations
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Book chapter on "Distributed Representations" .pdf 

Lecture 17: The effects of hardware damage on distributed representations
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Scientific American article on "Simulating Brain Damage" .pdf 

Lecture 18: Hopfield Nets and simulated annealing
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: For a gentle introduction to the idea of memories as energy minima ( .ps ) ( .html )
Reading: For a gentle introduction to how to add new memories by creating new minima ( .ps ) ( .html ) 

Lecture 19: Boltzmann machines as probabilistic models
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 20: Learning in Boltzmann machines
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading: Scholarpedia entry on Boltzmann machines .pdf [ web page] 

Lecture 21: Some demonstrations of learning in restricted Boltzmann machines
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Lecture 22: Learning features one layer at a time
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading for lectures 22 and 23: "Learning multiple layers of representation". .pdf

Lecture 23: Sigmoid belief nets and the wake-sleep algorithm
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) 

Reading for lectures 22 and 23: "Learning multiple layers of representation". .pdf 

Lecture 24: Using backpropagation to fine-tune deep networks
(notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page))
Reading for lecture 24: "Reducing the dimensionality of data with neural networks" .pdf 

Lecture 25: Learning distributed representations for sequential data.

 (notes as .ppt ) (notes for all browsers)) (notes as .ps, 4 per page)) .pdf