Neural networks: Theory and Applications

Spring 2018 schedule:  COS495

Organization of synaptic connectivity as the basis of neural computation and learning. Perceptrons, convolutional nets, and recurrent nets. Backpropagation and Hebbian learning. Engineering applications including computer vision and natural language processing.


  • Two 80 minute lectures and one precept per week.
  • Grades (A-F) will be based on class participation (5%), problem sets (25%), midterm (30%), and final exam (40%).
  • Participation includes speaking up in lecture and precept.
  • Participation also includes activity on Piazza—ideally asking good questions, giving good answers, and upvoting others’ contributions.


  • Familiarity with linear algebra.
  • Basics of optimization and probability theory.
  • Knowledge of Julia or Python (or willingness to learn).


Prof. Sebastian Seung will lecture twice a week.

MW 3-4:20pm McCosh Hall 28


  • Andrea Giovannucci               agiovann at          Office hours: Mon 3.15-5.15 PM
  • Mark Ioffe                               mioffe at              Office hours: Mon 3.15-5.15 PM

Lecture Schedule

  • Feb. 25. Hierarchical perceptrons for vision.
    • Assignment 4. Backpropagation for large datasets (ps4). Download HW4StarterCode (Due Fri March 6)
  • Mar. 2. Backprop-through-time for recurrent networks.
  • Mar. 4. Hebb rule and competitive learning.
  • Mar. 9. Oja’s rule and principal components analysis.
  • Mar. 11. Midterm Exam.
  • spring recess
  • Mar. 23. Linear networks. Amplification and attenuation.
  • Mar. 25. Invertebrate and vertebrate retinas.
  • Mar. 30. Hybrid analog-digital computation. Permitted and forbidden sets.
  • Apr. 1. Constraint satisfaction. Stereopsis.
  • Apr. 6. More ConvNets
  • Apr. 8. Deep reinforcement learning.
  • Apr. 13. LSTM and handwriting generation.
  • Apr. 15. Language modeling.
  • Apr. 20. Translation.
  • Apr. 22. Caption generation.
  • Apr. 27. Question answering.
  • Apr. 29. Symbolic reasoning.