Prof. Yanjun Qi, PhD @ UVA (email: yanjun at virginia.edu)   

The List of papers we have read about Deep Learning

  • This page includes a (growing) list of papers and lectures we read about deep learning and related.
  • Please feel free to email me (yanjun@virginia.edu), if you have related comments, questions or recommendations.
  • BTW: The covered materials are by no means an exhaustive list, but are papers that we have read or plan to learn in our reading group.


Tag Title and Information URLs (Paper/Video/Slide) Year

Topic V: Optimization of Deep learning Models

Tag Title and Information URLs (Paper/Video/Slide) Year
SGD Tutorial: Optimization Methods for Large-Scale Machine Learning / Léon Bottou, Frank E. Curtis, Jorge Nocedal (PDF) 2016
SGD Tutorial: Efficient BackProp/ Yann Lecun, Leon Bottou, Genevieve Orr, Klaus-Robert Muler (PDF) 1998

Topic IV: Theory (Structures, generative, discriminative, ...) of Deep learning Models

Tag Title and Information URLs (Paper/Video/Slide) Year
RNN Paper: Long Short-Term Memory (PDF) (BlogExplain) 1997
NTM Paper: Neural Turing Machines / Alex Graves, Greg Wayne, Ivo Danihelka (PDF) 2015
NTM Paper: Hybrid computing using a neural network with dynamic external memory/ Alex Graves, etal, Koray Kavukcuoglu, Demis Hassabis (PDF) 2015
Memory Paper: End-To-End Memory Networks / Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob Fergus (PDF) 2015
s2sLSTM Paper: Sequence to Sequence Learning with Neural Networks / Ilya Sutskever, Oriol Vinyals, Quoc V. Le (PDF) 2015
s2sLSTM Paper: Neural Machine Translation by Jointly Learning to Align and Translate / Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio (PDF) 2016
s2sLSTM Paper: Pointer Networks /Oriol Vinyals, Meire Fortunato, Navdeep Jaitly (PDF) 2015
s2sLSTM Paper: Order Matters: Sequence to Sequence for Sets / Oriol Vinyals, Samy Bengio, Manjunath Kudlur (PDF) 2016
s2sLSTM Paper: Matching Networks for One Shot Learning / Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, Daan Wierstra (PDF) 2016

Topic III: Adversarial Deep Learning Papers we read

Tag Title and Information URLs (Paper/Video/Slide) Year
GAN Paper: Generative Adversarial Networks / Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio (PDF) 2014
GAN Paper: Energy Based Generative Adversarial Networks / Junbo Zhao, Michael Mathieu and Yann LeCun (PDF) 2017
GAN Paper: Towards Principled Methods for Training Generative Adversarial Networks /Martin Arjovsky, Soumith Chintala, Léon Bottou (PDF) 2017
GAN Paper: Wasserstein GAN /Martin Arjovsky, Léon Bottou (PDF) 2017
Robust Paper: Intriguing properties of neural networks / Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, Rob Fergus (PDF) 2013
Robust Paper: Explaining and Harnessing Adversarial Examples / Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy (PDF) 2014

Topic II: Application and Benchmarking Deep learning papers

Tag Title and Information URLs (Paper/Video/Slide) Year
bench paper: Comparative Study of Deep Learning Software Frameworks (PDF) 2015
bench paper: Benchmarking State-of-the-Art Deep Learning Software Tools (PDF) 2016
App Paper: Learning to rank with (a lot of) word features (PDF) 2010
App Paper: Natural Language Processing (almost) from Scratch / Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, Pavel Kuksa (PDF) 2011

Topic I: Deep Learning Basics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓DeepBasic the list of video lectures related to DEEP Learning we have learned @ (ListVideo) 2015-now
✓DeepBasic Nando de Freitas: Course: Deep learning at Oxford 2015 (ListVideo) 2015
✓Deep DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part1 - deepNN supervised) (Video) + (PDFslide) 2012
✓Deep DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part2 - deepNN unsupervised) (Video) + (PDFslide) 2012
✓DeepStructured DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part3 - deepNN graph transformer network) (Video) + (PDFslide) + (PDF2) 2012
✓DeepGM DeepLearningSummerSchool12: Geoffrey Hinton: Introduction to Deep Learning , Deep Belief Nets (Parts 1 / Relevant Paper: A fast learning algorithm for deep belief nets ) (Video) + (PDFslide) 2012
✓Hardware Parallel DeepLearningSummerSchool12: Marc'Aurelio Ranzato (Google Inc.), Large Scale Deep Learning (Video) + (PDFslide) 2012
✓Deep MLSS2005: Yann Lecun, Tutorial of Energy-based models (Video) +(Slide) 2005
✓Deep Geoffry Hinton: Learning Energy-Based Models of High-Dimensional Data (Video) + (PDFslide) 2012
✓DeepScaling KDD14: Yoshua Bengio, Scaling Up Deep Learning (Video) + (PDFslide) 2014
✓Deep Yoshua Bengio (University of Montreal): Representation Learning with auto-encoder / decoder variants (Video) 2012
✓Deep EML07: Yoshua Bengio (University of Montreal): Speeding Up Stochastic Gradient Descent (Video) + (PDFslide) 2007
✓Deep Matt Zeiler ( Founder and CEO of Clarifai Inc, ) : Visualizing and Understanding Deep Neural Networks (Video) 2015
✓DeepTheory DeepLearningSummerSchool12: Nando de Freitas (University of British Columbia) An Informal Mathematical Tour of Feature Learning (Video) + (PDFslide) 2012
✓Deep KDD14: Ruslan Salakhutdinov, Deep Learning (Video) + (PDFslide) 2014
✓RNN Nando de Freitas: Deep Learning Lecture 12: Recurrent Neural Nets and LSTMs (Video) + (Slide) 2015
✓DeepGenerative Hugo Larochelle: DLSS15: Deep Learning for Distribution Estimation (Video) + (Slide) 2015
✓DeepTheory Yoshua Bengio: DLSS15: Deep Learning:Theoretical Motivations (Video) + (Slide) 2015
✓DeepGenerative Yoshua Bengio: DLSS15: Deep Generative Models (Video) + (Slide) 2015
✓DeepOptim DLSS15: Ian Goodfellow: Tutorial on Neural Network Optimization Problems (Video) + (Slide) 2015
✓Deep DLSS15: Ian Goodfellow: Deep Adversarial Examples (Video) + (Slide) 2015
✓DeepOptim DLSS15: Adam Coates: Deep Learning (hopefully faster) (Video) + (Slide) 2015
✓Deep DLSS15: Leon Bottou: Multilayer Neural Networks (Video) + (Slide) 2015
✓DeepNLP DLSS15: Christopher Manning: NLP and Deep Learning 2: Compositional Deep Learning;
Graham Taylor, Deep Learning to compared
(Video1) (Video2) (Slide) + (VideoLearn2Compare) 2015
DeepRBM DLSS15: multiple tutorials related to RBM (Video-DeepGM) + (Video-RBM) + (Video-deepRBM) 2015
DeepRBM DLSS15: Aaron Courville, Variational Autoencoder and Extensions (Video-autoencoder) + (Video-variational) 2015
✓Deep DeepLearningSummerSchool16: Jeffrey Dean, Google, Inc. : Large Scale Deep Learning with TensorFlow (Video) + (PDFslide) 2016
✓Deep DeepLearningSummerSchool16: Alex Wiltschko, Twitter, Inc : Introduction to Torch (Video) + (PDFslide) 2016
✓Deep DeepLearningSummerSchool16:Ryan Olson, NVIDIA Corporation, GPU programming for Deep Learning (Video) + (PDFslide) 2016
✓Deep DSLL16: Edward Grefenstette, Google, Inc. : Beyond Seq2Seq with Augmented RNNs (Video) + (PDFslide) 2016
✓Deep DSLL16: Sumit Chopra, Facebook Reasoning, Attention and Memory (Video) + (PDFslide) 2016
✓Deep DSLL16: Shakir Mohamed, Google, Inc, Building Machines that Imagine and Reason: Principles and Applications of Deep Generative Models (Video) + (PDFslide) 2016
✓Deep DSLL16: Yoshua Bengio, Department of Computer Science and Operations Research, University of Montreal; A Brief Review of Recurrent Neural Networks (Video) + (PDFslide) 2016
many more exciting video tutorials @ http://videolectures.net
V:Optim VI:Structure III:Adversarial II:Apps I:Basics BackTop