Prof. Yanjun Qi, PhD @ UVA (email: yanjun at virginia.edu)   

The List of tutorials we have collected for Learning Machine Learning

  • As a group, we constantly need to improve our knowledge of machine learning, to educate new members with basic tutorials, and to help existing members understand advanced topics.
  • This page includes a (growing) list of tutorials and video lectures for such a purpose. We hope this page is helpful to people who share simiar research interests or are interested with learning advanced topics related to machine learning.
  • Please feel free to email me (yanjun@virginia.edu), if you have related comments, questions or recommendations.
  • BTW: The covered tutorials are by no means an exhaustive list, but are topics which we have learned or plan to learn in our reading group.


Tag Title and Information URLs (Video + Slide) Talk Year

Basic Machine Learning Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓Basic Yanjun Qi (UVA CS6316), Course: Introduction to machine learning (ListSlide) 2015
✓Basic Andrew Ng, Course: machine learning (ListVideo) 2008
✓Basic Nando de Freitas, Course: machine learning (under-level) (ListVideo)
(ListLecture)
2013
✓Basic Nando de Freitas, Course: machine learning (graduate-level) (ListVideo)
(ListLecture)
2013
✓Basic Yaser Abu-Mostafa : Caltech course: Learning from data (ListVideo)
(ListLecture)
2013

Basics of Large-Scale Machine Learning Topics

✓BasicLarge Sanjiv Kumar (Columbia EECS 6898), Lecture: Introduction to large-scale machine learning (PDFSlideList) 2010
✓BasicLarge Alex Smola - Berkeley SML: Scalable Machine Learning: Syllabus (SyllabusURL) 2012
✓BasicLarge William Cohen - CMU Machine Learning with Large Datasets 10-605: Syllabus (SyllabusURL) 2014

Topic I: Deep Learning Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓DeepBasic Nando de Freitas: Course: Deep learning at Oxford 2015 (ListVideo) 2015
✓Deep DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part1 - deepNN supervised) (Video) + (PDFslide) 2012
✓Deep DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part2 - deepNN unsupervised) (Video) + (PDFslide) 2012
✓DeepStructured DeepLearningSummerSchool12: Yann LeCun (New York University), Deep Learning, Graphical Models, Energy-Based Models, Structured Prediction (Part3 - deepNN graph transformer network) (Video) + (PDFslide) + (PDF2) 2012
✓DeepGM DeepLearningSummerSchool12: Geoffrey Hinton: Introduction to Deep Learning , Deep Belief Nets (Parts 1 / Relevant Paper: A fast learning algorithm for deep belief nets ) (Video) + (PDFslide) 2012
✓Hardware Parallel DeepLearningSummerSchool12: Marc'Aurelio Ranzato (Google Inc.), Large Scale Deep Learning (Video) + (PDFslide) 2012
✓Deep MLSS2005: Yann Lecun, Tutorial of Energy-based models (Video) +(Slide) 2005
✓Deep Geoffry Hinton: Learning Energy-Based Models of High-Dimensional Data (Video) + (PDFslide) 2012
✓DeepScaling KDD14: Yoshua Bengio, Scaling Up Deep Learning (Video) + (PDFslide) 2014
✓Deep Yoshua Bengio (University of Montreal): Representation Learning with auto-encoder / decoder variants (Video) 2012
✓Deep EML07: Yoshua Bengio (University of Montreal): Speeding Up Stochastic Gradient Descent (Video) + (PDFslide) 2007
✓Deep Matt Zeiler ( Founder and CEO of Clarifai Inc, ) : Visualizing and Understanding Deep Neural Networks (Video) 2015
✓DeepTheory DeepLearningSummerSchool12: Nando de Freitas (University of British Columbia) An Informal Mathematical Tour of Feature Learning (Video) + (PDFslide) 2012
✓Deep KDD14: Ruslan Salakhutdinov, Deep Learning (Video) + (PDFslide) 2014
✓RNN Nando de Freitas: Deep Learning Lecture 12: Recurrent Neural Nets and LSTMs (Video) + (Slide) 2015
✓DeepGenerative Hugo Larochelle: DLSS15: Deep Learning for Distribution Estimation (Video) + (Slide) 2015
✓DeepTheory Yoshua Bengio: DLSS15: Deep Learning:Theoretical Motivations (Video) + (Slide) 2015
✓DeepGenerative Yoshua Bengio: DLSS15: Deep Generative Models (Video) + (Slide) 2015
✓DeepOptim DLSS15: Ian Goodfellow: Tutorial on Neural Network Optimization Problems (Video) + (Slide) 2015
✓Deep DLSS15: Ian Goodfellow: Deep Adversarial Examples (Video) + (Slide) 2015
✓DeepOptim DLSS15: Adam Coates: Deep Learning (hopefully faster) (Video) + (Slide) 2015
✓Deep DLSS15: Leon Bottou: Multilayer Neural Networks (Video) + (Slide) 2015
✓DeepNLP DLSS15: Christopher Manning: NLP and Deep Learning 2: Compositional Deep Learning;
Graham Taylor, Deep Learning to compared
(Video1) (Video2) (Slide) + (VideoLearn2Compare) 2015
DeepRBM DLSS15: multiple tutorials related to RBM (Video-DeepGM) + (Video-RBM) + (Video-deepRBM) 2015
DeepRBM DLSS15: Aaron Courville, Variational Autoencoder and Extensions (Video-autoencoder) + (Video-variational) 2015
✓Deep DeepLearningSummerSchool16: Jeffrey Dean, Google, Inc. : Large Scale Deep Learning with TensorFlow (Video) + (PDFslide) 2016
✓Deep DeepLearningSummerSchool16: Alex Wiltschko, Twitter, Inc : Introduction to Torch (Video) + (PDFslide) 2016
✓Deep DeepLearningSummerSchool16:Ryan Olson, NVIDIA Corporation, GPU programming for Deep Learning (Video) + (PDFslide) 2016
✓Deep DSLL16: Edward Grefenstette, Google, Inc. : Beyond Seq2Seq with Augmented RNNs (Video) + (PDFslide) 2016
✓Deep DSLL16: Sumit Chopra, Facebook Reasoning, Attention and Memory (Video) + (PDFslide) 2016
✓Deep DSLL16: Shakir Mohamed, Google, Inc, Building Machines that Imagine and Reason: Principles and Applications of Deep Generative Models (Video) + (PDFslide) 2016
✓Deep DSLL16: Yoshua Bengio, Department of Computer Science and Operations Research, University of Montreal; A Brief Review of Recurrent Neural Networks (Video) + (PDFslide) 2016

Topic II: Kernel Methods Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓Kernel Alexander J. Smola, Kernel methods and Support Vector Machines (Part3 ) (Video) + (PDFslide) 2008
✓ParamReduct Sanjiv Kumar (Columbia EECS 6898), Lecture: Kernel Methods (I: Scaling up kernel methods) (PDFslide) 2010
✓Hardware Parallel ICML08: Fast Support Vector Machine Training and Classification on Graphics Processors (Video) + (PDFslide) 2008
✓DataStructure ECML2007: Large Scale Learning with String Kernels, (Video) + (PDFslide) 2007
✓Advanced Francis R. Bach, INRIA: Multiple kernel learning for multiple sources (Video) + (PDFslide) 2008
✓Random Sanjiv Kumar (Columbia EECS 6898), Lecture: Randomized Algorithms (PDFSlide) 2010
✓Random ECML2007: Efficient Machine Learning using Random Projections (Video) 2007
✓Random NIPS2007: Random features for large-scale kernel machines (original paper PDF) + ECCV2012: Fourier Kernel Learning (Video) + (PDFslide) 2007
✓Random Fast Random Feature Expansions for Nonlinear Regression (Video) + (PDFslide) 2010
✓FastOptim NIPS2010: Multiple Kernel Learning and the SMO Algorithm (Video) + (PDFPaper) 2010
✓FastOptim Fast training of support vector machines using sequential minimal optimization. In Book: Advances in Kernel Methods - Support Vector Learning, MIT Press (PaperPDF) 1999
✓FastOptim ICML2006: Collbort & Bottou: Trading Convexity for Scalability. (PaperPDF)+ (Video) 2006
✓FastOptim ICML2008: Bordes & Bottou: LaRank, SGD-QN - Fast Optimizers for Linear SVM (SlidePDF)+ (Video) 2006
✓FastOptim JMLR 2005: Working Set Selection Using Second Order Information for Training Support Vector Machines (PaperPDF) 2005
✓FastOptim PEGASOS: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM (Slide) + (PaperPDF) 2005
✓Kernel NIPS2009: Fast Subtree Kernels on Graphs (Video) 2009
✓Kernel NIPS09: Locality-Sensitive Binary Codes from Shift-Invariant Kernels (Video) + (PDF) 2009
✓Kernel PASCAL07: Graph kernels and applications in chemoinformatics (Video) 2007
✓Kernel S.V.N. Vishwanathan, Random walk graph kernels and rational kernels (Video) 2007

Topic III: Optimization for ML or High-Dim/Sparsity Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓Basic Optim Professor Stephen Boyd, Stanford University, Stanford EE364a: Convex Optimization I (CourseList+Video+Lecture) 2014
✓Basic Optim Professor Stephen Boyd, Stanford University, Stanford EE364b: Convex Optimization II (CourseList+Video+Lecture) 2008
✓Basic Optim Convex Optimization and Applications - Stephen Boyd (Video) 2015
✓Basic Optim Mark Schmidt's MLSS (machine learning summer school) 2015 tutorial: (Video) + (Slide) 2015
✓Basic Optim Mark Schmidt's Note: Least Squares Optimization with L1-Norm Regularization (NotePDF) 2005
✓SparsityOptim KDD08: Trevor Hastie: Regularization Paths and Coordinate Descent (Video)+ (PDFslide) 2008
✓Sparsity ICML09: Group Lasso with Overlaps and Graph Lasso.
(Original Paper PDF)
(Video) 2009
✓ Optim Sparse Mark Schmidt: Fast Non-Smooth and Big-Data Optimization (Video) 2014
✓FastOptim Sanjiv Kumar (Columbia EECS 6898), Lecture: Kernel Methods (II: fast optimization of kernel methods) (PDFslide) 2010
✓FastOptim Sanjiv Kumar (Columbia EECS 6898), Lecture: Large-Scale Optimization Techniques (PDFslide) 2010
✓Optim MLSS2013: Stephen Wright (University of Wisconsin-Madison) Optimization 1-3 (video) + (slide) 2013
✓Sparsity Sanjiv Kumar (Columbia EECS 6898),
Lecture: Sparse Methods
(PDFslide) 2010
✓NonConvex Mark Schmidt's DLSS (deep learning summer school) 2015 tutorial: Non Smooth, Non Finite, and Non Convex Optimization (Video) + (Slide) 2015
✓NonConvex NIPS 2015 Workshop (LeCun) 15599 Non-convex Optimization for Machine Learning: Theory and Practice (Video) 2015
✓NonConvex NIPS 2015 Workshop (Anandkumar) 15598 Non-convex Optimization for Machine Learning: Theory and and Practice (Video) 2015
OptimDiscrete MLSS2014: Submodularity and Optimization -- Jeff Bilmes (VideoI-III)+ (PDFslide) 2014
Optim DeepSummer12: Jorge Nocedal (Northwestern University) Tutorial on Optimization methods for machine learning (video) + (PDFslide) 2012
Optim DeepSummerSchool12: Stephen Wright (University of Wisconsin-Madison) Some Relevant Topics in Optimization (PartI+II) (video) + (PDFslide) 2010
SparsityOptim DeepSummer12: Stephen Wright (University of Wisconsin-Madison) Sparse and Regularized Optimization (video) + (PDFslide) 2012
Sparse NIPS2009 tutorial: Francis R. Bach: Sparse Methods for Machine Learning: Theory and Algorithms (video) + (PDFslide) 2009
OptimAdvance NIPS10 tutorial: Stephen J. Wright: Optimization Algorithms in Machine Learning Tutorial (video) + (PDFslide) 2010
OptimDiscrete NIPS12: Satoru Fujishige, Submodularity and Discrete Convexity (Video) 2012
OptimDiscrete ICML13: Tutorial, Submodularity In Machine Learning New-Directions (Video) 2013
OptimDiscrete NIPS11: Francis R. Bach, Learning with Submodular Functions: A Convex Optimization Perspective (Video) 2011
✓HighDim Martin J. Wainwright, High-Dimensional Statistics: Intro @ SimonInstitute Bootcamp (Video1)(Video2)+ (slide1) (slide2) 2013
HighDim NIPS2010: Peter Buhlmann, High-dimensional Statistics: Prediction, Association and Causal Inference (Video)+ (PDFslide) 2011
HighDim Martin J. Wainwright, High-Dimensional Statistics: Some progress and challenges ahead (PDFslide) 2010
HighDim AISTAT11: Martin J. Wainwright, Convex Relaxation and Estimation of High-Dimensional Matrices (Video)+ (PDFslide) 2011
MiniMax ICM2014 VideoSeries IL12.13 : Martin Wainwright on constrained form of statistical MinMax, Privacy, Communication and Computation (Video) 2014

Topic IV: Graphical Model and Bayesian and Variational Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓GM MLSS2006: Sam Roweis, Machine Learning, Probability and Graphical Models (Part 1-4) (Video)+ (PDFslide) 2007
✓Basic David MacKay: course: Lecture 10: An Introduction To Bayesian Inference (II): Inference Of Parameters And Models (Video) + (PDFslide) + (CourseVideoList) 2012
✓BasicVariational David MacKay: course: Lecture 14: Approximating Probability Distributions (IV): Variational Methods (Video) + (PDFslide) 2012
✓BasicMCMC David MacKay: course: Lecture 12+13: Approximating Probability Distributions (II+III): Monte Carlo Methods (I): Importance Sampling, Rejection Sampling, Gibbs Sampling, Metropolis Method, Slice Sampling, Hybrid Monte Carlo, Over-relaxation, Exact Sampling (Video12)(Video13) + (slide12)(slide13) 2012
✓MCMC Basic MLSS2009: Iain Murray : Markov Chain Monte Carlo (Video)+ (Slide) 2009
✓GMTopic MLSS2009: David Blei, Topic Models (Part I+II) (Video)+ (PDFslide) 2009
✓GMBasic MLSS2012: Martin J. Wainwright: Tutorial Materials on Graphical Models, Variational Methods and Message-Passing (PDFNote) (Video-07)+ (Part1)+ (Part2)+ (Part3) 2012
✓MCMC MLSS2008: Nando de Freitas, Monte Carlo Simulation for Statistical Inference, Model Selection and Decision Making (Part 1-6) (Video) + (PDFslide) 2008
GMExp Wainwright and Jordan monograph:
More advanced material on exponential families, duality, and variational methods
(PDFpaper) 2008
GM MLSS2007: Zoubin Ghahramani, Graphical models (Part 1-6) (Video)+ (PDFslide) 2007
GM MLSS2004: Christopher Bishop: Graphical Models and Variational Methods (Video)+ (PDFslide) 2007
Density DeepSummer12: Iain Murray (University of Edinburgh) Density estimation (Video)+ (PDFpaper) 2008
✓GProcess Gaussian Processes in Practice Workshop 2006, David MacKay, : Gaussian Process Basics (Video)+ (PDFslide) 2006
DProcess MLSS12: Dilan Gorur, Yahoo! Research, Dirichlet Process: Practical Course (Video)+ (PDFpaper) 2012
Copulas ICML13 Tutorial: Gal Elidan, Copulas in Machine Learning (Part I+II) (Video) 2013
✓BayesianScaling (LSOLDM)2013: Nando de Freitas, Bayesian Optimization in a Billion Dimensions via Random Embeddings (Video)+ (PDFslide) 2013
✓GMScaling KDD2011: Ron Bekkerman, Misha Bilenko and John Langford, Scaling Up Graphical Model Inference (PDFSlide) 2011
✓GMScaling MLSS2009: Tom Minka, Microsoft Research, Approximate Inference (Video) + (PDFslide) 2009
GMScaling NIPS09: Pedro Domingos, Large-Scale Learning and Inference: What We Have Learned with Markov Logic Networks (Video)+ (PDFslide) 2009
✓GMScaling KDD14: Pedro Domingos, Principles of Very Large Scale Modeling (Video) + (PDFslide) 2014
✓GMScaling Ralf Herbrich, Distributed, Real-Time Bayesian Learning in Online Service (Video) + (PDFpaper) 2013

Topic V: Assorted: structured, low-rank, Metric, and more

Tag Title and Information URLs (Paper/Video/Slide) Year
✓Metric ICML07 Best Paper - Information-Theoretic Metric Learning (Video) + (PDF) 2007
✓Structured CIKM08: Charles Elkan, Log-linear Models and Conditional Random Fields (Video) + (PDF) 2008
✓Structured ECML2012: Thomas Gartner, Fraunhofer IAIS , Algorithms for Predicting Structured Data (Part 1-3) (Video) + (PDF) 2012
✓Structured MLG08: Thorsten Joachims, Structured Output Prediction with Structural SVMs (Video) + (PDF) 2008
✓Matrix MLSS2009 : Emmanuel Candes, Department of Statistics, Stanford University : Tutorial, Matrix Completion via Convex Optimization: Theory and Algorithms (Video) 2009
✓LowRank MLSS2011: Emmanuel Candes, Department of Statistics, Stanford University, Title: Low-rank modeling (Video) + (PDF) 2011
Matrix Matrix completion paper list, e.g. singular value thresholding (PaperList) 2008-11
✓LowRank Sanjiv Kumar (Columbia EECS 6898), Lecture: Matrix Approximations (Part I + Part II) (PDF-1)+ (PDF-2) 2010
✓LowRank ICML13 Tutorial: Tensor Decomposition Algorithms for Latent Variable Model Estimation (Video) 2013
✓Spectral Sham Kakade, Scalable Spectral Approaches for Learning Topics, Clusters, and Communities (JMLR paper: Tensor Decompositions for Learning Latent Variable Models) (Video) + (PDFpaper) 2014
✓Spectral Arik Azran, Department of Engineering, University of Cambridge: Tutorial, Spectral Clustering (Video) + (PDF) 2008
✓DimReduct Sanjiv Kumar (Columbia EECS 6898), Lecture: Dimensionality Reduction: (PDF) 2010
✓comSensing MLSS09: Emmanuel Candes, An Overview of Compressed Sensing and Sparse Signal Recovery via L1 Minimization (Video) 2009
✓comSensing Richard Baraniuk, "Compressive Sensing," ECE Lecturer Series, U.Delware (Video) 2012

Topic VI: Scalable / Parallel / Random / Streaming Related Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓ApproxNN Sanjiv Kumar (Columbia EECS 6898), Lecture: Approximate Nearest Neighbor Search (Part I + Part II) (PDF-1)+ (PDF-2) 2010
✓scalable Alex Smola: MLSS 2014: Scalable machine learning (Video) 2014
Basic Alex Smola: Scalable ML Course: statistics (Video) 2012
System Alex Smola: Scalable ML Course: System (Video) 2012
StochasticG Leon Bottou, ICML2016 Tutorial, Stochastic Gradient (Video) 2016
Hashing John Langford, NYU Course on Big Data, Large Scale Machine Learning - Feature Hashing (Video) 2012
random Michael Mahoney on Recent Results in Randomized Numerical Linear Algebra (NIPS 2013 Workshop on Randomized Algorithms) (Video) 2013
random Francis Bach on Beyond stochastic gradient descent for large-scale machine learning (NIPS 2013 Workshop on Randomized Algorithms) (Video) 2013
random Gautam Dasarathy: Sketching Sparse Covariance Matrices (NIPS 2013 Workshop on Randomized Algorithms) (Video) 2013

Topic VII: Reinforcement Learning Related Topics

Tag Title and Information URLs (Paper/Video/Slide) Year
✓ BasicRL Basics of Reinforcement Learning: Michael Littman, on Conference on Reinforcement Learning and Decision Making (RLDM)15 (Video) + (Slide) 2015
✓BasicRL David Silver Course on Reinforcement Learning (10 lectures) (ListVideo)+ (Slide) 2015
✓ RLFunc NIPS15 Tutorial: Introduction to Reinforcement Learning with Function Approximation (Video) + (Slide) 2015
✓ DeepRL David Silver: Deep Reinforcement Learning (Video) + (Slide) 2015
DeepRL Nando de Freitas: Deep Learning Lecture 15: Deep Reinforcement Learning - Policy search (Video) + (Slide) 2015
DeepRL Nando de Freitas: Deep Learning Lecture 16: Reinforcement learning and neuro-dynamic programming (Video) + (Slide) 2015

Topic VIII: Advanced / Recent Tutorials helpful for Research

Tag Title and Information URLs (Paper/Video/Slide) Year
MLAssorted Simon Institute Worshop: Workshop on Information Theory, Learning and Big Data talk Videos (VideoList) 2016
MLAssorted Simon Institute Big Data Boot Camp talk Videos (VideoList) 2015
MLAssorted NIPS 2016 conference talk Videos (VideoList) 2016
MLAssorted ICML16 conference talk Videos (VideoList) 2016
MLAssorted ICML15 conference talk Videos (VideoList) 2015
MLAssorted NIPS 2015 Workshop talk Videos (VideoList) 2015
MLAssorted AISTAT14 conference talk Videos (VideoList) 2014
MLAssorted KDD 2014 conference talk Videos (VideoList) 2014
CBAssorted Simon Institute Worshop: Network Biology Conference 16 talk Videos (VideoList) 2016
CBAssorted Simon Institute Worshop: Regulatory Genomics and Epigenomics talk Videos (VideoList) 2016
CBAssorted Simon Institute Worshop: Computational Cancer Biology talk Videos (VideoList) 2016
✓Other Simon Institute Worshop Talk: Obfuscation: Past, Present, and Possible Futures (Video) 2016
many more exciting video tutorials @ http://videolectures.net
0:Intro. I:Deep II:KernelSVM III:OptimHigh IV:GMBayes V:MatrixEx VI:Scalable VII:Reinforce VIII:Research
BackTop