Understanding SVM (and associated kernel machines)
through the development of a Matlab toolbox
Stéphane Canu
Lectures given at the Institute of Mathematics and Statistics, University
of Sao Paulo, feb 2014
Moodle site of
the lecture at IME-USP thanks to Pr. R. Hirata Jr
Sylabus (pdf)
The Objective of this course is to provide an overview of all these issues related with kernels machines.
To do so, we will introduce kernel machines and associated mathematical foundations through practical implementation.
All lectures will be devoted to the writing of some Matlab functions that, putting all together, will provide a toolbox for learning with kernels.
Lecture 1: Two classes classification problem: Introduction to Support Vector Machines (SVM), Linear SVM in the primal as a quadratic program and LPSVM for variable selection.
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Data (.mat)
Lecture 2: Karush-Kuhn-Tucker (KKT) optimality conditions, lagrangian duality, Linear SVM in the dual, support vectors
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Lecture 3: Linear SVM in the non separable case, reformulations: convex hulls and regularized hinge loss minimization. Libsvm and other QP solver
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Data (.mat)
Lecture 4: Positive definite kernels, associated RKHS, differentiability, subdifferentiablility and other kernels
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Lecture 5: SVM in a reproducing kernel Hilbert spaces (RKHS), representer theorem and other kernel machines (splines and kernel logistic regression)
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Checker
data (.mat)
Lecture 6: Novelty detection, Support Vector Data Description (SVDD), One Class SVM, minimum enclosing balls and level sets
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file,
An
other solution for 2 class SVDD as a matlab file
Data
(.mat)
Lecture 7: Tuning the hyperparamters, C, the bandwidth and the KKT gap
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Splice
data
(.mat)
Lecture 8: MultiClass SVM, estimating the posterior probabilities and structured output
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Data
(.mat)
Lecture 9: How to choose the kernel, multiple kernel learning
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Data
(.mat)
Lectrure 10: L0 SVDD,
Slides (pdf)
Practical Session (pdf)
My
solution as a matlab file
Matlab toolbox and other usefull software resources:
A lecture
on SVM in french
Dernière mise à jour : 21 Mai 2014,
Stéphane Canu (
scanu@insa-rouen.fr)