16 – Autoregressive Gen. (11/042018)

In these lectures, we discuss autoregressive generative models such as NADE, MADE, PixelCNN, PixelRNN, and the PixelVAE.

Slides:

Reference: (* = you are responsible for this material)

15 – GANs (04/04/2018)

In this lecture, we will discuss Generative Adversarial Networks (GANs). GANs are a recent and very popular generative model paradigm. We will discuss the GAN formalism, some theory and practical considerations.

Slides:

Reference: (* = you are responsible for this material)

14 – RBMs & DBMs (25/03/2018)

In this lecture, we will discuss undirected generative models. Specifically we will look at the Restricted Boltzmann Machine and (to the extent that time permits) the Deep Boltzmann Machine.

Slides:

Reference: (* = you are responsible for this material)

  • *Sections 20.1 to 20.4.4 (inclusively) of the Deep Learning textbook.
  • *Sections 17.3-17.4 (MCMC, Gibbs), chap. 19 (Approximate Inference) of the Deep Learning textbook.

13 – Variational Autoencoders (21/03/2018)

In this lecture, Chin-Wei will talk about a form of autoencoder known as the Variational Autoencoder (VAE). We’ll see how a deep latent gaussian model can be seen as an autoencoder via Amortized variational inference, and how such an autoencoder can be used as a generative model. At the end, we’ll take a look at variants of VAE and different ways to improve inference.

 

Slides:

Reference: (* = you are responsible for this material)

11 – Memory (12/03/2018)

In this lecture, we will discuss recent models that incorporate a distinct memory module into neural networks.

Slides:

Reference: (* = you are responsible for this material)

10 – Attention (28/02/2018)

In this lecture, Dzmitry (Dima) Bahdanau will discuss attention in neural networks.

Slides:

Reference: (* = you are responsible for this material)

07 – Regularization (12/02/2018)

In these lectures, we will have a rather detailed discussion of regularization methods and their interpretation.

Slides:

Reference: (* = you are responsible for this material)