LECTURE Part A: http://bit.ly/DLSP20-08-1
In this section, we focused on the introduction of contrastive methods in Energy-Based Models in several aspects. First, we discuss the advantage brought by applying contrastive methods in self-supervised learning. Second, we discussed the architecture of denoising autoencoders and their weakness in image reconstruction tasks. We also talked about other contrastive methods, like contrastive divergence and persistent contrastive divergence.
0:00:05 – Recap on EBM and Characteristics of Different Contrastive Methods
0:10:13 – Contrastive Methods in Self-Supervised Learning
0:23:04 – Denoising Autoencoder and other Contrastive methods
LECTURE Part B: http://bit.ly/DLSP20-08-2
In this section, we discussed regularized latent variable EBMs in detail covering concepts of conditional and unconditional versions of these models. We then discussed the algorithms of ISTA, FISTA and LISTA and look at examples of sparse coding and filters learned from convolutional sparse encoders. Finally we talked about Variational Auto-Encoders and the underlying concepts involved.
0:37:13 – Overview of Regularized Latent Variable Energy Based Models and Sparse Coding
1:07:46 – Convolutional Sparse Auto-Encoders
1:12:51 – Variational Auto-Encoders