- 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

Course website: http://bit.ly/DLSP21-web
Playlist: http://bit.ly/DLSP21-YouTube
Speaker: Alfredo Canziani

Chapters
00:00 – 2021 edition disclaimer
00:49 – Conditional and unconditional LV EBM
02:08 – Variables' name: x, y, z, h, ỹ
03:34 – LV EBM training recap, warm case
10:54 – LV EBM training...
Course website: http://bit.ly/DLSP21-web
Playlist: http://bit.ly/DLSP21-YouTube
Speaker: Alfredo Canziani

Chapters
00:00 – 2021 edition disclaimer
00:49 – Conditional and unconditional LV EBM
02:08 – Variables' name: x, y, z, h, ỹ
03:34 – LV EBM training recap, warm case
10:54 – LV EBM training recap, zero-temperature limit
11:30 – Today's plan: the missing step
12:08 – Target prop(agation)
19:01 – From target prop to autoencoder
20:54 – Reconstruction costs
21:06 – Loss functional
21:22 – Under and over complete hidden layer
24:40 – Denoising autoencoder
32:00 – Contractive autoencoder
37:50 – Autoencoders recap
38:38 – From autoencoder to variational autoencoder
45:17 – Comparison between variational autoencoder and denoising autoencoder
45:54 – How a variational autoencoder actually works
48:29 – The bubble-of-bubble variational autoencoder interpretation
1:00:08 – And that was it :)

#PyTorch #NYU #Yann LeCun #Deep Learning #neural networks
– 2021 edition disclaimer - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– 2021 edition disclaimer

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:00:00 - 00:00:49
– Conditional and unconditional LV EBM - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Conditional and unconditional LV EBM

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:00:49 - 00:02:08
– Variables' name: x, y, z, h, ỹ - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Variables' name: x, y, z, h, ỹ

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:02:08 - 00:03:34
– LV EBM training recap, warm case - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– LV EBM training recap, warm case

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:03:34 - 00:10:54
I have a question. At  you said that we cannot use L2 norm for regularizing Z space, because the Dec(z, h) simply increase the weights. What if we normalize the weights of each layer of the decoder? It was also my question in previous lectures that why do there exist a lot of work on L1 norm and not L2 norm? - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

I have a question. At you said that we cannot use L2 norm for regularizing Z space, because the Dec(z, h) simply increase the weights. What if we normalize the weights of each layer of the decoder? It was also my question in previous lectures that why do there exist a lot of work on L1 norm and not L2 norm?

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:07:50 - 01:00:35
– LV EBM training recap, zero-temperature limit - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– LV EBM training recap, zero-temperature limit

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:10:54 - 00:11:30
– Today's plan: the missing step - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Today's plan: the missing step

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:11:30 - 00:12:08
– Target prop(agation) - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Target prop(agation)

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:12:08 - 00:19:01
– From target prop to autoencoder - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– From target prop to autoencoder

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:19:01 - 00:20:54
– Reconstruction costs - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Reconstruction costs

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:20:54 - 00:21:06
– Loss functional - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Loss functional

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:21:06 - 00:21:22
– Under and over complete hidden layer - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Under and over complete hidden layer

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:21:22 - 00:24:40
– Denoising autoencoder - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Denoising autoencoder

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:24:40 - 00:32:00
– Contractive autoencoder - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Contractive autoencoder

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:32:00 - 00:37:50
– Autoencoders recap - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Autoencoders recap

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:37:50 - 00:38:38
– From autoencoder to variational autoencoder - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– From autoencoder to variational autoencoder

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:38:38 - 00:45:17
– Comparison between variational autoencoder and denoising autoencoder - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– Comparison between variational autoencoder and denoising autoencoder

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:45:17 - 00:45:54
– How a variational autoencoder actually works - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– How a variational autoencoder actually works

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:45:54 - 00:48:29
– The bubble-of-bubble variational autoencoder interpretation - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– The bubble-of-bubble variational autoencoder interpretation

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:48:29 - 01:00:08
This is a great explanation of what happens in VAE especially the impact of KL loss. I have a small doubt about that. In  for calculating the KL distance, you are using the distribution of z as the true one. Should it be the other way by taking the normal distribution with zero mean and unit variance as true one? Will it make difference in the results? - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

This is a great explanation of what happens in VAE especially the impact of KL loss. I have a small doubt about that. In for calculating the KL distance, you are using the distribution of z as the true one. Should it be the other way by taking the normal distribution with zero mean and unit variance as true one? Will it make difference in the results?

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:49:39 - 01:00:35
At , I am not too clear on why we should constraint the bubbles from moving far away from each other. - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

At , I am not too clear on why we should constraint the bubbles from moving far away from each other.

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:52:15 - 01:00:35
Hi Professor, Thanks for sharing the video recording. I learnt a lot. I got some conceptual questions about the bubbles around .1. Is the outer purple hypersphere a d-dimensional unit hypersphere(and that is the reason why we can use standard normal to generate sample after training?)? and, the yellow inner bubbles have to live inside it?2. Will yellow inner bubbles also be unit hypersphere, but less than d-dimensional so that to make space for other yellow bubbles? But it seems to me to make KL[N(u, v) || N(0, I)] smallest, every yellow bubble should be the same as outer purple hypersphere(which is an N(0, I)). - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

Hi Professor, Thanks for sharing the video recording. I learnt a lot. I got some conceptual questions about the bubbles around .1. Is the outer purple hypersphere a d-dimensional unit hypersphere(and that is the reason why we can use standard normal to generate sample after training?)? and, the yellow inner bubbles have to live inside it?2. Will yellow inner bubbles also be unit hypersphere, but less than d-dimensional so that to make space for other yellow bubbles? But it seems to me to make KL[N(u, v) || N(0, I)] smallest, every yellow bubble should be the same as outer purple hypersphere(which is an N(0, I)).

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
00:58:36 - 01:00:35
– And that was it :) - 08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder

– And that was it :)

08 – From LV-EBM to target prop to (vanilla, denoising, contractive, variational) autoencoder
2021年05月11日 
01:00:08 - 01:00:35

Alfredo Canziani

※本サイトに掲載されているチャンネル情報や動画情報はYouTube公式のAPIを使って取得・表示しています。

Timetable

動画タイムテーブル