- Week 6 – Lecture: CNN applications, RNN, and attention

Week 6 – Lecture: CNN applications, RNN, and attention

Course website: http://bit.ly/pDL-home
Playlist: http://bit.ly/pDL-YouTube
Speaker: Yann LeCun
Week 6: http://bit.ly/pDL-en-06

0:00:00 – Week 6 – Lecture

LECTURE Part A: http://bit.ly/pDL-en-06-1
We discussed three applications of convolutional neural networks. We started with digit recognition...
Course website: http://bit.ly/pDL-home
Playlist: http://bit.ly/pDL-YouTube
Speaker: Yann LeCun
Week 6: http://bit.ly/pDL-en-06

0:00:00 – Week 6 – Lecture

LECTURE Part A: http://bit.ly/pDL-en-06-1
We discussed three applications of convolutional neural networks. We started with digit recognition and the application to a 5-digit zip code recognition. In object detection, we talk about how to use multi-scale architecture in a face detection setting. Lastly, we saw how ConvNets are used in semantic segmentation tasks with concrete examples in a robotic vision system and object segmentation in an urban environment.
0:00:43 – Word-level training with minimal supervision
0:20:41 – Face Detection and Semantic Segmentation
0:27:49 – ConvNet for Long Range Adaptive Robot Vision and Scene Parsing

LECTURE Part B: http://bit.ly/pDL-en-06-2
We examine Recurrent Neural Networks, their problems, and common techniques for mitigating these issues. We then review a variety of modules developed to resolve RNN model issues including Attention, GRUs (Gated Recurrent Unit), LSTMs (Long Short-Term Memory), and Seq2Seq.
0:43:40 – Recurrent Neural Networks and Attention Mechanisms
0:59:09 – GRUs, LSTMs, and Seq2Seq Models
1:16:15 – Memory Networks

#CNN #Yann LeCun #Deep Learning #RNN #LSTM #Attention #PyTorch #NYU
– Week 6 – Lecture - Week 6 – Lecture: CNN applications, RNN, and attention

– Week 6 – Lecture

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:00:00 - 00:00:43
– Word-level training with minimal supervision - Week 6 – Lecture: CNN applications, RNN, and attention

– Word-level training with minimal supervision

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:00:43 - 00:20:41
Hi, Thanks very much for all these videos!  At around , the 2nd last layer is 5x5, and we use different widths to signify different windows. But if for instance you use a 5x4 kernel as the last layer, wouldnt you have an output thats 1x2? You will have more columns now, but it seems in the picture whether you used a 5x5 or 5x4 or 5x3 or 5x2 kernel the number of columns remain the same. Do you have to do additional steps after changing the kernel size of the last layer? - Week 6 – Lecture: CNN applications, RNN, and attention

Hi, Thanks very much for all these videos! At around , the 2nd last layer is 5x5, and we use different widths to signify different windows. But if for instance you use a 5x4 kernel as the last layer, wouldnt you have an output thats 1x2? You will have more columns now, but it seems in the picture whether you used a 5x5 or 5x4 or 5x3 or 5x2 kernel the number of columns remain the same. Do you have to do additional steps after changing the kernel size of the last layer?

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:06:14 - 01:28:48
What values are in the big array at ? Does he even say? - Week 6 – Lecture: CNN applications, RNN, and attention

What values are in the big array at ? Does he even say?

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:13:10 - 01:28:48
– Face Detection and Semantic Segmentation - Week 6 – Lecture: CNN applications, RNN, and attention

– Face Detection and Semantic Segmentation

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:20:41 - 00:27:49
– ConvNet for Long Range Adaptive Robot Vision and Scene Parsing - Week 6 – Lecture: CNN applications, RNN, and attention

– ConvNet for Long Range Adaptive Robot Vision and Scene Parsing

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:27:49 - 00:43:40
At , it does not look like all the "paths" share the same weights - except for the last layer (just before the output.) But he says they use the same kernels. I guess the illustration is maybe a bit misleading and the truth is that they do indeed share weights? - Week 6 – Lecture: CNN applications, RNN, and attention

At , it does not look like all the "paths" share the same weights - except for the last layer (just before the output.) But he says they use the same kernels. I guess the illustration is maybe a bit misleading and the truth is that they do indeed share weights?

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:39:23 - 01:28:48
– Recurrent Neural Networks and Attention Mechanisms - Week 6 – Lecture: CNN applications, RNN, and attention

– Recurrent Neural Networks and Attention Mechanisms

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:43:40 - 00:59:09
– GRUs, LSTMs, and Seq2Seq Models - Week 6 – Lecture: CNN applications, RNN, and attention

– GRUs, LSTMs, and Seq2Seq Models

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
00:59:09 - 01:16:15
I have a question. From slide @. Could it be that the bottom equation for h_t does not corresponds exactly to the green diagram? Because z_t and 1-z_t seems to be switched: - Week 6 – Lecture: CNN applications, RNN, and attention

I have a question. From slide @. Could it be that the bottom equation for h_t does not corresponds exactly to the green diagram? Because z_t and 1-z_t seems to be switched:

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
01:01:51 - 01:28:48
– Memory Networks - Week 6 – Lecture: CNN applications, RNN, and attention

– Memory Networks

Week 6 – Lecture: CNN applications, RNN, and attention
2020年04月13日
01:16:15 - 01:28:48
Alfredo Canziani

Alfredo Canziani

🎉 32,000 人達成! 🎉

【予測】4万人まであと800日(2024年12月15日)

チャンネル登録 RSS
Music, math, and deep learning from scratch

Timetable

動画タイムテーブル

動画数:76件

– Welcome to class - 10P – Non-contrastive joint embedding methods (JEMs) for self-supervised learning (SSL)

– Welcome to class

10P – Non-contrastive joint embedding methods (JEMs) for self-supervised learning (SSL)
2022年06月07日
00:00:00 - 01:05:28
– Welcome to class - 09P – Contrastive joint embedding methods (JEMs) for self-supervised learning (SSL)

– Welcome to class

09P – Contrastive joint embedding methods (JEMs) for self-supervised learning (SSL)
2022年05月28日
00:00:00 - 00:56:52
– Welcome to class - 14L – Lagrangian backpropagation, final project winners, and Q&A session

– Welcome to class

14L – Lagrangian backpropagation, final project winners, and Q&A session
2021年08月18日
00:00:00 - 02:12:36
– Welcome to class - 13L – Optimisation for Deep Learning

– Welcome to class

13L – Optimisation for Deep Learning
2021年08月18日
00:00:00 - 01:51:32
– Welcome to class - 07L – PCA, AE, K-means, Gaussian mixture model, sparse coding, and intuitive VAE

– Welcome to class

07L – PCA, AE, K-means, Gaussian mixture model, sparse coding, and intuitive VAE
2021年08月12日
00:00:00 - 01:54:23
I am awake at  as well 🤣Joke aside, 2021 videos are quite different from 2020, which is a great treat! I am being introduced to VAE from EBM's R(z). Also, thanks for sharing the homework 3 questions which help me to think and understand EMB better. Thank you Professor Yann and Professor Alfredo! 🥰 - 07L – PCA, AE, K-means, Gaussian mixture model, sparse coding, and intuitive VAE

I am awake at as well 🤣Joke aside, 2021 videos are quite different from 2020, which is a great treat! I am being introduced to VAE from EBM's R(z). Also, thanks for sharing the homework 3 questions which help me to think and understand EMB better. Thank you Professor Yann and Professor Alfredo! 🥰

07L – PCA, AE, K-means, Gaussian mixture model, sparse coding, and intuitive VAE
2021年08月12日
00:52:09 - 01:54:23
In  Yann Lecun says that the brain doesn't do reconstruction, that it doesn't reconstruct an input from an embedding. This seems very counter intuitive to me... Why not? What are dreams then? Aren't they reconstructions of input signals (images, sounds etc.) from some sort of embeddings? - 07L – PCA, AE, K-means, Gaussian mixture model, sparse coding, and intuitive VAE

In Yann Lecun says that the brain doesn't do reconstruction, that it doesn't reconstruct an input from an embedding. This seems very counter intuitive to me... Why not? What are dreams then? Aren't they reconstructions of input signals (images, sounds etc.) from some sort of embeddings?

07L – PCA, AE, K-means, Gaussian mixture model, sparse coding, and intuitive VAE
2021年08月12日
01:12:00 - 01:54:23
– Summary - 08L – Self-supervised learning and variational inference

– Summary

08L – Self-supervised learning and variational inference
2021年08月12日
00:00:00 - 00:01:00
– Welcome to class - 08L – Self-supervised learning and variational inference

– Welcome to class

08L – Self-supervised learning and variational inference
2021年08月12日
00:00:00 - 01:54:44
– GANs - 08L – Self-supervised learning and variational inference

– GANs

08L – Self-supervised learning and variational inference
2021年08月12日
00:01:00 - 00:17:10
– How do Humans and Animals learn quickly - 08L – Self-supervised learning and variational inference

– How do Humans and Animals learn quickly

08L – Self-supervised learning and variational inference
2021年08月12日
00:17:10 - 00:28:05
– Self Supervised Learning - 08L – Self-supervised learning and variational inference

– Self Supervised Learning

08L – Self-supervised learning and variational inference
2021年08月12日
00:28:05 - 00:32:00
– Sparse Coding Sparce Modeling - 08L – Self-supervised learning and variational inference

– Sparse Coding Sparce Modeling

08L – Self-supervised learning and variational inference
2021年08月12日
00:32:00 - 01:07:45
@Alfredo Canziani Hi Alf, at , Yann mentioned there are dataset that the NYU students can use for their SSL project. I was wondering if it is possible to release those to students outside of NYU so that we can try them out as well? 🤔 - 08L – Self-supervised learning and variational inference

@Alfredo Canziani Hi Alf, at , Yann mentioned there are dataset that the NYU students can use for their SSL project. I was wondering if it is possible to release those to students outside of NYU so that we can try them out as well? 🤔

08L – Self-supervised learning and variational inference
2021年08月12日
00:57:27 - 01:54:44
If this way of making features ( , 1:12:06) is so cool and more "natural" (kinda same as a brain works with visual features), why the research wasn't turned in that direction starting from 2010 when it was proposed? 🤔  I suggest there are some limitations Yann didn't mention? Or the reason is that the topic is still kinda more complex than the usual convolutions?Thanks for the vid, Alfredo and Yann 🤗 - 08L – Self-supervised learning and variational inference

If this way of making features ( , 1:12:06) is so cool and more "natural" (kinda same as a brain works with visual features), why the research wasn't turned in that direction starting from 2010 when it was proposed? 🤔 I suggest there are some limitations Yann didn't mention? Or the reason is that the topic is still kinda more complex than the usual convolutions?Thanks for the vid, Alfredo and Yann 🤗

08L – Self-supervised learning and variational inference
2021年08月12日
00:58:55 - 01:54:44
– Regularization Through Temporal Consistency - 08L – Self-supervised learning and variational inference

– Regularization Through Temporal Consistency

08L – Self-supervised learning and variational inference
2021年08月12日
01:07:45 - 01:12:05
at  , how do you know which parts of z to allow to vary, and which to not, exactly? How do you know which parts represent the "objects", and which parts represents the things that are changing, like the location of the objects? - 08L – Self-supervised learning and variational inference

at , how do you know which parts of z to allow to vary, and which to not, exactly? How do you know which parts represent the "objects", and which parts represents the things that are changing, like the location of the objects?

08L – Self-supervised learning and variational inference
2021年08月12日
01:11:40 - 01:54:44
– Variational AE - 08L – Self-supervised learning and variational inference

– Variational AE

08L – Self-supervised learning and variational inference
2021年08月12日
01:12:05 - 01:54:44
– Welcome to class - 09L – Differentiable associative memories, attention, and transformers

– Welcome to class

09L – Differentiable associative memories, attention, and transformers
2021年08月12日
00:00:00 - 02:00:29
Yann gets sad at  while he is talking about attention mecanishm might take the place of convolution at images :/ - 09L – Differentiable associative memories, attention, and transformers

Yann gets sad at while he is talking about attention mecanishm might take the place of convolution at images :/

09L – Differentiable associative memories, attention, and transformers
2021年08月12日
01:26:09 - 02:00:29
For masking, is there a strategy to remove words instead of random masking, as if the object of interest, eg: curtain @ were to be removed from both English and French, wouldn't it make the prediction task much more difficult, as a lot of objects could be substituted in its place. - 09L – Differentiable associative memories, attention, and transformers

For masking, is there a strategy to remove words instead of random masking, as if the object of interest, eg: curtain @ were to be removed from both English and French, wouldn't it make the prediction task much more difficult, as a lot of objects could be substituted in its place.

09L – Differentiable associative memories, attention, and transformers
2021年08月12日
01:29:19 - 02:00:29
– Welcome to class - 14 – Prediction and Planning Under Uncertainty

– Welcome to class

14 – Prediction and Planning Under Uncertainty
2021年08月03日
00:00:00 - 01:14:45
– Welcome to class - 06L – Latent variable EBMs for structured prediction

– Welcome to class

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:00:00 - 00:00:17
– Training of an EBM - 06L – Latent variable EBMs for structured prediction

– Training of an EBM

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:00:17 - 00:04:27
– Contrastive vs. regularised / architectural methods - 06L – Latent variable EBMs for structured prediction

– Contrastive vs. regularised / architectural methods

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:04:27 - 00:05:21
– General margin loss - 06L – Latent variable EBMs for structured prediction

– General margin loss

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:05:21 - 00:09:34
– List of loss functions - 06L – Latent variable EBMs for structured prediction

– List of loss functions

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:09:34 - 00:13:45
– Generalised additive margin loss - 06L – Latent variable EBMs for structured prediction

– Generalised additive margin loss

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:13:45 - 00:17:53
– Joint embedding architectures - 06L – Latent variable EBMs for structured prediction

– Joint embedding architectures

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:17:53 - 00:21:29
– Wav2Vec 2.0 - 06L – Latent variable EBMs for structured prediction

– Wav2Vec 2.0

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:21:29 - 00:27:14
– XLSR: multilingual speech recognition - 06L – Latent variable EBMs for structured prediction

– XLSR: multilingual speech recognition

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:27:14 - 00:29:16
– Generative adversarial networks (GANs) - 06L – Latent variable EBMs for structured prediction

– Generative adversarial networks (GANs)

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:29:16 - 00:37:24
– Mode collapse - 06L – Latent variable EBMs for structured prediction

– Mode collapse

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:37:24 - 00:41:45
– Non-contrastive methods - 06L – Latent variable EBMs for structured prediction

– Non-contrastive methods

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:41:45 - 00:43:19
– BYOL: bootstrap your own latent - 06L – Latent variable EBMs for structured prediction

– BYOL: bootstrap your own latent

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:43:19 - 00:44:27
What does "averaging the weights over time" mean exactly? at - 06L – Latent variable EBMs for structured prediction

What does "averaging the weights over time" mean exactly? at

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:43:40 - 01:48:54
– SwAV - 06L – Latent variable EBMs for structured prediction

– SwAV

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:44:27 - 00:46:45
– Barlow twins - 06L – Latent variable EBMs for structured prediction

– Barlow twins

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:46:45 - 00:51:29
– SEER - 06L – Latent variable EBMs for structured prediction

– SEER

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:51:29 - 00:54:29
– Latent variable models in practice - 06L – Latent variable EBMs for structured prediction

– Latent variable models in practice

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:54:29 - 00:57:34
,,,, hi from Turkey :) - 06L – Latent variable EBMs for structured prediction

,,,, hi from Turkey :)

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:56:35 - 01:48:54
– DETR - 06L – Latent variable EBMs for structured prediction

– DETR

06L – Latent variable EBMs for structured prediction
2021年07月28日
00:57:34 - 01:01:21
– Structured prediction - 06L – Latent variable EBMs for structured prediction

– Structured prediction

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:01:21 - 01:04:53
– Factor graph - 06L – Latent variable EBMs for structured prediction

– Factor graph

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:04:53 - 01:12:47
Just a small doubt at  (at end of factor graph), when Yann mentioned that the algo is dp and it is in linear time.But the way he explained the algo, it was more like Dijkstras greedy search, which is O(V log E). As far as I remember, Dp based shortest path that work on network exhaust ively, have O(VE) time complexity, like bellman-ford. Please do correct me if I am wrong. I know this isn't of much concern here, but it bugged me a bit, thus wanted to clarify. Thank you. - 06L – Latent variable EBMs for structured prediction

Just a small doubt at (at end of factor graph), when Yann mentioned that the algo is dp and it is in linear time.But the way he explained the algo, it was more like Dijkstras greedy search, which is O(V log E). As far as I remember, Dp based shortest path that work on network exhaust ively, have O(VE) time complexity, like bellman-ford. Please do correct me if I am wrong. I know this isn't of much concern here, but it bugged me a bit, thus wanted to clarify. Thank you.

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:11:30 - 01:48:54
– Viterbi algorithm whiteboard time - 06L – Latent variable EBMs for structured prediction

– Viterbi algorithm whiteboard time

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:12:47 - 01:30:24
– Graph transformer networks - 06L – Latent variable EBMs for structured prediction

– Graph transformer networks

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:30:24 - 01:46:48
– Graph composition, transducers - 06L – Latent variable EBMs for structured prediction

– Graph composition, transducers

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:46:48 - 01:48:38
– Final remarks - 06L – Latent variable EBMs for structured prediction

– Final remarks

06L – Latent variable EBMs for structured prediction
2021年07月28日
01:48:38 - 01:48:54
– Welcome to class - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Welcome to class

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:00:00 - 00:00:39
– Predictive models - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Predictive models

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:00:39 - 00:02:25
What are the research papers from Facebook mentioned around ? - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

What are the research papers from Facebook mentioned around ?

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:01:30 - 01:51:31
– Multi-output system - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Multi-output system

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:02:25 - 00:06:36
– Notation (factor graph) - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Notation (factor graph)

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:06:36 - 00:07:41
– The energy function F(x, y) - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– The energy function F(x, y)

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:07:41 - 00:08:53
– Inference - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Inference

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:08:53 - 00:11:59
– Implicit function - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Implicit function

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:11:59 - 00:15:53
– Conditional EBM - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Conditional EBM

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:15:53 - 00:16:24
– Unconditional EBM - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Unconditional EBM

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:16:24 - 00:19:18
and 50:43 => Unconditional model is when the input is partially observed but you dont know exactly what part.- What is test/inference in these unconditional EBM models? Is there a proper split between training and inference/test in the unconditional models?- How does models like PCA or K-means fit here, what are the partially observed inputs Y? For example in K-MEans you receive all the components of Y, I dont see that they are partially observed - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

and 50:43 => Unconditional model is when the input is partially observed but you dont know exactly what part.- What is test/inference in these unconditional EBM models? Is there a proper split between training and inference/test in the unconditional models?- How does models like PCA or K-means fit here, what are the partially observed inputs Y? For example in K-MEans you receive all the components of Y, I dont see that they are partially observed

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:16:34 - 00:25:10
– EBM vs. probabilistic models - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– EBM vs. probabilistic models

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:19:18 - 00:21:33
– Do we need a y at inference? - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Do we need a y at inference?

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:21:33 - 00:23:29
– When inference is hard - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– When inference is hard

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:23:29 - 00:25:02
– Joint embeddings - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Joint embeddings

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:25:02 - 00:28:29
and 1:01:50 => With the joint embedding architecture- What would be inference with this architecture, inferring a Y from a given X minimizing the cost C(h, h')? I know that you could run gradient descent to the Y backward the Pred(y) network but it is not clear to me the purpose of inferring Y given X in this architecure.- What does the "Advange: no pixel-level reconstruction" in green means? (I suspect that this may have something to do with my just above question)- Can this architecture also be trained as a Latent Variable EBM? or it is always trained in a Contrastive way - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

and 1:01:50 => With the joint embedding architecture- What would be inference with this architecture, inferring a Y from a given X minimizing the cost C(h, h')? I know that you could run gradient descent to the Y backward the Pred(y) network but it is not clear to me the purpose of inferring Y given X in this architecure.- What does the "Advange: no pixel-level reconstruction" in green means? (I suspect that this may have something to do with my just above question)- Can this architecture also be trained as a Latent Variable EBM? or it is always trained in a Contrastive way

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:25:10 - 01:51:31
– Latent variables - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Latent variables

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:28:29 - 00:33:54
– Inference with latent variables - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Inference with latent variables

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:33:54 - 00:37:58
– Energies E and F - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Energies E and F

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:37:58 - 00:42:35
– Preview on the EBM practicum - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Preview on the EBM practicum

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:42:35 - 00:44:30
– From energy to probabilities - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– From energy to probabilities

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:44:30 - 00:50:37
– Examples: K-means and sparse coding - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Examples: K-means and sparse coding

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:50:37 - 00:53:56
– Limiting the information capacity of the latent variable - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Limiting the information capacity of the latent variable

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:53:56 - 00:57:24
– Training EBMs - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Training EBMs

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
00:57:24 - 01:04:02
– Maximum likelihood - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Maximum likelihood

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:04:02 - 01:13:58
I spent some time to derive the step mention in . I made my best effort to get the final result. But, I am not sure if my steps are correct. I hope my fellow students can help to point out my mistakes. Due to the lack of LaTex support in Youtube comment, I try my best to make my steps as clear as possible. I use partial derivative for log to get to the second step. Then, I use Leibniz integral rule to move the partial derivative inside the integral in the third step. The rest is pretty straightforward, hopefully. Thank you! - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

I spent some time to derive the step mention in . I made my best effort to get the final result. But, I am not sure if my steps are correct. I hope my fellow students can help to point out my mistakes. Due to the lack of LaTex support in Youtube comment, I try my best to make my steps as clear as possible. I use partial derivative for log to get to the second step. Then, I use Leibniz integral rule to move the partial derivative inside the integral in the third step. The rest is pretty straightforward, hopefully. Thank you!

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:07:44 - 01:51:31
I tried to calculate the derivative Yann said (), but probably I am missing something because in my final result I don't have the integral (only -P_w(.) ...). Is there any supplementary material with these calculations?Thanks again for your amazing and hard work! - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

I tried to calculate the derivative Yann said (), but probably I am missing something because in my final result I don't have the integral (only -P_w(.) ...). Is there any supplementary material with these calculations?Thanks again for your amazing and hard work!

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:07:45 - 01:51:31
– How to pick β? - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– How to pick β?

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:13:58 - 01:17:28
– Problems with maximum likelihood - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Problems with maximum likelihood

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:17:28 - 01:20:20
– Other types of loss functions - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Other types of loss functions

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:20:20 - 01:26:32
– Generalised margin loss - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Generalised margin loss

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:26:32 - 01:27:22
– General group loss - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– General group loss

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:27:22 - 01:28:26
– Contrastive joint embeddings - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Contrastive joint embeddings

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:28:26 - 01:34:51
Am I missing something, or in this lecture there is no "non-contrastive joint embeddings" methods Yann was talking about at   ? I also briefly checked the next lectures but didn't find something related to this. Could you please point me out? 😇Thank you for the video, btw, brilliant as always :) - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

Am I missing something, or in this lecture there is no "non-contrastive joint embeddings" methods Yann was talking about at ? I also briefly checked the next lectures but didn't find something related to this. Could you please point me out? 😇Thank you for the video, btw, brilliant as always :)

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:34:40 - 01:51:31
– Denoising or mask autoencoder - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Denoising or mask autoencoder

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:34:51 - 01:46:14
– Summary and final remarks - 05L – Joint embedding method and latent variable energy based models (LV-EBMs)

– Summary and final remarks

05L – Joint embedding method and latent variable energy based models (LV-EBMs)
2021年07月28日
01:46:14 - 01:51:31