– Supervised learning
@Alfredo Canziani Oh right! sorry. It's at
– Parametrised models
– Block diagram
– Loss function, average loss
Discussion on stochastic gradient descent () and with adams (1:16:15) are great. General misconception.
– Gradient descent
Doesn't he contradict himself? First he mentions that smaller batches are better (I assume that by "better" he meant model quality) in most cases, and a few seconds later he says that it's just a hardware matter.
Hi Alfredo, in Yann mentioned "objective function need to be Continuous mostly and differentiable almost everywhere". What does he mean? isn't the function differentiable is always continuous? also is there a function where some part only differentiable? Can someone give me one example in deep learning functions? pls help me out.
My question was from
– Traditional neural nets
@Alfredo Canziani ah, sorry. Was just to add on, at ~ when Prof. LeCun explains why people don't like to refer to the units as 'neurons' persay
– Backprop through a non-linear function
At the section - is the purpose of using back propagation to find the derivative of the cost function wrt z to find the best direction to "move"? I've only gotten through half of the lecture so forgive me if this is answered later
– Backprop through a weighted sum
– PyTorch implementation
About the code in PyTorch... ( in the video)... the code instantiates the mynet class and stores the reference in model variable... but nowhere it calls the "forward" method... so how does the out variable receive any output from the model object? Is there some Pytorch magic which is not explained here ?
shouldn't it be self.m0(z0) as it takes in the flattened input?
@Alfredo Canziani Actually I asked that before I watched it at , Regards 🤞
Just FYI, at Yann correctly says dc/dzg, but the diagram has dc/zg. Should that also be dc/dwg and dc/dwf?
– Backprop through a functional module
At Yann is explaining the two jacobians, but I was having trouble getting the intuition. Then I realized that the first jacobian was getting the gradient to modify the weights w[k+1] for function z[k+1] and the second jacobian was back propagating the gradient to function z[k] which can then be used to calculate the gradient at k for yet another jacobian to adjust weights w[k]. So one jacobian is for the parameters and the other is for the state since both the parameter variable and state variable are column vectors. Yann explains it really well. I'm amazed that I seem to be understanding this complicated mix of symbols and logic. Thank you.
– Backprop in practice
Does the trick explained in normalizing training samples () applies also to convolutional neural networks?
It's just after .
– Learning representations
– Shallow networks are universal approximators!
– Multilayer architectures == compositional structure of data
I thought that haar-like features were not that recognizable. ()
