Stanford CS25: V2 I Represent part-whole hierarchies in a neural network, Geoff Hinton

"I will present a single idea about representation which allows advances made by several different groups to be combined into an imaginary system called GLOM. The advances include transformers, neural fields, contrastive representation learning, distillation and capsules. GLOM answers the question: How can a neural network with a fixed architecture parse an image into a part-whole hierarchy which has a different structure for each image? The idea is simply to use islands of identical vectors to represent the nodes in the parse tree. The talk will discuss the many ramifications of this idea.  If GLOM can be made to work, it should significantly improve the interpretability of the representations produced by transformer-like systems when applied to vision or language."

Geoff Hinton is an emeritus professor at the University of Toronto, a VP Engineering fellow at Google and a Chief Scientific Adviser at the Vector Institute. Hinton is a leading figure in the deep learning community, was one of the pioneering researchers who introduced the backpropagation algorithm, and, together with his research group, made significant breakthroughs and contributions to advance the field of neural networks. He is a fellow of the UK Royal Society, and has received numerous awards to acknowledge his work, including the David E. Rumelhart prize, the IJCAI award for research excellence, the Killam prize for Engineering, the IEEE Frank Rosenblatt medal, the NSERC Herzberg Gold Medal, the IEEE James Clerk Maxwell Gold medal, the NEC C&C award, the BBVA award, the Honda Prize and the Turing Award.

View the entire CS25 Transformers United playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rNiJRchCzutFw5ItR_Z27CM

#Stanford #Stanford Online #Stanford Seminar #Neural Network #AI