Stanford CS25: V1 I Self Attention and Non-parametric transformers (NPTs)
Aidan will chat about the Transformer origins and intuitions for 15 minutes. Then Neil and Jannik will take over to talk about Non-Parametric Transformers (NPTs). NPTs have just been accepted to NeurIPs and you can check them out at https://arxiv.org/abs/2106.02584.
Aidan is a PhD student at Oxf...
Aidan will chat about the Transformer origins and intuitions for 15 minutes. Then Neil and Jannik will take over to talk about Non-Parametric Transformers (NPTs). NPTs have just been accepted to NeurIPs and you can check them out at https://arxiv.org/abs/2106.02584.
Aidan is a PhD student at Oxford supervised by Yarin Gal, and one of the cofounders of Cohere. He is fascinated by building massive neural networks and getting them into the hands of more engineers and researchers.
Jannik is a PhD student at the University of Oxford. He is supervised by Yarin Gal and Tom Rainforth. His research interests include Bayesian Deep Learning, Active Learning, and, apparently, building non-parametric models with Transformers.
Neil is a Masters by Research student at the University of Oxford, supervised by Yarin Gal. He is interested in models with relational inductive biases such as Transformers and graph neural networks, as well as Bayesian deep learning.
