hackertube.io

CS480/680 Lecture 19: Attention and Transformer Networks

Pascal Poupart · lecture · Watch on YouTube

LLMs & generative AIMachine learning & deep learningUniversity lectures & courses

What it is

A university lecture from CS480/680 by Pascal Poupart on attention mechanisms and transformer networks, worked through on a chalkboard.

Why Hacker News recommends it

HN recommends it to newcomers as a clear explanation of attention, including the query/key/value intuition.

  • 3 people recommended it
  • 3 different months
  • 0 strong recommendations
  • first 2023, last 2023

What HN commenters said

  1. “I would also recommend Pascal Poupart's talk [1] on Attention and Transformers.”

    Chalkboard lecture working through attention and transformer details clearly.

    abhgh, 2023-04 · Recommendation · in “Transformers from Scratch”

  2. “These are very excellent resources”

    Excellent resource on attention and transformers for newcomers.

    elankart, 2023-05 · Recommendation · in “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”

  3. “Another good video on attention that tries to give a bit more intuition on the query/key/value stuff”

    Gives more intuition on query/key/value in attention.

    metanonsense, 2023-06 · Recommendation · in “Attention is Turing complete [pdf]”

Excerpts from Hacker News comments, linked to the originals. Summaries and labels were written by a language model from those comments.

Topics: attention, transformers, query key value, deep learning, nlp

Also recommended by the same people