hackertube.io

Let's build GPT: from scratch, in code, spelled out.

Andrej Karpathy · tutorial · Watch on YouTube

LLMs & generative AIMachine learning & deep learning

What it is

Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.

Why Hacker News recommends it

It is the go-to recommendation for learning how transformers and LLMs actually work, praised for its clarity and Karpathy's teaching.

  • 14 people recommended it
  • 8 different months
  • 4 strong recommendations
  • first 2023, last 2025

What HN commenters said

  1. “Andrej Karpathy's GPT video is a must have companion for this”

    Clarifies key, query and value in attention.

    yonz, 2023-01 · Strong recommendation · in “The Transformer Family”

  2. “I was blown away”

    Amazing lecture on building GPT from scratch.

    durdn, 2023-02 · Strong recommendation · in “GPT in 60 Lines of NumPy”

  3. “the best video I have seen on the topic”

    Intuitive, easy-to-follow explanation of how models are built.

    wesammikhail, 2023-05 · Strong recommendation · in “Google DeepMind CEO Says Some Form of AGI Possible in a Few Years”

  4. “Andrey is an amazing educator!”

    High-quality teaching by Karpathy.

    villedespommes, 2024-01 · Strong recommendation · in “Implementing a ChatGPT-like LLM from scratch, step by step”

  5. “I recommend this video on the subject.”

    Build GPT from scratch in code.

    amrb, 2023-01 · Recommendation · in “Deep Learning Tuning Playbook”

  6. “I would highly recommend watching and working though the code (if possible) of NanoGPT by Andrej Karpathy.”

    Demystifies how LLMs work via NanoGPT code.

    qazpot, 2023-03 · Recommendation · in “Flow-Based Programming, a way for AI and humans to develop together”

  7. “There is also the nanoGPT series of videos by Karpathy on youtube.”

    NanoGPT video series for learning LLM fundamentals.

    bfg_damien, 2023-04 · Recommendation · in “Ask HN: How to Learn LLM Fundamentals”

  8. “Karpathy's YouTube video is an excellent resource for understanding Transformers from scratch.”

    Hands-on implementation of transformers from scratch.

    pankajdoharey, 2023-04 · Recommendation · in “What Are Transformer Models and How Do They Work?”

Excerpts from Hacker News comments, linked to the originals. Summaries and labels were written by a language model from those comments.

Topics: gpt, transformer, pytorch, nanogpt, attention, from scratch

Also recommended by the same people