Wiki / entities / wiki

Andrej Karpathy

Former OpenAI founding member and Tesla AI director, now an educator; coined "vibe coding" (vibe-coding). Karpathy's "Deep Dive into LLMs like ChatGPT" (Feb 2025, 9.7M views, 3.5h) is a general-audience explainer built on three training stages (pretraining โ†’ SFT โ†’ RL) mapped to how children learn from textbooks (exposition โ†’ worked examples โ†’ practice problems), delivered as a single-take Excalidraw + live-browser screencast with zero hook engineering.

Signature mental models from the lecture: base model as lossy-compression-of-the-internet; assistant as labeler-simulation ("you're not talking to a magical AI, you're talking to an average labeler"); params-vs-context-memory; models-need-tokens-to-think; "RLHF is not RL" (gameable reward models vs indefinite verifiable-domain RL); Swiss-cheese capabilities. Reproduced GPT-2 for $600/24h (lm.c) vs the original $40k.

Karpathy's closing advice โ€” "use them as tools in the toolbox, check their work, and own the product of your work" โ€” is the same conclusion matt-pocock and dhh reach from the practitioner side: the human owns taste and verification.

Practical sequel, How I use LLMs (Feb 2025, 2.7M views, 2h11m): a tour of ChatGPT, Claude, Gemini, Grok and Perplexity organised by what goes into the context window: tool-selection-by-freshness, thinking models only when the fast answer fails, and verify-like-a-junior-analyst. Personal stack at the time: ChatGPT Pro ($200/mo) as default, Perplexity for search, Claude Artifacts for diagrams, Cursor Composer for code, SuperWhisper on F5 for ~half of all queries, and few-shot custom GPTs for learning Korean. Karpathy reads books alongside an LLM ("don't read books alone").

Source: report ยท report

Linked from

AlphaGoLLM training pipelineLossy compression of the internetModel parameters (weights)Models need tokens to thinkMove 37Open weightsParameters vs context memoryRLHF is not RLSleep-time computeSmart zone / dumb zoneSuperhuman blind spotsTool selection by freshnessVerify like a junior analystVibe coding