Open weights
A model is open-weight when its creator publishes the trained parameters, so anyone can download it, host it, and build on it. With a closed model like ChatGPT or Claude you only get the vendor's app or API.
andrej-karpathy breaks a release into two parts [43:04]. One is the code for the network's forward pass, "a couple hundred lines" and "fairly standard". The other is the parameters, "where the actual value is": for GPT-2 (2019), one list of about 1.5 billion numbers. See model-parameters.
- GPT-2 (OpenAI, 2019) was a base model released with its weights.
- Llama 3.1 405B (Meta) is Karpathy's hands-on base model; Karpathy notes that companies rarely publish base models, because a base model is "only a step one" before post-training (llm-training-pipeline).
- DeepSeek-R1 [2:35:50, 3:16:06] is "an MIT license model... anyone can download them, anyone can host their own version." Karpathy calls it "unprecedented that a model this strong was released with open weights."
"Open weights" is narrower than open source. The training data and training code usually stay private, and licenses vary: R1 is MIT, while Llama ships under Meta's own license.
Running one takes hardware. Full-precision R1 "you won't run on a MacBook", so people use inference providers that host the weights: together.ai for chat models, Hyperbolic for base models, and Perplexity hosts R1 in its model picker. Smaller models run locally in LM Studio.
What local hardware a big open-weight model needs, per Kai's Mac mini breakdown (Sept 2026): gpt-oss-120b needs about 66 GB in its native 4-bit format, and owners of 128 GB machines see 64 to 65 GB resident, so a 64 GB Mac mini is about 1 GB short before macOS and before any context. A sparse 35B model at about 17 GB fits in 24 GB (mixture-of-experts).
A side effect shows up in dhh's interview: a Chinese open-weight model answered a Tiananmen question bluntly, while a US frontier model refused to translate an essay. Each model's refusals come from whoever post-trained it.
Cost of one open-weight release, per yann-dubois's back-of-envelope for Llama 3 405B, which Dubois called the best open model in mid-2024: about 26M H100-hours (Meta reported 30M), roughly $52M of rented compute plus $25M in salaries.
dhh's timeline from the Rails World 2026 keynote: open-weight models caught up within months of Opus 4.5 (DHH used Kimi K2.5 in fast mode at about 200 tokens per second), and DeepSeek 4.1 Flash in September 2026 "disabused of any notion that frontier intelligence was going to forever be a domain of just some large American tech companies".
Sources: Deep Dive · How I use LLMs · DHH on Lex Fridman · CS229 lecture · Rails World 2026 keynote · Kai on the Mac mini
Linked from
Memory-bandwidth-bound generationMixture of expertsModel parameters (weights)