Yann Dubois

Stanford CS PhD student who worked on Alpaca (Llama 7B fine-tuned on 52k LLM-generated instructions), AlpacaFarm and AlpacaEval (GPT-4 as a judge, with a length correction). His Summer 2024 CS229 guest lecture, "Building Large Language Models", is the archive's most technical walk through the llm-training-pipeline.

His position is that academia over-invests in architecture while industry wins on data, evaluation and systems. He backs it with scaling-laws: small architecture changes mostly shift a curve's intercept, while better data improves the whole curve. His advice, after Sutton's "bitter lesson": "do the simple things, do them well, scale them."

Source: CS229 lecture