AlphaGo

DeepMind's Go program, the first to beat top professionals: 5-0 over European champion fan-hui (Oct 2015, done in secret) and 4-1 over lee-sedol in Seoul (March 2016), about a decade earlier than experts predicted.

Architecture as explained on camera: a policy network (trained to imitate about 100k strong amateur games) proposes candidate moves; a value network estimates the probability of winning from a position; a tree search explores variations 50-60+ moves deep. After imitation it improved through millions of games of self-play reinforcement learning. It maximises win probability, not margin (win-probability-not-margin).

Signature moments: move-37 in game 2 (a move it rated at 1 in 10,000 for a human) and its game-4 collapse after Lee's move 78, when it wandered into one of its superhuman-blind-spots ("delusions" the team already knew about and couldn't fix before freezing version 18). andrej-karpathy cites move 37 as the template for RL surpassing human imitation (rlhf-is-not-rl).

Source: report