Sign in

Convolutional neural networks

1989Big moment

Yann LeCun's team trains a convolutional network with backpropagation to read handwritten zip codes. The design later powers the deep learning boom in vision.

The story

In 1989 Yann LeCun and colleagues at AT&T Bell Laboratories in New Jersey published a paper showing that a neural network trained with backpropagation could read handwritten zip code digits from U.S. Postal Service mail. The network used a design called a convolutional neural network, in which small filters slide across an image to detect features such as edges and strokes. The system, later named LeNet, was the ancestor of the networks that now power most computer vision.

The design borrowed from biology and earlier computing. In the 1950s and 1960s David Hubel and Torsten Wiesel found that neurons in the cat visual cortex respond to simple patterns in small regions of the visual field, and in 1980 the Japanese researcher Kunihiko Fukushima built the Neocognitron, a layered network inspired by that finding. But Fukushima's network was not trained by backpropagation. LeCun combined the layered structure with the training method popularized in 1986, which let the network learn the filters from examples instead of having them designed by hand.

The key ideas were local connections, so each unit looks at a small patch; shared weights, so the same feature detector is applied across the whole image, which drastically cuts the number of parameters; and pooling, which shrinks the image and makes the result less sensitive to small shifts. The network was trained on about 7,300 digits from the postal service and tested on about 2,000 others. A refined version, LeNet-5 of 1998, was deployed in bank check readers and processed a large share of checks in the United States. The approach then waited for more data and faster hardware. In 2012 AlexNet, a much larger convolutional network by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won the ImageNet contest by a wide margin and began the deep learning boom. LeCun, Hinton and Yoshua Bengio shared the 2018 Turing Award.

Why it mattered

  • The deep learning breakthrough of 2012 used a scaled-up convolutional network, AlexNet, to win the ImageNet competition.
  • Convolutional networks became the standard for image recognition, from photo tagging to medical imaging and self-driving car vision.
  • Handwriting and check reading became an early commercial success for neural networks.
  • Ideas such as weight sharing influenced the design of later neural network architectures.

Drafted by AI (Claude) from the sources above and checked against them by AI; no historian has reviewed this page. Spot a mistake? Suggest a correction. How pages are made

Community notes

Loading notes…

Learn more ranked by votes

BookCade Metz
Genius Makers
Search Open Library
BookTerrence J. Sejnowski
The Deep Learning Revolution
Search Open Library
VideoSearch YouTube
LeNet convolutional neural network history
Search YouTube
VideoSearch YouTube
Yann LeCun 1989 handwritten digits
Search YouTube