In 1989, Yann LeCun helped show that neural networks could read handwritten ZIP codes, laying groundwork for systems that later read bank checks

Back in 1989, computer scientist Yann LeCun and colleagues decided to focus on one of the most common yet infuriating issues of the time: the automated reading of messy, handwritten zip codes on mail envelopes.

Namely, he intended to make a computer capable of reading handwritten ZIP code numbers on letters for the US Postal Service. Although, at first sight, it did not seem to be an interesting challenge, since mail sorting was boring and people's handwriting was terrible. They often wrote numbers in odd directions, sometimes stretching them, cramping them, or leaving gaps in the writing.Nevertheless, the everyday postal issue became an excellent impetus for the great revolution in technologies.

The problem became the ultimate test for computer vision . This term refers to the science of enabling computers to perceive images. By solving this practical problem, scientists showed that machines can distinguish complex visual patterns without human assistance, paving the way for the photo apps we use today.Why handwritten ZIP codes were hard for computersPrior to this groundbreaking achievement, computers used to be taught to distinguish between various objects and characters through a very rigid approach, where engineers would have to develop very precise algorithms for them to use.

For example, a programmer would instruct the computer to look for the number seven in its form of one horizontal line and one slanted vertical line. But since the human handwriting of the same figure might not precisely match this mathematical definition, the computer would not recognize it.

Real-life mail is littered with such examples.As stated by a 2024 publication in Cancers, this 1989 project represented the very point where LeCun and colleagues came up with his idea of using a convolutional neural network to read handwritten digits.

In contrast to previous attempts to impose a fixed rule on the computer, the new system worked by analysing small parts of each image and then combining those visual clues through layers. This helped the system recognise variations in handwriting, as it learned from patterns from examples without depneding only on fixed rules written by engineers.

Leaving the laboratory for the real worldWhat made this discovery notable was that it was not just a laboratory experiment carried out in an academic environment.

It was a solution for a practical problem that required speed and efficiency. The need to design such a system was dictated by the necessity to sort thousands of envelopes per hour.This practical success is often traced back to LeCun's 1989 work on handwritten digit recognition. In particular, the review from 2024, which can be found on the website of the medical database PubMed, notes the resulting model, LeNet-5, became the first practical system to use backpropagation for automatically recognizing handwritten ZIP codes on U.S.

Postal Service letters. Backpropagation is a technique that lets a computer measure its errors during training and adjust its settings to improve the next prediction. Thus, by proving that such a technique works in practice, the developers have shown that neural networks finally reached the moment to come out of the laboratory and solve practical problems of people.From envelopes to modern visionSince the machine successfully decoded the human handwriting from envelopes, the basic principle behind it started spreading far beyond the walls of a mailroom.

Such a technique appeared to be extremely flexible. If a computer could learn how to distinguish the unique shape of a single digit written in a handwritten way, it could easily learn to distinguish any other document.Indeed, the impact of this modest endeavor stretches far beyond anything those first scientists could ever think of back in 1989. The same general approach has since influenced deep learning tools used by doctors to interpret medical images such as scans and X-rays.One lesson from this 1989 story is that practical problems can drive major technological advances. Significant advances in artificial intelligence often come from solving concrete problems. Rather, they arise when scientists come up with an innovative approach to solving a practical issue. In other words, through the task of teaching machines to read envelopes, researchers were opening up new vistas on computer vision.