AI-rewritten: This is a summary of an article from Quanta Magazine, rewritten by AI (Qwen, running locally) to make it easier to read. The facts come from the original article – read it for the full story.

---

Dzmitry Bahdanau, a 23-year-old researcher at the University of Montreal, helped develop "attention," a mechanism that solved a major bottleneck in artificial neural networks. In 2014, existing models struggled to translate long sentences because they had limited memory capacity. Bahdanau proposed a mathematical operation allowing networks to focus on specific parts of a sentence when generating new words. His colleague Kyunghyun Cho and adviser Yoshua Bengio supported the idea. The resulting approach enabled machines to handle complex translations just as well as short ones, making the technology scalable.

In 2017, Google researchers integrated this method into a new architecture called a transformer in a paper titled "Attention Is All You Need." This team stripped away previous components and reported that transformers achieved state-of-the-art translation results while requiring far less computation than prior models. The paper has been cited more than 250,000 times and is considered a watershed moment in AI. Large language models like ChatGPT and Claude follow this design. Attention allows a single model to converse, diagnose diseases, generate videos, and solve math problems by understanding context.

The attention mechanism works by converting words into numerical vectors and using weights to determine how much information from other words influences the current word’s meaning. The system uses "attention heads" to compare a word’s query with the keys of all other words in the sentence. If vectors point in similar directions, the information from that other word contributes more heavily to the updated representation. This process happens in parallel through multiple layers. In the encoder layer, words build up contextual understanding; for example, the word "it" learns to refer to "animal" rather than "street" based on which key matches its query best. Later layers accumulate these associations, giving words representations resembling meaning before the decoder generates the final translation.