Neural attention for translation
2014Major
Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio let a translation network learn which input words to focus on. Attention becomes the core of the Transformer.
▲ Matters Save Share Trace roots
Longest chain: The Big Bang → First stars ignite → Supernovae seed heavy elements → The Sun forms → Earth forms → First oceans → First life → Photosynthesis evolves → Great Oxidation Event → Complex cells (eukaryotes) → Multicellular life → First animals → Cambrian Explosion → First vertebrates → Jawed fish → Fish walk onto land → The amniotic egg → The Great Dying → Dinosaurs appear → Asteroid ends the dinosaurs → Early primates → Apes split from monkeys → Human line splits from chimpanzees → Walking upright → First stone tools → Genus Homo appears → Homo erectus → Control of fire → Habitual fire and cooking → Homo sapiens appears → Complex language → Natufian villages → Oldest known bread → Agriculture begins → Irrigation → Uruk and the first cities → Cylinder seals → Writing (cuneiform) → Babylonian mathematics → Babylonian astronomical diaries → Thales and natural philosophy → Trial and death of Socrates → Plato's Academy → Aristotle's Lyceum → Alexander's conquests → Death of Alexander and the Hellenistic kingdoms → Seleucid Empire founded → Parthian Empire founded → Sasanian Empire → Birth of Muhammad → Muhammad's first revelations → The Hijra → Conquest of Mecca → Death of Muhammad and the Rashidun caliphate → Arab conquests → Abbasid Caliphate → Battle of Talas → Paper reaches the Islamic world → House of Wisdom → Ibn al-Haytham's Book of Optics → Bacon's Novum Organum → Royal Society → Papin's steam digester → Newcomen steam engine → Watt's steam engine → Thermodynamics → Conservation of energy → Kinetic theory of gases → Boltzmann's statistical entropy → Quantum hypothesis → Bohr model of the atom → Quantum mechanics → Transistor → Integrated circuit → ARPANET → TCP/IP → World Wide Web → ImageNet → Deep learning breakthrough → Neural attention for translation
Community notes Loading notes…
Learn more ranked by votes
Book Ian Goodfellow, Yoshua Bengio and Aaron Courville
Deep Learning ↗ Editor's pick