In high-level, philosophical discussions of deep learning, questions arise such as: when does theory follow empirical results and when do empirical results follow theory? (See Yann Lecun’s talk.) Another question is: how much does progress depend on strokes of genius by individuals versus an emergent process involving whole research communities? These questions of course apply to science and technology in general, not just deep learning.
Since I like to think about high-level, philosophical topics in deep learning and AI on this forum, I created the “Esoteric stuff” category to catch anything weird or miscellaneous that might be relevant. In particular, the philosophy, history, and science of science and technology.
On that note, here’s a blog post highlighting a fascinating and weird phenomenon known as simultaneous invention or multiple discovery:
Thomas Edison invented the light bulb in 1879. What if he had never been born, Would we still have light bulbs? And would they still have been invented in 1879? It turns out that this is not just a philosophical question and the answer is yes, the light bulb would have been invented at roughly the same time. We know this because at least 23 other people built prototype light bulbs before Edison, including two groups who filed patents and fought legal battles with him over the rights (Sawyer and Mann in the U.S. and Swan in England).
This is not a strange coincidence that happened with electric lighting, it is the norm in both technological invention and scientific and mathematical discovery. Newton and Leibniz independently invented calculus, Alexander Graham Bell and Elisha Gray both filed a patent for the telephone on the same day — within three hours of each other — and sunspots were simultaneously discovered by four scientists living in four different countries. The list of simultaneous independent inventions includes the airplane (2 people), the steamboat (5 people), photography (2 people), the telegraph (5 people), and the telescope (9 people). In science and math it includes decimal fractions (2 people), the theory of natural selection (2 people), the discovery of oxygen (2 people), molecular theory (2 people), and the conservation of energy (4 people).
A study by Ogburn and Thomas in 1922 produced a list of 148 major inventions and discoveries that were made independently by two or more groups at the same time. A similar study by Merton in 1960 led him to conclude that “the pattern of independent multiple discoveries in science is in principle the dominant pattern, rather than a subsidiary one”.