David Deutsch: What is knowledge, and where does it come from?


Lately I’ve been reading (listening to) The Beginning of Infinity by David Deutsch. The theme of the book is how knowledge is created. For example, how does evolution create the knowledge in a beaver of how to build a dam? How do humans create scientific knowledge? In an AI system or an artificially evolved entity, where does the knowledge about how to perform tasks come from?

I keep returning to Deutsch’s theory of knowledge when thinking about AI. For instance, lately I’ve been learning about imitation learning — e.g. how to transfer humans’ knowledge of how to drive a car to a neural network by letting the neural network observe human driving and state/action pairs. But recently it occurred to me that all supervised learning — the dominant family of techniques in AI today — is attempting to solve that same problem. Manually labelling a training dataset of millions of images is an attempt to transfer knowledge of object categories from human brains to a neural network. (It might have been Lex Fridman in one of his recent lectures who said that neural networks have many human teachers.)

With reinforcement learning, the knowledge inherent in the policy is created through trial and error (or generate and test) — i.e. random exploration through the space of possible actions. But there is also some knowledge embedded in the reward function, which is either explicitly specified by humans, or in the case of inverse reinforcement learning, learned from humans through observation. Any reward function for autonomous driving, for instance, is going to include human knowledge about physical road infrastructure, road users, and best practices derived from a century of human driving. It might seem trivial, but even specifying that the car should prefer to stay on the road rather than drive on the sidewalk or on the grass is imparting some knowledge about how to drive well.

When discussing artificial general intelligence (AGI), or human-level AI, it is useful to ask: where will the knowledge about how to build such a system come from? What knowledge of nature, or scientific knowledge — if any — is required? Where do ideas about how to improve neural networks come from? Where did foundational ideas like artificial neurons (or perceptrons), backpropagation, and stochastic gradient descent come from? How much is AI research and development influenced, either consciously or subconsciously, by ideas derived from the study of biological systems?

Here are some short-form introductions to the ideas in The Beginning of Infinity.

17-minute TED Talk:

Hour-long podcast interview: