Simulation

First, the thing this is all in service of. The AI alignment problem is the challenge of making a powerful AI system reliably pursue what we actually want it to pursue - getting its goals, values, and behaviour to line up with human intentions, and to stay lined up even as the system becomes more capable than the people supervising it. It sounds simple and is not: we struggle to state our own values precisely, those values conflict between people, and an AI optimising hard for a slightly-wrong objective can produce outcomes nobody asked for. The multi-agent version - aligning one system with all of humanity at once, rather than a single person - is harder still, and it is the specific version Personal Universes is trying to dodge. ...

“Are we living in a simulation?” is one of those questions that sounds like late-night dorm-room talk and then turns out to have a serious literature behind it. The honest short answer to “how likely” is that nobody knows, and that the question may not even have a clean numerical answer. But that is not a reason to wave it away. The reasons we cannot confidently put a number on it are themselves interesting, and they tell us something real about the limits of probability, the nature of consciousness, and what counts as science. ...