Alternatives to exploration? Moving up and down the ladder of causation in humans
Making adaptive decisions under uncertainty reflects a difficult yet ubiquitous challenge for human intelligence. Balancing decisions between those aimed at seeking information about the uncertain state of the environment (exploration) and those aimed at maximizing reward (exploitation) is key for artificial agents to behave adaptively. The large variability of human decisions under uncertainty has therefore been theorized and understood in terms of explicit policies aimed at solving this ‘explore-exploit trade-off’. In this talk, I will argue that the way we think about exploration, and the tasks that we use to study exploration in the lab, have biased our understanding of human decision-making under uncertainty. Across several studies, ranging from sensory- to reward-guided decisions, I will show that the variability of human decisions is not mainly driven by exploration policies, but rather by the scheme and precision with which humans learn the state of uncertain environments. By simulating the same effects in artificial agents trained and tested in the same conditions, I will defend the idea that the adaptability of human decisions under uncertainty does not only arise from the way humans choose as previously thought, but also from the way humans learn about their environment.
