Intrinsically Motivated Open-ended Learning

Link to IMOL page on neurips.cc

Mentorship session: Sign up here to chat with one of our research mentors! The list includes Anima Anandkumar, Dani Bassett, Doina Precup, Georg Martius, Jim Fan, Laura Schulz and Natalia Vélez.

Description

How do humans develop broad and flexible repertoires of knowledge and skills? How can we design autonomous lifelong learning machines with the same abilities?

A promising computational and scientific approach to these questions comes from the study of intrinsically motivated learning, sometimes called curiosity-driven learning (Oudeyer et al., 2007; Barto, 2013; Mirolli and Baldassarre, 2013, Schmidhuber, 2021); a framework that finds inspiration in the drive of humans and other animals to seek "interesting" situations for their own sake (White, 1959; Berlyne, 1960; Deci and Ryan, 1985). These intrinsic motivations (IM) have evolved in animals to drive exploratory behaviors, an essential component of efficient learning (Singh et al., 2010). When implemented in machines, they support the autonomous exploration of complex environments; a key component of many recent breakthrough in reinforcement learning (Bellemare et al., 2016; Pathak et al., 2017; Burda et al., 2019; Eysenbach et al., 2019; Warde-Farley et al., 2019; Pong et al., 2020; Raileanu and Rocktäschel, 2020; Sekar et al., 2020; Ecoffet et al., 2021; Stooke et al., 2021; Colas et al., 2022; Du et al., 2023; Adaptive Agent Team et al., 2023). In short, intrinsic motivations free artificial agents from relying on predefined learning signals and thereby offer a path towards autonomy and open-ended learning, a longstanding objective in the field of artificial intelligence.

Despite recent successes, today’s agents still lack the autonomy and flexibility required to learn and thrive in realistic open-ended environments. Such versatility requires the capacity to generalize to domains different from the ones encountered at design time, to adaptively create goals and switch between them, and to integrate incremental learning of skills and knowledge over longer periods of time. These issues are especially relevant for efforts to deploy artificial intelligence in the real world without human intervention, a topic of key concern in the NeurIPS community.

Better understanding and engineering of such flexible learning systems will require fresh approaches and cross-disciplinary conversations. We propose to bring these conversations to NeurIPS by introducing the growing field of Intrinsically Motivated Open-ended Learning (IMOL) . Taking roots in developmental robotics (Lungarella et al., 2003; Cangelosi and Schlesinger, 2015) , IMOL aims at a unified study of the motivational forces, learning architectures, and developmental and environmental constraints that support the development of open-ended repertoires of skills and knowledge over learners' lifetimes (e.g. , Barto et al., 2004; Baldassarre, 2011; Baranes and Oudeyer, 2013; Kulkarni et al., 2016; Santucci et al., 2016; Eysenbach et al., 2019; Colas et al., 2022).

More than a scientific approach, IMOL also represents an associated research community that emerged at the first IMOL workshop in 2009 and progressively developed into an active community across years of scientific events and activities. With this full-day workshop, we propose to reflect on recent advances, showcase on-going research and discuss open challenges for the future of IMOL research. To this end, we will bring together speakers, presenters and attendees from a diversity of IMOL-related fields including robotics, reinforcement learning, developmental psychology, evolutionary psychology, computational cognitive science, and philosophy.