Most successful machine learning algorithms of today use either carefully curated, human-labeled datasets, or large amounts of experience aimed at achieving well-defined goals within specific environments. In contrast, people learn through their agency: they interact with their environments, exploring and building complex mental models of their world so as to be able to flexibly adapt to a wide variety of tasks. One crucial next direction in artificial intelligence is to create artificial agents that learn in this flexible and robust way. Class will be a seminar, with discussion. Students will read and take turns presenting current works, and they will produce a proposal of a feasible next research direction. Topics build upon reinforcement learning techniques and include intrinsic motivation, model-based RL, hierarchical RL, physical simulation, self-supervised representation learning, multi-agent RL, grounded language learning, emergent communication, and optimal experiment design, with an eye towards learning complementary topics we see as fertile ground for new research.
The primary goal of this course is to provide a survey of techniques related to agents learning rich models of their world through interaction. Readings and discussions have been planned with an eye towards germinating new work, and the project component should serve as a way to get started. The course is suitable for students looking to dive into recent works (from e.g. NeurIPS, ICML, and ICLR). Students should walk away from the course understanding not only relevant state-of-the-art approaches, but also its limitations, gaps in our understanding, benchmarks waiting to be created, and useful methods waiting to be fruitfully combined. This is an exciting, rapidly-growing field with many opportunities to quickly get started!
Students will read required papers for each class, and students/groups of students (depending on enrollment) will take turns presenting. Note: each class has a required paper as well as a number of supplementary papers. The supplementary lists can get quite long -- this is not meant to be imposing! We simply think that each represents a handy constellation of papers for those interested in getting more acquainted with each subarea. We will try to provide a roadmap as we go, and feel free to ask for more context!
Students will produce a 6 page research proposal exploring a feasible next direction in depth, or some working demonstration of an idea. These will be “peer reviewed” by fellow students. For details, see our project timeline.
For full details including required papers and supplemental sources, please see readings.
Topics (particularly the later, more special-topic-oriented classes) subject to modification.