Speaker: Yining Hong, University of California, Los Angeles

Time: 15:00 p.m., January 8, 2024, GMT+8

Venue: Room 204, Courtyard No.5, Jingyuan, PKU

Abstract: 

Powerful as recent large language models and vision language models can be, these models are not grounded in the 3D physical world like human beings, let alone explore and interact within the richer realm of 3D embodied environments. Yining Hong's work emphasizes the development of 3D embodied foundation models, dedicated to building general-purpose embodied agents that could actively explore and interact with the 3D physical world, and perform common sense reasoning within the embodied environment. These models facilitate dynamic interactions with 3D spaces, incorporating essential embodied concepts such as spatial relationships, affordances, physics, layout, multisensory learning and so on. Yining Hong's research specifically emphasizes on three critical perspectives in building such generalist embodied agents: Building 3D world models; embodied foundation models and common sense reasoning.

Source: Center on Frontiers of Computing Studies, PKU