Title: 3D Robot Vision for Structured World Understanding: A Physics and Symmetry Perspective
Abstract: Deploying robots in diverse real-world environments is a fundamental challenge. While recent AI advances are impressive, robots still struggle to generalize. I argue that a key missing piece in embodied intelligence is the structured understanding of the world: how geometries compose, how physics governs interactions, and how dynamics unfold. My research in 3D vision develops this understanding with two complementary principles: physics-based reasoning and symmetry-aware learning. First, I present Vysics, fusing vision and contact-rich physics to overcome heavy occlusions in object reconstruction, and my recent follow-up work that further incorporates 3D generative prior for reconstructions with both high visual fidelity and physical compliance. Then, I demonstrate my work on leveraging symmetry for efficient modeling of 3D geometry and dynamics. I introduce my algorithmic contributions in equivariant learning, including E2PN which improves the efficiency of point cloud learning by 7x compared with prior work, and Lie Neurons and Reductive Lie Neurons, which expand the scope of symmetry preserved by equivariant networks from rotations to general linear transformations. These advances enable significant progress in various robotic tasks by incorporating symmetry, from segmentation and place recognition to odometry and dynamics learning. I will close with my vision of building structured world representations that are simultaneously grounded in physics, informed by data, and structured by symmetry, toward robots that truly understand their physical world.
Bio: Minghan Zhu is a Postdoctoral Researcher at the GRASP Lab, University of Pennsylvania, advised by Michael Posa, and an Assistant Research Scientist at the University of Michigan, advised by Maani Ghaffari. He received his Ph.D. in Mechanical Engineering from the University of Michigan, advised by Huei Peng and Maani Ghaffari.
His research interest is building structured 3D representations as the foundation for robot perception, reasoning, and action. His work focuses on developing robust spatial understanding, such as geometry, pose, physical properties, and dynamics models, in visually challenging environments by incorporating data-driven 3D priors and real-world structures, such as symmetry and physics. His goal is to enable efficient and generalizable robot operation in cluttered environments.
Speaker: Minghan Zhu, Postdoc at UMich and UPenn
⏰ Friday, Nov 14, 2025 – 2:30 to 3:30pm EST
📍 Montgomery Knight – 317
🔗 https://gatech.zoom.us/j/99474053718
Host: Lu Gan