Diamond
Agent
Embodied Learning

Multimodal and Embodied Learning Expecially for Science Discovery

In this research direction, we aim to develop systems that can leverage multimodal data and knowledge to solve expert tasks in various domains and accelerate scientific discovery. We desire to efficiently learn expert models which have lower resource requirements, greater generalizability, and better explainability of their behaviors compared to existing foundation models. Specifically, we aim to develop theories and techniques that combine geometric multimodal representation learning with analogy-based reasoning for learning embodied agents in a multimodal and multitask setting, improving learning efficiency, model generalizability, and model explainability.

Relevant Materials

Publications