Toward Generalizable and Intelligent Visual Reasoning Systems
PhD Thesis / Paper
Joy Hsu
I am a Member of Technical Staff at OpenAI and an incoming Assistant Professor in the Department of Computing at Imperial College. Previously, I completed my PhD in Computer Science at Stanford University as a Knight-Hennessy Scholar and NSF Fellow, advised by Jiajun Wu. I also received my BS and MS from Stanford University.
You can reach me at joy.cj.hsu@gmail.com.
Research
My research focuses on multimodal reasoning models & agentic systems: machines that can perceive, interpret, and interact safely with their environments. I am particularly interested in building models that generalize across diverse domains and long-horizon tasks, and do so robustly and reliably in the real world.
Publications
/ /
APIVOT: Adaptive Planning with Interleaved Vision-Language Thoughts
NeurIPS 2026 / Paper / Project Page
Learning Situated Awareness in the Real World
ICML 2026 / Paper / Project Page
Spotlight Presentation
A Tool Bottleneck Framework for Clinically-Informed and Interpretable Medical Image Understanding
MIDL 2026 / Paper / Project Page
Neuro-Symbolic Decoding of Neural Activity
ICLR 2026 / Paper / Project Page
Discovering Hybrid World Representations with Co-Evolving Foundation Models
AAAI 2026 / Paper
From Programs to Poses: Factored Real-World Scene Generation via Learned Program Libraries
NeurIPS 2025 / Paper / Project Page
What Makes a Maze Look Like a Maze?
ICLR 2025 / Paper / Project Page
Predicate Hierarchies Improve Few-Shot State Classification
ICLR 2025 / Paper / Project Page
Visually Descriptive Language Model for Vector Graphics Reasoning
TMLR / Paper / Project Page
CVPR Workshop On Multimodal Algorithmic Reasoning Spotlight Presentation
Naturally Supervised 3D Visual Grounding with Language-Regularized Concept Learners
CVPR 2024 / Paper
Learning Planning Abstractions from Language
ICLR 2024 / Paper / Project Page
What’s Left? Concept Grounding with Logic-Enhanced Foundation Models
NeurIPS 2023 / Paper / Project Page
Can Visual Scratchpads With Diagrammatic Abstractions Augment LLM Reasoning?
PMLR / Paper
NeurIPS ICBINB Workshop Best Poster Award
Composable Part-Based Manipulation
CoRL 2023 / Paper / Project Page
Motion Question Answering via Modular Motion Programs
ICML 2023 / Paper / Project Page
NS3D: Neuro-Symbolic Grounding of 3D Objects and Relations
CVPR 2023 / Paper / Project Page
CVPR Workshop On Compositional 3D Vision Oral Presentation
Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
ICLR 2023 / Paper / Project Page
Spotlight Presentation
DisCo: Improving Compositional Generalization in Visual Reasoning through Distribution Coverage
TMLR / Paper / Project Page
Geoclidean: Few-Shot Generalization in Euclidean Geometry
NeurIPS 2022 / Paper / Project Page
Capturing Implicit Hierarchical Structure in 3D Biomedical Images with Self-Supervised Hyperbolic Representations
NeurIPS 2021 / Paper / Project Page
NeurIPS Differential Geometry Workshop Contributed Talk
DARCNN: Domain Adaptive Region-based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images
CVPR 2021 / Paper / Project Page
Unsupervised Learning for Discovery in 2D & 3D Scenes: Towards Unbiased Understanding of Biomedical Images
Undergraduate Honors Thesis / Paper
Stanford Ben Wegbreit Prize for Best Thesis