Problem Identification

The development and iteration of autonomous driving and robotics policies are limited by the high costs, low efficiency, and safety risks of real-world testing. The rise of Generative AI offers a potential breakthrough by enabling high-fidelity, interactive, and editable simulation testing.

Technical Approach

My research focused on building "World Models" to drive simulation with generative methods. Technically, I explored two core directions: 1) High-fidelity Scene Reconstruction: Building "digital twins" of real scenes using technologies like NeRF or Gaussian splatting. 2) Controllable Content Generation: On the basis of reconstructed scenes, leveraging the generative priors of diffusion models to provide endless, controllable scene variations and edge cases.

Selected Publications

• ^†PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Model, CVPR 2025.
• ^†Ctrl-U: Robust Conditional Image Generation Via Uncertainty-aware Reward Modeling, ICLR 2025.
• ^†SCP-Diff: Spatial-Categorical Joint Prior for Diffusion Based Semantic Image Synthesis, ECCV 2024.

Problem Identification

2D/3D perception is fundamental to embodied intelligence, but the extremely high cost of data annotation severely restricts the development of perception models.

Technical Approach

My early research focused on data-efficient perception learning algorithms, particularly semi-supervised learning and domain adaptation. In my first ICCV paper, DQS3D, I proposed a single-stage, densely-matched semi-supervised learning framework for 3D object detection, addressing the issue of insufficient training signals caused by sparse matching in previous methods. I also explored various levels of perception tasks such as self-supervised depth estimation, indoor layout estimation, and HD map generation, mastering task-oriented neural network and representation design methods.

Publications

• ^†DQS3D: Densely-matched Quantization-aware Semi-supervised 3D Detection, ICCV 2023.
• ^†From Semi-supervised to Omni-supervised Room Layout Estimation Using Point Clouds, ICRA 2023.
• ^†Training-Free Model Merging for Multi-target Domain Adaptation, ECCV 2024.