About ME

Jiawei Liu (刘佳玮)

jiawei.liu@u.nus.edu Google Scholar Linkedin Github

Hello! I am a Research Scientist at the SAM Team at Meta Superintelligence Labs, ex-FAIR Perception. I obtained my PhD in Computer Science from National University of Singapore, advised by Prof. Mike Zheng Shou and Prof. Jussi Keppo. I obtained my Master's degree from Tsinghua University.
My research focuses on building generalist intelligence for the physical world that can perceive, predict, and act in real-world environments.
I pursue this goal through both vision and language foundation models. On the vision side, I have been working on multi-modal 3D/4D/video generation and perception. On the language side, I focus on enabling frontier LLMs with physical intelligence through mid- and post-training for coding capabilities.

News

  • 2026.04. Two papers SAM 3D and SAM 3D Body have been accepted to CVPR 2026, both as Oral Presentations!

  • 2025.11. Thrilled to announce our SAM 3D — two new foundation models that lift ANY object and body to 3D! Try it now: link | Paper/Code/Checkpoint: link

  • 2025.07. "SMS" is accepted to ICCV 2025. Congratulations!

  • 2024.10. Two papers "Exo2Ego-V", and "MeshRet" (Spotlight) are accepted to NeurIPS 2024. Congratulations!

  • 2024.10. Show-1 is accepted to International Journal of Computer Vision (IJCV). Congratulations!

  • 2024.08. I am awarded with Doctoral Consortium at ECCV 2024, "4D-Video Dynamic Scene Reconstruction, Editing, and Generation". Congratulations!

  • 2024.07. Our paper "MotionDirector: Motion Customization of Text-to-Video Diffusion Models." is accepted to ECCV 2024 (oral). Congratulations!

  • 2024.04. Our paper "Instant3D: Instant Text-to-3D Generation." is accepted to International Journal of Computer Vision (IJCV). Congratulations!

  • 2024.02. Six papers DynVideo-E, Ego-Exo4D, MagicAnimate, X-Adapter, VideoSwap, and VideoLLM-online are accepted to Computer Vision and Pattern Recognition (CVPR), 2024. Congratulations!

  • 2023.12. I am awarded with SINGAPORE DATA SCIENCE CONSORTIUM (SDSC) DISSERTATION RESEARCH FELLOWSHIP 2023, recognized as 1 out of 10 most innovative and impactful PhD students focusing on Data Science in Singapore in 2023. Congratulations!

  • 2023.07. Two papers HOSNeRF and STPrivacy are accepted to International Conference on Computer Vision (ICCV), 2023. Congratulations!

  • 2022.10. I am awarded with NeurIPS 2022 Scholar Award and will attend NeurIPS 2022 conference. Welcome to approach me and chat!

  • 2022.09. Our paper "DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes." is accepted by Thirty-sixth Conference on Neural Information Processing Systems (NeurIPS), 2022.

  • 2021.01. I am awarded with President's Graduate Fellowship (PGF) and start my PhD career in National University of Singapore.

  • 2020.06. I am awarded with Outstanding Graduate, which is awarded to TOP 8% students in Tsinghua University.

  • 2019.10. I am awarded with Scholarship of Second Prize in Tsinghua University, which is awarded to TOP 5% students in Tsinghua University.

  • 2019.08. Our paper "An efficient multibody dynamic model of three-dimensional meshing contacts in helical gear-shaft system and its solution." is accepted by Mechanism and Machine Theory (SCI source, Q1).

Selected Publications

SAM 3D: 3Dfy Anything in Images

SAM 3D Team (Contributed model mid-training and post-training).
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026 Oral Presentation
Paper Project Demo Code

A 1.8B flow matching model (6T training tokens) for single-image 3D generation, that predicts complete shape, pose, and texture for arbitrary masked objects. Contributed mid-training for layout model and post-training for geometry model. Created 4.5M high-quality and semantically-rich 3D layout data through physical simulation, and 130K high-fidelity hand shape data. Designed layout post-optimization method (+8.7% 3D IoU, +26.1% 2D IoU).

SAM 3D Body: Robust Full-Body Human Mesh Recovery

SAM 3D Body Team (Led SAM3D unification, qualitative model evaluation, and public release).
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026 Oral Presentation
Paper Project Demo Code

A promptable model for full-body single-image 3D human mesh recovery (HMR), that estimates complete pose and shape of the body, feet, and hands for arbitrary masked humans. Led SAM 3D unification, qualitative model evaluation, and user preference study. Drove end-to-end system release, including repository (GitHub & Hugging Face), paper revision, and live demo deployment.

Exocentric-to-Egocentric Video Generation.

Jia-Wei Liu*, Weijia Mao*, Zhongcong Xu, Jussi Keppo, Mike Zheng Shou.
Advances in Neural Information Processing Systems (NeurIPS) 2024.
PDF BibTex Code

DynVideo-E: Harnessing Dynamic NeRF for Large-Scale Motion- and View-Change Human-Centric Video Editing.

Jia-Wei Liu, Yan-Pei Cao, Jay Zhangjie Wu, Weijia Mao, Yuchao Gu, Rui Zhao, Jussi Keppo, Ying Shan, Mike Zheng Shou.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
PDF BibTex Project

HOSNeRF: Dynamic Human-Object-Scene Neural Radiance Fields from a Single Video.

Jia-Wei Liu, Yan-Pei Cao, Tianyuan Yang, Zhongcong Xu, Jussi Keppo, Ying Shan, Xiaohu Qie, Mike Zheng Shou.
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2023.
PDF BibTex Project Code

DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes.

Jia-Wei Liu, Yan-Pei Cao, Weijia Mao, Wenqiao Zhang, David Junhao Zhang, Jussi Keppo, Ying Shan, Xiaohu Qie, Mike Zheng Shou.
Advances in Neural Information Processing Systems (NeurIPS) 2022.
PDF BibTex Project Code

Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation.

David Junhao Zhang*, Jay Zhangjie Wu*, Jia-Wei Liu*, Rui Zhao, Lingmin Ran, Yuchao Gu, Difei Gao, Mike Zheng Shou. (* equal contribution)
International Journal of Computer Vision, 2024.
PDF BibTex Project Code


MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model.

Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, Mike Zheng Shou.
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
PDF BibTex Project Code

MotionDirector: Motion Customization of Text-to-Video Diffusion Models.

Rui Zhao, Yuchao Gu, Jay Zhangjie Wu, David Junhao Zhang, Jia-Wei Liu, Weijia Wu, Jussi Keppo, Mike Zheng Shou.
European Conference on Computer Vision (ECCV) 2024, oral.
PDF BibTex Project Code

An efficient multibody dynamic model of three-dimensional meshing contacts in helical gear-shaft system and its solution.

Jia-Wei Liu, Jia-Peng Liu, Xuan-Bo Shu, Aki Mikkola, Ge-Xue Ren.
Mechanism and Machine Theory (SCI source, Q1).
PDF BibTex

Dynamic characteristics of helical gear based on Arbitrary Lagrangian Eulerian formulation.

Jia-Wei Liu, Jia-Peng Liu, Jia-Lun Yao, Ge-Xue Ren.
ECCOMAS 2019 Multibody Dynamic Conference, Duisburg, Germany.
PDF BibTex


Selected Awards

  • Doctoral Consortium at ECCV 2024, "4D-Video Dynamic Scene Reconstruction, Editing, and Generation", 2024

  • SINGAPORE DATA SCIENCE CONSORTIUM (SDSC) DISSERTATION RESEARCH FELLOWSHIP 2023 (10000 S$) (1 out of 10 most innovative and impactful PhD students focusing on Data Science in Singapore in 2023), 2023

  • Show Lab Annual Award (4000 S$), National University of Singapore, 2023

  • Research Incentive Award, National University of Singapore, 2023

  • NeurIPS 2022 Scholar Award, 2022

  • President's Graduate Fellowship (PGF), National University of Singapore, 2021

  • Outstanding Graduate, Tsinghua University (Awarded to top 8% students), 2020

  • Scholarship of Second Prize in Tsinghua University (Awarded to top 5% students), 2019

  • National Scholarship (Awarded to top 1% students in Honors College), 2015

Academic Attendance and Services

  • Attendance: NeurIPS 2022@New Orleans, Meta Ego4D Meetup 2023@New York, ICCV 2023@Paris, Meta Research Summit 2024@Seattle, CVPR 2024@Seattle, ECCV 2024@Milan, ECCOMAS 2019@Duisburg.

  • Reviewer: IEEE TPAMI, TNNLS, TCSVT, TMLR, CVPR 2023-2026, ICCV 2023-2025, NeurIPS 2024-2025, ECCV 2024, SIGGRAPH 2024-2025, ICLR 2025, ICML 2025, SIGGRAPH Asia 2025, AAAI 2024-2026, MICCAI 2024-2025, ACMMM 2023-2024, etc.