Modern Magnetic Systems
Note: This department has relocated.

BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion

2023

Conference Paper

ps


We show, for the first time, that neural networks trained only on synthetic data achieve state-of-the-art accuracy on the problem of 3D human pose and shape (HPS) estimation from real images. Previous synthetic datasets have been small, unrealistic, or lacked realistic clothing. Achieving sufficient realism is non-trivial and we show how to do this for full bodies in motion. Specifically, our BEDLAM dataset contains monocular RGB videos with ground-truth 3D bodies in SMPL-X format. It includes a diversity of body shapes, motions, skin tones, hair, and clothing. The clothing is realistically simulated on the moving bodies using commercial clothing physics simulation. We render varying numbers of people in realistic scenes with varied lighting and camera motions. We then train various HPS regressors using BEDLAM and achieve state-of-the-art accuracy on real-image benchmarks despite training with synthetic data. We use BEDLAM to gain insights into what model design choices are important for accuracy. With good synthetic training data, we find that a basic method like HMR approaches the accuracy of the current SOTA method (CLIFF). BEDLAM is useful for a variety of tasks and all images, ground truth bodies, 3D clothing, support code, and more are available for research purposes. Additionally, we provide detailed information about our synthetic data generation pipeline, enabling others to generate their own datasets. See the project page: https://bedlam.is.tue.mpg.de/.

Award: (Highlight Paper)
Author(s): Black, Michael J. and Patel, Priyanka and Tesch, Joachim and Yang, Jinlong
Book Title: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)
Pages: 8726--8737
Year: 2023
Month: June

Department(s): Perceiving Systems
Bibtex Type: Conference Paper (inproceedings)

Event Name: CVPR 2023
Event Place: Vancouver

Award Paper: Highlight Paper

Links: pdf
project
CVF
code

BibTex

@inproceedings{Black_2023_CVPR,
  title = {{BEDLAM}: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion},
  author = {Black, Michael J. and Patel, Priyanka and Tesch, Joachim and Yang, Jinlong},
  booktitle = {IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
  pages = {8726--8737},
  month = jun,
  year = {2023},
  doi = {},
  month_numeric = {6}
}