MEBOW: Monocular Estimation of Body Orientation In the Wild

Chenyan Wu (1,2), Yukun Chen (1), Jiajia Luo (2), Che-Chun Su (2), Anuja Dawane (2),
Bikramjot Hanzra (2), Zhuo Deng (2), Bilan Liu (2), James Z. Wang (1), Cheng-hao Kuo (2)
(1) The Pennsylvania State University, University Park
(2) Amazon Lab126

Abstract:

Body orientation estimation provides crucial visual cues in many applications, including robotics and autonomous driving. It is particularly desirable when 3-D pose estimation is difficult to infer due to poor image resolution, occlusion, or indistinguishable body parts. We present COCO-MEBOW (Monocular Estimation of Body Orientation in the Wild), a new large-scale dataset for orientation estimation from a single in-the-wild image. The body-orientation labels for around 130K human bodies within 55K images from the COCO dataset have been collected using an efficient and high-precision annotation pipeline. We also validated the benefits of the dataset. First, we show that our dataset can substantially improve the performance and the robustness of a human body orientation estimation model, the development of which was previously limited by the scale and diversity of the available training data. Additionally, we present a novel triple-source solution for 3-D human pose estimation, where 3-D pose labels, 2-D pose labels, and our body-orientation labels are all used in joint training. Our model significantly outperforms state-of-the-art dual-source solutions for monocular 3-D human pose estimation, where training only uses 3-D pose labels and 2-D pose labels. This substantiates an important advantage of MEBOW for 3-D human pose estimation, which is particularly appealing because the per-instance labeling cost for body orientations is far less than that for 3-D poses. The work demonstrates high potential of MEBOW in addressing real-world challenges involving understanding human behaviors. Further information of this work is available at https://chenyanwu.github.io/MEBOW/ .


Full Paper

(PDF, 6.8MB)


Citation: Chenyan Wu, Yukun Chen, Jiajia Luo, Che-Chun Su, Anuja Dawane, Bikramjot Hanzra, Zhuo Deng, Bilan Liu, James Z. Wang and Cheng-hao Kuo, ``MEBOW: Monocular Estimation of Body Orientation In the Wild,'' Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 3451-3461, 2020.

© 2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.

Last Modified: March 29, 2020.
© 2020