TY - JOUR
T1 - Unified structured learning for simultaneous human pose estimation and garment attribute classification
AU - Shen, Jie
AU - Liu, Guangcan
AU - Chen, Jia
AU - Fang, Yuqiang
AU - Xie, Jianbin
AU - Yu, Yong
AU - Yan, Shuicheng
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2014/11/1
Y1 - 2014/11/1
N2 - In this paper, we utilize structured learning to simultaneously address two intertwined problems: 1) human pose estimation (HPE) and 2) garment attribute classification (GAC), which are valuable for a variety of computer vision and multimedia applications. Unlike previous works that usually handle the two problems separately, our approach aims to produce an optimal joint estimation for both HPE and GAC via a unified inference procedure. To this end, we adopt a preprocessing step to detect potential human parts from each image (i.e., a set of candidates) that allows us to have a manageable input space. In this way, the simultaneous inference of HPE and GAC is converted to a structured learning problem, where the inputs are the collections of candidate ensembles, outputs are the joint labels of human parts and garment attributes, and joint feature representation involves various cues such as pose-specific features, garment-specific features, and cross-task features that encode correlations between human parts and garment attributes. Furthermore, we explore the strong edge evidence around the potential human parts so as to derive more powerful representations for oriented human parts. Such evidences can be seamlessly integrated into our structured learning model as a kind of energy function, and the learning process could be performed by standard structured support vector machines algorithm. However, the joint structure of the two problems is a cyclic graph, which hinders efficient inference. To resolve this issue, we compute instead approximate optima using an iterative procedure, where in each iteration, the variables of one problem are fixed. In this way, satisfactory solutions can be efficiently computed by dynamic programming. Experimental results on two benchmark data sets show the state-of-the-art performance of our approach.
AB - In this paper, we utilize structured learning to simultaneously address two intertwined problems: 1) human pose estimation (HPE) and 2) garment attribute classification (GAC), which are valuable for a variety of computer vision and multimedia applications. Unlike previous works that usually handle the two problems separately, our approach aims to produce an optimal joint estimation for both HPE and GAC via a unified inference procedure. To this end, we adopt a preprocessing step to detect potential human parts from each image (i.e., a set of candidates) that allows us to have a manageable input space. In this way, the simultaneous inference of HPE and GAC is converted to a structured learning problem, where the inputs are the collections of candidate ensembles, outputs are the joint labels of human parts and garment attributes, and joint feature representation involves various cues such as pose-specific features, garment-specific features, and cross-task features that encode correlations between human parts and garment attributes. Furthermore, we explore the strong edge evidence around the potential human parts so as to derive more powerful representations for oriented human parts. Such evidences can be seamlessly integrated into our structured learning model as a kind of energy function, and the learning process could be performed by standard structured support vector machines algorithm. However, the joint structure of the two problems is a cyclic graph, which hinders efficient inference. To resolve this issue, we compute instead approximate optima using an iterative procedure, where in each iteration, the variables of one problem are fixed. In this way, satisfactory solutions can be efficiently computed by dynamic programming. Experimental results on two benchmark data sets show the state-of-the-art performance of our approach.
KW - Garment attribute classification
KW - Human pose estimation
KW - Joint inference
KW - Structured learning
UR - http://www.scopus.com/inward/record.url?scp=84908024723&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84908024723&partnerID=8YFLogxK
U2 - 10.1109/TIP.2014.2358082
DO - 10.1109/TIP.2014.2358082
M3 - Article
C2 - 25248181
AN - SCOPUS:84908024723
SN - 1057-7149
VL - 23
SP - 4786
EP - 4798
JO - IEEE Transactions on Image Processing
JF - IEEE Transactions on Image Processing
IS - 11
M1 - 2358082
ER -