TY - JOUR
T1 - Deformable Generator Networks
T2 - Unsupervised Disentanglement of Appearance and Geometry
AU - Xing, Xianglei
AU - Gao, Ruiqi
AU - Han, Tian
AU - Zhu, Song Chun
AU - Wu, Ying Nian
N1 - Publisher Copyright:
© 1979-2012 IEEE.
PY - 2022/3/1
Y1 - 2022/3/1
N2 - We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field which is used to warp the generated appearance to obtain the final image or video sequences. Two generators take independent latent vectors as input to disentangle the appearance and geometric information from image or video sequences. For video data, a nonlinear transition model is introduced to both the appearance and geometric generators to capture the dynamics over time. The proposed scheme is general and can be easily integrated into different generative models. An extensive set of qualitative and quantitative experiments shows that the appearance and geometric information can be well disentangled, and the learned geometric generator can be conveniently transferred to other image datasets that share similar structure regularity to facilitate knowledge transfer tasks.
AB - We present a deformable generator model to disentangle the appearance and geometric information for both image and video data in a purely unsupervised manner. The appearance generator network models the information related to appearance, including color, illumination, identity or category, while the geometric generator performs geometric warping, such as rotation and stretching, through generating deformation field which is used to warp the generated appearance to obtain the final image or video sequences. Two generators take independent latent vectors as input to disentangle the appearance and geometric information from image or video sequences. For video data, a nonlinear transition model is introduced to both the appearance and geometric generators to capture the dynamics over time. The proposed scheme is general and can be easily integrated into different generative models. An extensive set of qualitative and quantitative experiments shows that the appearance and geometric information can be well disentangled, and the learned geometric generator can be conveniently transferred to other image datasets that share similar structure regularity to facilitate knowledge transfer tasks.
KW - Unsupervised learning
KW - deep generative model
KW - deformable model
UR - http://www.scopus.com/inward/record.url?scp=85124052043&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85124052043&partnerID=8YFLogxK
U2 - 10.1109/TPAMI.2020.3013905
DO - 10.1109/TPAMI.2020.3013905
M3 - Article
C2 - 32749961
AN - SCOPUS:85124052043
SN - 0162-8828
VL - 44
SP - 1162
EP - 1179
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
JF - IEEE Transactions on Pattern Analysis and Machine Intelligence
IS - 3
ER -