TY - JOUR
T1 - A tale of three probabilistic families
T2 - Discriminative, descriptive, and generative models
AU - Wu, Ying Nian
AU - Gao, Ruiqi
AU - Han, Tian
AU - Zhu, Song Chun
N1 - Publisher Copyright:
© 2019 Brown University.
PY - 2019
Y1 - 2019
N2 - The pattern theory of Grenander is a mathematical framework where patterns are represented by probability models on random variables of algebraic structures. In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models. A discriminative model is in the form of a classifier. It specifies the conditional probability of the class label given the input signal. A descriptive model specifies the probability distribution of the signal, based on an energy function defined on the signal. A generative model assumes that the signal is generated by some latent variables via a transformation. We shall review these models within a common framework and explore their connections. We shall also review the recent developments that take advantage of the high approximation capacities of deep neural networks.
AB - The pattern theory of Grenander is a mathematical framework where patterns are represented by probability models on random variables of algebraic structures. In this paper, we review three families of probability models, namely, the discriminative models, the descriptive models, and the generative models. A discriminative model is in the form of a classifier. It specifies the conditional probability of the class label given the input signal. A descriptive model specifies the probability distribution of the signal, based on an energy function defined on the signal. A generative model assumes that the signal is generated by some latent variables via a transformation. We shall review these models within a common framework and explore their connections. We shall also review the recent developments that take advantage of the high approximation capacities of deep neural networks.
UR - http://www.scopus.com/inward/record.url?scp=85063681949&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85063681949&partnerID=8YFLogxK
U2 - 10.1090/qam/1528
DO - 10.1090/qam/1528
M3 - Article
AN - SCOPUS:85063681949
SN - 0033-569X
VL - 77
SP - 423
EP - 465
JO - Quarterly of Applied Mathematics
JF - Quarterly of Applied Mathematics
IS - 2
ER -