TY - JOUR
AU - 
AB - (QHUJ\'LIIHUHQFH This study investigates the effects of Markov chain Monte &RQWUDFWLRQ 2VFLOODWLRQ([SDQVLRQ Carlo (MCMC) sampling in unsupervised Maximum Likeli- 5HDOLVWLF hood (ML) learning. Our attention is restricted to the family VKRUWUXQ of unnormalized probability densities for which the negative VDPSOHV log density (or energy function) is a ConvNet. We find that many of the techniques used to stabilize training in previous ,QIRUPDWLYH YH 1RLVH ,QLWLDOL]DWLRQ RQ ,QLWLDOL]DWLRQ studies are not necessary. ML learning with a ConvNet poten- DQLVKLQJ9 ([SORGLQJ 3ULRU$UW UW UW 2XUV tial requires only a few hyper-parameters and no regulariza- JUDGLHQWV JUDGLHQWV tion. Using this minimal framework, we identify a variety of 5HDOLVWLF ML learning outcomes that depend solely on the implemen- ORQJUXQ tation of MCMC sampling. VDPSOHV On one hand, we show that it is easy to train an energy-based 0&0&&RQYHUJHQFH ,QIRUPDWLYH ,QLWLDOL]DWLRQ model which can sample realistic images with short-run &RQYHUJHQW1RQ&RQYHUJHQW 2XUV Langevin. ML can be effective and stable even when MCMC samples have much higher energy than true steady-state sam- ples throughout training. Based on this insight, we introduce an ML method with purely noise-initialized MCMC, high- Figure 1: Two axes characterize ML learning of ConvNet quality short-run synthesis, and the same budget as 
TI - On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models
JF - Proceedings of the AAAI Conference on Artificial Intelligence
DO - 10.1609/aaai.v34i04.5973
DA - 2020-04-03
UR - https://www.deepdyve.com/lp/unpaywall/on-the-anatomy-of-mcmc-based-maximum-likelihood-learning-of-energy-kxbuwuW5Rd
DP - DeepDyve
ER -