Ebook: Solving the reconstruction-generation trade-off: Generative model with implicit embedding learning

Variational Autoencoder (VAE) and Generative adversarial network (GAN) are two classic generative
models that generate realistic data from a predefined prior distribution, such as a Gaussian distribution.
One advantage of VAE over GAN is its ability to simultaneously generate high-dimensional data and learn
latent representations that are useful for data manipulation. However, it has been observed that a tradeoff exists between reconstruction and generation in VAE, as matching the prior distribution for the latent
representations may destroy the geometric structure of the data manifold. To address this issue, we propose an autoencoder-based generative model that allows the prior to learn the embedding distribution,
rather than imposing the latent variables to fit the prior. To preserve the geometric structure of the data
manifold to the maximum, the embedding distribution is trained using a simple regularized autoencoder
architecture. Then an adversarial strategy is employed to achieve a latent mapping. We provide both theoretical and experimental support for the effectiveness of our method, which eliminates the contradiction
between preserving the geometric structure of the data manifold and matching the distribution in latent
space. The code is available at https://github.com/gengcong940126/GMIEL.