Generative modeling has made tremendous progress with recent text-to-image systems like DALL-E 2 (Ramesh et al.,2022), Imagen (Saharia et al.,2022b), Parti (Yu et al.,2022), CogView (Ding et al.,2021) and Latent Diffusion (Rombach et al.,2022). Diffusion models (Sohl-Dickstein