UNITE-AUDIO: Joint Learning of Continuous Tokenization and Latent Flow Matching for Text-to-Audio Generation

Abstract

UNITE-AUDIO jointly trains continuous audio representation learning and latent flow matching for text-to-audio generation, instead of freezing a reconstruction-oriented tokenizer before generative training. The generative objective directly shapes the latent space, and Flow-GRPO post-training further improves text-conditioned generation. Experiments show competitive performance with a compact latent flow model and demonstrate the benefit of jointly optimizing the audio representation and generative model.

Publication
In arXiv preprint arXiv:2609.28206