PlayDiffusion: Non-Autoregressive Diffusion for Speech Editing

Describes PlayDiffusion, an open-source non-autoregressive (NAR) diffusion model engineered for speech editing, specifically tasks like inpainting (filling gaps) and word replacement.

Unlike traditional autoregressive (AR) models that regenerate entire sequences, PlayDiffusion employs a discrete diffusion process with iterative refinement of masked audio tokens and non-causal attention to efficiently make localized edits while preserving the surrounding context and speaker consistency.

This approach aims for seamless, high-quality edits and can also function as a fast NAR Text-to-Speech (TTS) system.

While promising for applications in audio production, accessibility, and interactive systems, challenges include computational cost, handling complex edits, ensuring multilingual robustness, and a current reliance on external APIs.