Skip to content

Separate Anything You Describe

Summary

Text-conditional source separation using CLAP (Contrastive Language-Audio Pretraining) embeddings. A user provides a natural language query (e.g., "banjo" or "separate the violin solo") and AudioSep separates the matching source from the mixture. Uses a ResUNet decoder conditioned on CLAP text embeddings. This is the first system to enable open-vocabulary source separation without predefined stem categories.

Key Claims

  • CLAP embeddings provide effective conditioning for open-vocabulary source separation
  • A single model can separate arbitrary sources described in natural language
  • Outperforms fixed-stem models on described sources, even for instrument categories not in training
  • Enables separation queries that fixed-stem models cannot express (e.g., "the second violin")