Music Source Separation in the Waveform Domain

Summary

First competitive waveform-domain music source separation model. Introduces a U-Net architecture with bidirectional LSTM that operates directly on raw audio rather than spectrograms. Achieved 6.3 dB SDR on MUSDB18 (6.8 with 150 extra songs), proving waveform-domain separation could compete with and surpass spectrogram methods. The model uses a convolutional encoder-decoder with LSTM bottleneck, trained end-to-end with L1 loss in the waveform domain.

Key Claims

• Waveform-domain models can match or exceed spectrogram-domain models without needing phase reconstruction (Griffin-Lim)
• The U-Net + BiLSTM architecture is effective for end-to-end waveform separation
• Training with data augmentation (pitch shift, time stretch, remixing) is critical
• Simple L1 loss on waveform outperforms more complex spectrogram losses

Related

• ../entities/demucs — the tool
• ../sources/2022-11-15-htdemucs — successor Hybrid Transformer Demucs
• ../concepts/spectrogram-unets — alternative spectrogram-domain approach
• ../entities/open-unmix — spectrogram-domain baseline