Onsets and Frames

About

Google Magenta's piano transcription system combining onset detection with frame-level note prediction. CNN acoustic model + BiLSTM temporal model, trained on MAESTRO (200+ hours of aligned piano). Dual-objective: onset predictions gate frame activations into clean note events. 457+ citations. Part of Magenta (now archived).

Relevance

The onset+frame architecture is well-suited to plucked string instruments with sharp attacks. The onset detector's ability to precisely locate note starts is directly applicable to banjo — banjo attacks are even sharper and more percussive than piano. Architecture could be adapted for banjo with different pitch range and training data.

Mentions

• ../sources/2017-10-30-onsets-and-frames — paper
• ../entities/basic-pitch — more recent, instrument-agnostic
• ../entities/mt3 — successor using T5 transformer

Links

• Magenta: https://github.com/magenta/magenta (19.8k stars, archived)
• Colab demo: https://magenta.tensorflow.org/onsets-frames