Affiliations: [a] Department of Industrial Engineering and Institute for Industrial Systems Innovation, Seoul National University, Seoul, Korea | [b] Institute of Engineering Research, Seoul National University, Seoul, Korea
Correspondence:
[*]
Corresponding author: Kyoyun Choi, Institute of Engineering Research, Seoul National University, Seoul, Korea. E-mail: andyhome1907@snu.ac.kr.
Abstract: Recognizing a singing melody from an audio signal in terms of the music notes’ pitch onset and offset, referred to as note-level singing melody transcription, has been studied as a critical task in the field of automatic music transcription. The task is challenging due to the different timbre and vibrato of each vocal and the ambiguity of onset and offset of the human voice compared with other instrumental sounds. This paper proposes a note-level singing melody transcription model using sequence-to-sequence Transformers. The singing melody annotation is expressed as a monophonic melody sequence and used as a decoder sequence. Overlapping decoding is introduced to solve the problem of the context between segments being broken. Applying pitch augmentation and and adding noisy dataset with data cleansing turns out to be effective in preventing overfitting and generalizing the model performance. Ablation studies demonstrate the effects of the proposed techniques in note-level singing melody transcription, both quantitatively and qualitatively. The proposed model outperforms other models in note-level singing melody transcription performance for all the metrics considered. For fundamental frequency metrics, the voice detection performance of the proposed model is comparable to that of a vocal melody extraction model. Finally, subjective human evaluation demonstrates that the results of the proposed models are perceived as more accurate than the results of a previous study.
Keywords: Automatic music transcription, deep learning, music information retrieval, sequence-to-sequence learning, singing melody transcription
