A Framework for the Digital Representation and Rendering of Chinese Jianpu Notation for Constructing a Synthetic OMR Dataset
摘要
The majority of Chinese music is disseminated through Jianpu, a distinctive musical notation system that uses numbers to denote pitches and underlines to indicate note durations. Despite its widespread use, Jianpu lacks robust optical recognition procedures, primarily due to the absence of large, annotated datasets. To address this challenge, we developed a comprehensive framework for the digital representation and rendering of Jianpu notation, aimed at constructing a synthetic dataset for Optical Music Recognition (OMR). Our framework employs a JSON-based representation system for Jianpu scores, enabling seamless conversion to and from other digital formats like MIDI and MusicXML. We implemented a hierarchical, highly parameterized algorithm for score layout and image rendering, ensuring diversity in the generated sheet images and the ability to produce corresponding semantic annotations, such as bounding boxes. By leveraging this framework, we created a synthetic dataset comprising images of Jianpu music sheets, both from existing and randomly generated scores. This dataset was subsequently used to train a convolutional neural network for OMR tasks. The trained network demonstrated outstanding performance when applied to real-world scanned Jianpu sheets, significantly reducing the need for manual annotation.