Dissertations / Theses on the topic 'Optical music recognition (OMR)'

Die Bayerische Staatsbibliothek hat im Sommer 2016 ein Projekt zur automatischen Notenerkennung (Optical Music Recognition, OMR) gestartet. Dieser Beitrag beschreibt zunächst die Funktionsweise und die Qualität von OMR-Programmen und geht dann auf die bisherigen Erfahrungen der Bayerischen Staatsbibliothek mit OMR ein. Anschließend werden die Anwendungsszenarien skizziert, die die Bayerische Staatsbibliothek aufsetzend auf den OMR-Daten entwickeln wird.<br>During summer 2016, the Bavarian State Library has started a project on optical music recognition (OMR). This paper describes the functionality and quality of OMR programs at first and then depicts the OMR-experiences of the Bavarian State Library. Finally, the use case scenarios are outlined that the Bavarian State Library will implement on top of the OMR data.

<p> Optical Music Recognition (OMR) is a process that automatically converts the image of a music score into symbolic data. OMR can be divided into two main steps: recognition, the goal of which is to recognize &ldquo;valid&rdquo; music symbols, and interpretation, to understand the music meaning, such as pitch and rhythm. We focus on the interpretation problem, and more specifically, rhythm interpretation on piano scores. </p><p> In this thesis work, we propose a graph-based algorithm, which interprets rhythm by building a <i>rhythm graph</i> on all symbols in a system measure. Our approach represents the notes and rests in a system measure as the vertices of a graph. Then we build the graph by adding <i>voice edges </i> and <i>coincidence edges</i> between pairs of vertices. The graph is constructed under the constraint such that it leads to a meaningful rhythm interpretation. We score the graph based on music notation rules and choose the graph that has the best score. The problem is thus converted into a constrained optimization problem of finding the graph with the highest score. The rhythmic interpretation follows simply from the connected rhythm graph. </p><p> To evaluate the graph-based algorithm, we perform an experiment on a dataset specifically built to cover different types of rhythmic challenges encountered in polyphonic piano scores. We conclude that our algorithm is capable of applying measure level notation rules and finding the globally optimal interpretation, even in examples with splitting and merging voices as well as missing tuplets.</p><p>

Der Beitrag beschreibt ein laufendes Projekt zur computergestützten Erschließung und Analyse einer großen Sammlung handschriftlicher Liedblätter mit Volksliedern aus dem deutschsprachigen Raum. Am Beispiel dieses praktischen Projekts werden Chancen und Herausforderungen diskutiert, die der Einsatz von Digital Humanities-Methoden für den Bereich der Musikwissenschaft mit sich bringt.<br>This article presents an ongoing project for the computer-based transcription and analysis of handwritten music scores from a large collection of German folk tunes. Based on this project, I will discuss the challenges and opportunities that arise when using Digital Humanities methods in musicology.

In recent years, great progress has been made in the area of Artificial Intelligence (AI) due to the possibilities of Deep Learning which steadily yielded new state-of-the-art results especially in many image recognition tasks. Currently, in some areas, human performance is achieved or already exceeded. This great development already had an impact on the area of Optical Music Recognition (OMR) as several novel methods relying on Deep Learning succeeded in specific tasks. Musicologists are interested in large-scale musical analysis and in publishing digital transcriptions in a collection enabling to develop tools for searching and data retrieving. The application of OMR promises to simplify and thus speed-up the transcription process by either providing fully-automatic or semi-automatic approaches. This thesis focuses on the automatic transcription of Medieval music with a focus on square notation which poses a challenging task due to complex layouts, highly varying handwritten notations, and degradation. However, since handwritten music notations are quite complex to read, even for an experienced musicologist, it is to be expected that even with new techniques of OMR manual corrections are required to obtain the transcriptions. This thesis presents several new approaches and open source software solutions for layout analysis and Automatic Text Recognition (ATR) for early documents and for OMR of Medieval manuscripts providing state-of-the-art technology. Fully Convolutional Networks (FCN) are applied for the segmentation of historical manuscripts and early printed books, to detect staff lines, and to recognize neume notations. The ATR engine Calamari is presented which allows for ATR of early prints and also the recognition of lyrics. Configurable CNN/LSTM-network architectures which are trained with the segmentation-free CTC-loss are applied to the sequential recognition of text but also monophonic music. Finally, a syllable-to-neume assignment algorithm is presented which represents the final step to obtain a complete transcription of the music. The evaluations show that the performances of any algorithm is highly depending on the material at hand and the number of training instances. The presented staff line detection correctly identifies staff lines and staves with an $F_1$-score of above $99.5\%$. The symbol recognition yields a diplomatic Symbol Accuracy Rate (dSAR) of above $90\%$ by counting the number of correct predictions in the symbols sequence normalized by its length. The ATR of lyrics achieved a Character Error Rate (CAR) (equivalently the number of correct predictions normalized by the sentence length) of above $93\%$ trained on 771 lyric lines of Medieval manuscripts and of 99.89\% when training on around 3.5 million lines of contemporary printed fonts. The assignment of syllables and their corresponding neumes reached $F_1$-scores of up to $99.2\%$. A direct comparison to previously published performances is difficult due to different materials and metrics. However, estimations show that the reported values of this thesis exceed the state-of-the-art in the area of square notation. A further goal of this thesis is to enable musicologists without technical background to apply the developed algorithms in a complete workflow by providing a user-friendly and comfortable Graphical User Interface (GUI) encapsulating the technical details. For this purpose, this thesis presents the web-application OMMR4all. Its fully-functional workflow includes the proposed state-of-the-art machine-learning algorithms and optionally allows for a manual intervention at any stage to correct the output preventing error propagation. To simplify the manual (post-) correction, OMMR4all provides an overlay-editor that superimposes the annotations with a scan of the original manuscripts so that errors can easily be spotted. The workflow is designed to be iteratively improvable by training better models as soon as new Ground Truth (GT) is available<br>In den letzten Jahre wurden aufgrund der Möglichkeiten durch Deep Learning, was insbesondere in vielen Bildbearbeitungsaufgaben stetig neue Bestwerte erzielte, große Fortschritte im Bereich der künstlichen Intelligenz (KI) gemacht. Derzeit wird in vielen Gebieten menschliche Performanz erreicht oder mittlerweile sogar übertroffen. Diese großen Entwicklungen hatten einen Einfluss auf den Forschungsbereich der optischen Musikerkennung (OMR), da verschiedenste Methodiken, die auf Deep Learning basierten in spezifischen Aufgaben erfolgreich waren. Musikwissenschaftler sind in großangelegter Musikanalyse und in das Veröffentlichen von digitalen Transkriptionen als Sammlungen interessiert, was eine Entwicklung von Werkzeugen zur Suche und Datenakquise ermöglicht. Die Anwendung von OMR verspricht diesen Transkriptionsprozess zu vereinfachen und zu beschleunigen indem vollautomatische oder semiautomatische Ansätze bereitgestellt werden. Diese Arbeit legt den Schwerpunkt auf die automatische Transkription von mittelalterlicher Musik mit einem Fokus auf Quadratnotation, die eine komplexe Aufgabe aufgrund der komplexen Layouts, der stark variierenden Notationen und der Alterungsprozesse der Originalmanuskripte darstellt. Da jedoch die handgeschriebenen Musiknotationen selbst für erfahrene Musikwissenschaftler aufgrund der Komplexität schwer zu lesen sind, ist davon auszugehen, dass selbst mit den neuesten OMR-Techniken manuelle Korrekturen erforderlich sind, um die Transkription zu erhalten. Diese Arbeit präsentiert mehrere neue Ansätze und Open-Source-Software-Lösungen zur Layoutanalyse und zur automatischen Texterkennung (ATR) von frühen Dokumenten und für OMR von Mittelalterlichen Mauskripten, die auf dem Stand der aktuellen Technik sind. Fully Convolutional Networks (FCN) werden zur Segmentierung der historischen Manuskripte und frühen Buchdrucke, zur Detektion von Notenlinien und zur Erkennung von Neumennotationen eingesetzt. Die ATR-Engine Calamari, die eine ATR von frühen Buchdrucken und ebenso eine Erkennung von Liedtexten ermöglicht wird vorgestellt. Konfigurierbare CNN/LSTM-Netzwerkarchitekturen, die mit dem segmentierungsfreien CTC-loss trainiert werden, werden zur sequentiellen Texterkennung, aber auch einstimmiger Musik, eingesetzt. Abschließend wird ein Silben-zu-Neumen-Algorithmus vorgestellt, der dem letzten Schritt entspricht eine vollständige Transkription der Musik zu erhalten. Die Evaluationen zeigen, dass die Performanz eines jeden Algorithmus hochgradig abhängig vom vorliegenden Material und der Anzahl der Trainingsbeispiele ist. Die vorgestellte Notenliniendetektion erkennt Notenlinien und -zeilen mit einem $F_1$-Wert von über 99,5%. Die Symbolerkennung erreichte eine diplomatische Symbolerkennungsrate (dSAR), die die Anzahl der korrekten Vorhersagen in der Symbolsequenz zählt und mit der Länge normalisiert, von über 90%. Die ATR von Liedtext erzielte eine Zeichengenauigkeit (CAR) (äquivalent zur Anzahl der korrekten Vorhersagen normalisiert durch die Sequenzlänge) von über 93% bei einem Training auf 771 Liedtextzeilen von mittelalterlichen Manuskripten und von 99,89%, wenn auf 3,5 Millionen Zeilen von moderner gedruckter Schrift trainiert wird. Die Zuordnung von Silben und den zugehörigen Neumen erreicht $F_1$-werte von über 99,2%. Ein direkter Vergleich zu bereits veröffentlichten Performanzen ist hierbei jedoch schwer, da mit verschiedenen Material und Metriken evaluiert wurde. Jedoch zeigen Abschätzungen, dass die Werte dieser Arbeit den aktuellen Stand der Technik darstellen. Ein weiteres Ziel dieser Arbeit war es, Musikwissenschaftlern ohne technischen Hintergrund das Anwenden der entwickelten Algorithmen in einem vollständigen Workflow zu ermöglichen, indem eine benutzerfreundliche und komfortable graphische Benutzerschnittstelle (GUI) bereitgestellt wird, die die technischen Details kapselt. Zu diesem Zweck präsentiert diese Arbeit die Web-Applikation OMMR4all. Ihr voll funktionsfähiger Workflow inkludiert die vorgestellten Algorithmen gemäß dem aktuellen Stand der Technik und erlaubt optional manuell zu jedem Schritt einzugreifen, um die Ausgabe zur Vermeidung von Folgefehlern zu korrigieren. Zur Vereinfachung der manuellen (Nach-)Korrektur stellt OMMR4all einen Overlay-Editor zur Verfügung, der die Annotationen mit dem Scan des Originalmanuskripts überlagert, wodurch Fehler leicht erkannt werden können. Das Design des Workflows erlaubt iterative Verbesserungen, indem neue performantere Modelle trainiert werden können, sobald neue Ground Truth (GT) verfügbar ist

Optical Music Recognition OMR refers to convert music scores into a machine interpretable form.Actually it is similar to Optical Character Recognition OCR problem.We have generated HOMUS dataset for recognition of music symbol.Staff line detection and removal is one of the main preprocessing steps for symbol recognition as recognition of symbol is easier after staff line removal.We proposed a new deterministic,robust staff line detection and removal technique that divides the music documents vertically into a number of strip and then path is computed using stable path approach method.Then paths are connceted between vertical strips where the paths deviation between vertical strips is less and thus resulting in detection of line of each path and remove those detected line.We have tested our algorithm on the dataset of the recent staff removal competition held under the International Conference of Document Analysis and Recognition(ICDAR) 2011.Experimentally it has been found that our algorithm gives better result in comparison to other existing approaches.In OMR for the recognition of music symbol HOMUS dataset is also generated

碩士<br>國立師範大學<br>資訊教育研究所<br>86<br>This paper addresses the problem of taking printed sheet music and translating it into a MIDI format computer file. The automatic optical music score recognition system provides a more efficient and convenient approach of converting printed sheet music into MIDI file. The system recognizes music scores in a two-step process: (1) music symbol detection phase, which obtains a digitized image of the sheet music and recognizes the musical notations in the digitized image; and (2) music symbol emendation phase, which uses musical knowledge to correct the mis-identified musical symbols and then outputs the result to a pre-defined intermediary format file. The output file contains information of music symbols and confidence level values, these information could be displayed through a musical editor system developed for this research.Position and scale of the sheet music on the scanner is not restricted. After analyzing the sheet music, most of the music symbols could be recognized with only single image processing techniques. The experiments show a 91.2% recognition rate is achievable at 100 dpi taking approximately 1 minute per page. At 150 dpi, it is about 92.5% with an average processing time of 2 minute per page. At 300 dpi, the recognition rate is raised to 93.4% with an average processing time of 10 minute per page.

碩士<br>國立臺灣科技大學<br>電機工程系<br>101<br>The purpose of the OMR(Optical Music Recognition) is to recognize the melody and rhythm in the music score by using the image processing techniques. The application of OMR includes musical assisted instruction, musical accompaniment, digital archives, etc. In this thesis, a skew detection method with high precision has been proposed. The noise of input data for the least square estimation can be filtered according to the characteristics of staff in every single column. Then the correctness of staff detection and removal that dependent on skew correction can be promoted simultaneously. The musical object can be classified with the number of stem and head. According to the different feature of classes, the value of musical object can be recognized by width-height ratio, template matching, regional projection and vertical segment analysis separately. The pitch of musical object can be recognized by the relative position between head and staff. The final recognition result is obtained by combining the recognition result of pitch and value. The experimental results show that the proposed percussion-based OMR system can recognize the measures sequentially with low computational complexity and high recognition rate.

碩士<br>明新科技大學<br>機械工程系精密機電工程碩士班<br>103<br>In recent years, service robots are widely used in difference fields. Entertainment has become an important issue for robot applications. Automatic recognition of music score must immediately offer the music information to the entertainment robot for playing music instruments in real time mode. This study proposes an approach to extract the music information from camera-captured music score. The system uses a monochrome progressive scan camera to capture the image of a printed music score. Present music recognition technology, often due to paper or camera placed skew cause a decline in the rate of recognition. As such, the development of a real-time music recognition technology, improve accuracy when playing the piano robot, indeed a need. The resolution of the gigabit Ethernet interface camera is 3296 by 2472 pixels, pixel size of 5.5μm.The distance between the sheet music and the camera is about 85 cm. The proposed method consists of three main modules: the music region segmentation and the distortion correction module, the note identification module, the music player code format merger and output module. First of all, Now note analyzes the characteristics depend on the horizontal or vertical, image distortion will affect the subsequent recognition rate, so the first image morphing process, next identify the notes using different vertical cross-sectional area changes to identify notes, to find lever position of notes, according to different types of notes and analyze the contents of the notes, then identification results were corrected of music theory, finally, the identification results to the robot and complete music recognition. Finally, previous use Matrox Image Library, because expensive and inconvenient in operation, so replaced “openCV” image Library. Using different Sheet Music for testing in this study, recognition time was about 3 seconds and the overall recognition rate of 98% or more.

碩士<br>淡江大學<br>電機工程學系<br>88<br>Presently, there are three different ways to learn music for blind persons: 1) They know the details of the music after they have listened to music played by tapes or CDs. However, only very few blind persons have such incredible talent. 2) They understand music by touching the Braille music scores. However, due to the complexity of manually translating music scores into Braille music scores and the low commercial profit, the quantity of Braille music scores is very low in Taiwan. 3) By incorporating translating software with a MIDI-Keyboard as an input device, the music played by the user can be directly translated into Braille music scores. Though is very efficient, only music without chord can be translated. Furthermore, there will be some timing problems. Based on the above discussions, we know that it is very difficult for blind persons to either work as musicians or play music instruments just for fun. One possible solution to the problem is to develop an optical Braille music score translating software. A user can first scan music scores and then run the translating software. The recognition results can be either output to Braille or used to generate Braille music scores. By using the software, we can greatly lower the labor fee and shorten the manufacturing time. The success of the optical Braille music score translating software greatly depends on the performance of the "optical music recognition" systems. Automation of generating Braille music scores becomes possible only if the "optical music recognition" system can efficiently recognize music scores. Since note heads and stems are the most ubiquitous in a score the recognition of them plays an important role in automatic computer recognition of printed music. Several different approaches have been proposed to solve the recognition problem. Each has its own merits and disadvantages. In this thesis, two effective and efficient methods were proposed - one is based on linear associative memories and the other is based on fuzzy neural networks. In additional to the recognition of heads, the corresponding intervals are also detected. Several scores were used to test these two methods. We obtained correct recognition rates no worse than 86%.

碩士<br>明新科技大學<br>精密機電工程研究所<br>100<br>In recent years, service robots are widely used in difference fields. Entertainment has become an important issue for robot applications. Automatic recognition of music score must immediately offer the music information to the entertainment robot for playing music instruments in real time mode. This study proposes an approach to extract the music information from camera-captured music score. The system uses a monochrome progressive scan camera to capture the image of a printed music score. The resolution of the gigabit Ethernet interface camera is 1380 by 1040 pixels. The distance between the sheet music and the camera is about 80 cm. The proposed method consists of four main modules: the search module, the staff recognition module, the note recognition module, and the MIDI output module. The search module employs the connected component labeling algorithm and the projection method to determine the stave's region. The staff determination module can separate the stave's region into treble stave and bass stave by identifying the position of the horizontal staff lines. The note recognition module recognizes the stem and its related note head, scale, tail or beam by examining the vertical projection profile. The MIDI output module presents the recognition results in a MIDI form music score. Experimental studies are conducted to evaluate the performance of the proposed system and results obtained verify its recognition abilities. The recognition time is less than 4 seconds and the accuracy is bigger than 91%. Keywords：Staff recognition, Note recognition, Vertical characteristic.

Optical Music Recognition (OMR) is the field of computationally reading music notation. This thesis presents, in the form of dissertation by publication, contributions to the theory, resources, and methods of OMR especially for handwritten notation. The main contributions are (1) the Music Notation Graph (MuNG) formalism for describing arbitrarily complex music notation using an oriented graph that can be unambiguously interpreted in terms of musical semantics, (2) the MUSCIMA++ dataset of musical manuscripts with MuNG as ground truth that can be used to train and evaluate OMR systems and subsystems from the image all the way to extracting the musical semantics encoded therein, and (3) a pipeline for performing OMR on musical manuscripts that relies on machine learning both for notation symbol detection and the notation assembly stage, and on properties of the inferred MuNG representation to deterministically extract the musical semantics. While the the OMR pipeline does not perform flawlessly, this is the first OMR system to perform at basic useful tasks over musical semantics extracted from handwritten music notation of arbitrary complexity.