Other exploitation areas

From MIReS

You do not have permission to edit this page, for the following reason:

You can view and copy the source of this page:

<onlyinclude> MIR can be used in settings outside of music distribution and creation, for example in musicology, digital libraries, education and eHealth. In computational musicology, MIR tools have become standard "tools of the trade" for a new generation of empirical musicologists. Likewise, MIR technology is used for content navigation, visualisation, and retrieval in digital music libraries. MIR also shows promise for educational applications, including music appreciation, instrument learning, theory and ear training, although many current applications are still at an experimental stage. eHealth (healthcare practice supported by electronic processes) is also starting to benefit from MIR. Thus, the challenge is to better exploit MIR technologies in order to produce useful applications for other fields of research and practice. For this, current practices and needs from the related communities should be carefully studied. The stakeholders include music professionals, musicologists, music students, music teachers, digital librarians, medical doctors and medical doctors and patients who can benefit from music therapy. </onlyinclude> [[Roadmap#Exploitation perspective|Back to → ''Roadmap:Exploitation perspective'']] __FORCETOC__ == State of the art == We review here the already existing and potential relations between MIR and musicology, digital libraries, education and eHealth, which we identified as particularly relevant for our field of research. ====='''''Applications in musicology'''''===== The use of technology in music research has a long history (e.g. see [Goebl et al., 2008] for a review of measurement techniques in music performance research). Before MIR tools became available, music analysis was often performed with hardware or software created for other purposes, such as audio editors or speech analysis tools. For example, Repp used software to display the time-domain audio signal, and he read the onset times from this display, using audio playback of short segments to resolve uncertainties [Repp, 1992]. This methodology required a large amount of human intervention in order to obtain sufficiently accurate data for the study of performance interpretation, limiting the size and number of studies that could be undertaken. For larger scale and quantitative studies, automatic analysis techniques are necessary. An example application of MIR to music analysis is the beat tracking system BeatRoot [Dixon, 2001], which has been used in studies of expressive timing [Widmer et al., 2003; [Grachten et al., 2009; Flossmann et al., 2009]. The [http://ddmal.music.mcgill.ca/research/salami SALAMI] (Structural Analysis of Large Amounts of Music Information) project is another example of facilitation of large-scale computational musicology through MIR-based tools. A general framework for visualisation and annotation of musical recordings is Sonic Visualiser [Cannam et al., 2010], which has an extensible architecture with analysis algorithms supplied by plug-ins. Such audio analysis systems are becoming part of the standard tools employed by empirical musicologists [Leech-Wilkinson, 2009; Cook, 2004; Cook, 2007], although there are still limitations on the aspects of the music that can be reliably extracted, with details such as tone duration, articulation and the use of the pedals on the piano being considered beyond the scope of current algorithms [McAdams et al., 2004]. Other software such as GRM Acousmographe, IRCAM Audiosculpt [Bogaards et al., 2008], Praat [Boersma and Weenink, 2006] and the [https://www.jyu.fi/hum/laitokset/musiikki/en/research/coe/materials/mirtoolbox MIRtoolbox], which supports the extraction of high-level descriptors suitable for systematic musicology applications, are also commonly used. For analysing musical scores, the ''Humdrum'' toolkit [Huron, 1999] has been used extensively. It is based on the UNIX operating system’s model of providing a large set of simple tools which can be combined to produce arbitrarily complex operations. Recently, ''music21'' [Cuthbert and Ariza, 2010] has provided a more contemporary toolkit, based on the Python programming language. ====='''''Applications in digital library'''''===== A digital library (DL) is a professionally curated collection of digital resources, which might include audio, video, scores and books, usually accessed remotely via a computer network. Digital libraries provide software services for management and access to their content. Music Digital Librarians were among the instigators of the ISMIR community, and the first ISMIR conference (2000) had a strong DL focus. Likewise the contributions from the MIR community to DL conferences (Joint Conference on Digital Libraries, ACM Conference on Digital Libraries, IEEE-CS Conference on Advances in Digital Libraries) were numerous. This could be due to the fact that at the end of 90s, musical libraries moved to digitisation of recordings and to multi-information access (video, score images, and text documents such as biographies and reviews) to create multimedia libraries [e.g. Fingerhut, 1999; Dunn and Isaacson, 2001; McPherson and Bainbridge, 2001]. In this first trend, the technological aspects of these libraries relied mainly on the server, database, media digitisation, text search, and synchronisation (often manual) between media. Today this trend still exists and is accessible online for a wide audience. Examples of this are the "Live TV" of the Cite de la Musique (large audience) with synchronisation of video concerts with libretto, scores and comments. A second trend, that appeared in the mid-2000s, reverses the relationship between Libraries and Music Information Research and Technology. Research and technology enable content estimation, visualisation, search and synchronisation, which are then used in the context of Digital Libraries to improve the usability and access of the multi-documents in libraries (online or not). Examples of this are: inclusion of automatic audio summaries in the IRCAM Library [Mislin et al., 2005], the Bachotheque to compare automatically synchronised interpretations of a same piece [Soulez et al., 2003], optical score recognition and audio alignment for the Bavarian State Library [Damm et al., 2011]. Also, thanks to the development of technologies (Flash, html5, Java-Script), the de-serialisation of media becomes a major theme, along with improved browsing and access to the temporal aspect of media. New concepts of interfaces to enhance listening have been developed which make use of time-based musical annotations ([http://apm.ircam.fr Ecoute augmentee/Increased-listening], or today’s [https://soundcloud.com SoundCloud]). A third trend concerns the aggregation of content and the use of user-generated annotation. The content of dedicated libraries can be aggregated to form meta-libraries (e.g. www.musiquecontemporaine.fr) using the shared protocol OAI-PMH. Content can be distributed over the web or aggregated to a local collection. Using Semantic Web technologies such as Linked Data and ontologies, web content can be re-purposed (e.g. the BBC’s use of the Music Ontology). This trend is also found in new forms of music access (such as Spotify) which aggregate content related to the music item (AMG reviews, wikipedia artist biography). Comparing the suggestions of [Bonardi, 2000] and the observations of [Barthet and Dixon, 2011] a decade later, it is clear that much work is still to be done before MIR technology is fully incorporated into traditional libraries. The European [http://ec.europa.eu/information_society/apps/projects/factsheet/index.cfm?project_ref=250527 ASSETS] project, working with the Europeana multi-lingual European cultural collection, aims to improve search and browsing access to the collection, including multimedia objects. ====='''''Applications in education'''''===== [Dittmar et al., 2012] partitions MIR methods that are utilised in music education into three categories: music transcription, solo and accompaniment track creation, and generation of performance instructions. Systems for music education that exploit MIR technology include video games, music education software (e.g. [http://www.songs2see.com Songs2See]), and music-related apps focused on learning (e.g. [http://www.rockprodigy.com Rock Prodigy]). Regarding instrument learning scenarios, MIR is also seeing an uptake via provision of feedback to learners in the absence of a teacher [Wang and Zhang, 2008], interactive ear training exercises (e.g. the Karajan iPhone apps), automatic accompaniment [Dannenberg and Raphael, 2006], page turning [Arzt et al., 2008] and enhanced listening (e.g. iNotes: Orchestral Performance Companion). Research projects focused on music education include IMUTUS (Interactive Music Tuition System), i-Maestro (Interactive Multimedia Environment for Technology Enhanced Music Education and Creative Collaborative Composition and Performance) and M4M (Musicology for the Masses) – for the latter, a web-based tool called [http://yanno.eecs.qmul.ac.uk/ Yanno], based on automatic chord detection, was proposed for secondary school music classes. Although much work still remains to be done, large-scale experiments have already taken place, such as the IRCAM Music Lab 1 and 2 for the French National Education. Education is one of the most understudied and yet promising application domains for MIR. While Piaget’s constructivism and Papert's constructionism are classics of pedagogy and interaction design relating to children, mashup, remix and recycling of contents might be considered a much more controversial and radical approach, especially for their social, ethical and legal implications. However, it is undeniable that young people are embracing remix en masse, and it is integral to how they make things and express ideas. The cultural practices of mashup and remix brought to school, will force us to rethink the role of teachers as part of this knowledge-building process (Erstad, 2008). The development of learning strategies that support such models of creation represents an ongoing challenge as it defies the current model of schooling, with students taking a more active role in developing knowledge. The introduction of MIR-powered tools for musical education and creation among younger children, combined with recent developments in portable devices, opens a new line of research for suitable novel interfaces and applications. ====='''''Applications in eHealth (healthcare practice supported by electronic processes)'''''===== Use of music information research for eHealth is still in its infancy. Its main use to date has been in music therapy, where it has been employed for quantitative analysis of therapy sessions and selection of musical material appropriate to the user’s ability and taste. Music information technologies have traditionally been used to characterise one’s musical preferences for applications such as music retrieval or recommendation (see for example the Musical Avatar of [Bogdanov et al., 2013]). Moreover, there has been much research on technologies for affective analysis of music, e.g. on music emotion characterisation. These technologies have a great potential for contributing to music therapy, e.g. providing personalised music tools. For instance, according to E. Bigand, advances in cognitive neurosciences of music have revealed the potential importance of music for brain and cognitive stimulation [Bigand, 2012]. At this ISMIR 2012 keynote speech, he referred to some examples of the relationship between music technologies and cognitive stimulation (e.g. [http://www.happy-neuron.com/ "Happy Neuron"] project). Systems making use of MIR for music therapy have already been proposed inside the MIR community, e.g. the work by the team led by [http://www.comp.nus.edu.sg/~wangye/ Ye Wang] at the National University of Singapore [Zhao et al., 2010; Li et al., 2010]. In [Zhao et al., 2010], an MIR system is used to automatically recommend music for users according to their sleep quality in the goal of improving their sleep. In [Li et al., 2010] an MIR system that incorporates tempo, cultural, and beat strength features is proposed to help music therapists to provide appropriate music for gait training for Parkinson’s patients. The Mogat system of [Li et al., 2010] is used to help cochlear implant recipients, especially pre-lingually deafened children. In this system, three musical games on mobile devices are used to train their pitch perception and intonation skills, and a cloud-based web service allows music therapists to monitor and design individual training for the children. ==== '''''References''''' ==== * [Arzt et al., 2008] A. Arzt, G. Widmer, and S. Dixon. Automatic page turning for musicians via real-time machine listening. In ''Proceedings of the 18th European Conference on Artificial Intelligence'', pp. 241-245, Patras, Greece, 2008. * [Barthet and Dixon, 2011] M. Barthet and S. Dixon. Ethnographic observations of musicologists at the British library: Implications for music information retrieval. In ''Proceedings of the 12th International Society for Music Information Retrieval Conference'', Miami, Florida, USA, 2011. * [Bigand, 2012] E. Bigand. Cognitive estimulation with music and new technologies. In ''Proceedings of the 13th International Society for Music Information Retrieval Conference'', Keynote speech, Porto, Portugal, 2012. * [Boersma and Weenink, 2006] P. Boersma and D. Weenink. ''Praat: Doing phonetics by computer'', 2006. * [Bogaards et al., 2008] N. Bogaards, C. Yeh, and J. Burred. Introducing ASAnnotation: A Tool for Sound Analysis and Annotation. In ''Proceedings of the International Computer Music Conference'', Belfast, Northern Ireland, 2008. * [Bogdanov et al., 2013] D. Bogdanov, M. Haro, F. Fuhrmann, A. Xambó, E. Gómez, and P. Herrera. Semantic audio content-based music recommendation and visualization based on user preference examples. ''Information Processing & Management'', 49: 13-33, 2013. * [Bonardi, 2000] A Bonardi. IR for Contemporary Music: What the Musicologist Needs. In ''Proceedings of the International Symposium on Music Information Retrieval'', Plymouth, Massachusetts, USA, 2000. * [Cannam et al., 2010] C. Cannam, C. Landone, and M. Sandler. Sonic Visualiser: an open source application for viewing, analysing, and annotating music audio files. In ''Proceedings of the ACM Multimedia International Conference'', pp. 1467-1468, Florence, Italy, 2010. * [Cook, 2004] N. Cook. Computational and comparative musicology. In: E. Clarke and N. Cook, editors, ''Empirical Musicology: Aims, Methods, and Prospects'', pp. 103-126. Oxford University Press, New York, 2004. * [Cook, 2007] N. Cook. Performance analysis and Chopin's mazurkas. ''Musicae Scientae'', 11(2): 183-205, 2007. * [Cuthbert and Ariza, 2010] M.S. Cuthbert and C. Ariza. music21: A toolkit for computer-aided musicology and symbolic music data. In ''Proceedings of the 11th International Society for Music Information Retrieval Conference'', pp. 637-642, Utrecht, Netherlands, 2010. * [Damm et al., 2011] D Damm, Ch. Fremerey, V. Thomas, and M. Clausen. A Demonstration of the Probado Music System. In ''Proceedings of the 12th International Society for Music Information Retrieval Conference'', Late-breaking session, Miami, Florida, USA, 2011. * [Dannenberg and Raphael, 2006] R.B. Dannenberg and C. Raphael. Music score alignment and computer accompaniment. ''Communications of the ACM'', 49(8): 38-43, 2006. * [Dittmar et al., 2012] C. Dittmar, E. Cano, J. Abesser, and S. Grollmisch. Music information retrieval meets music education. In: M. Müller, M. Goto, and M. Schedl, editors, ''Multimodal Music Processing'', volume 3 of Dagstuhl Follow-Ups, pp. 95-120. Dagstuhl Publishing, 2012. * [Dixon, 2001] S. Dixon. Automatic extraction of tempo and beat from expressive performances. ''Journal of New Music Research'', 30(1): 39-58, 2001. * [Dunn and Isaacson, 2001] Jon W. Dunn and Eric J. Isaacson. Indiana university digital music library project. In ''Proceedings of the ACM/IEEE-CS Joint Conference on Digital Libraries'', 2001. * [Fingerhut, 1999] Michael Fingerhut. The IRCAM Multimedia Library: a Digital Music Library. In ''IEEE Forum on Research and Technology Advances in Digital Libraries (IEEE ADL'99)'', Baltimore, USA, 1999. * [Flossmann et al., 2009] S. Flossmann, W. Goebl, and G. Widmer. Maintaining skill across the life span: Magaloff's entire Chopin at age 77. In ''Proceedings of the International Symposium on Performance Science'', pp. 119-€“124, Auckland, New Zealand, 2009. * [Goebl et al., 2008] W. Goebl, S. Dixon, G. De Poli, A. Friberg, R. Bresin, and G. Widmer. Sense in expressive music performance: data acquisition, computational studies, and models. In P. Polotti and D. Rocchesso, editors, ''Sound to Sense - Sense to Sound: A State of the Art in Sound and Music Computing'', pp. 195-242. Logos Verlag, Berlin, 2008. * [Grachten et al., 2009] M. Grachten, W. Goebl, S. Flossmann, and G. Widmer. Phase-€plane representation and visualization of gestural structure in expressive timing. ''Journal of New Music Research'', 38(2): 183-€“195, 2009. * [Huron, 1999] D. Huron. ''Music Research Using Humdrum: A User's Guide''. Center for Computer Assisted Research in the Humanities. Stanford, California, 1999. * [Leech-Wilkinson, 2009] D. Leech-Wilkinson. ''The Changing Sound of Music: Approaches to Studying Recorded Musical Performance''. CHARM, London, 2009. * [Li et al., 2010] Z. Li, Q. Xiang, J. Hockman, J. Yang, Y. Yi, I. Fujinaga, and Y. Wang. A music search engine for therapeutic gait training. In ''Proceedings of the ACM Multimedia'', pp. 627-630, Florence, Italy, 2010. * [McAdams et al., 2004] S. McAdams, P. Depalle, and E. Clarke. Analyzing musical sound. In: E. Clarke and N. Cook, editors, ''Empirical Musicology: Aims, Methods, and Prospects'', pp. 157-196. Oxford University Press, New York, 2004. * [McPherson and Bainbridge, 2001] John R. McPherson and David Bainbridge. Usage of the MELDEX digital music library. In ''Proceedings of the 2nd International Symposium on Music Information Retrieval'', Bloomington, Indiana, USA, 2001. * [Mislin et al., 2005] F. Mislin, Michael Fingerhut, and Geoffroy Peeters. ''Automatisation de la production et de la mise en ligne de resumes sonores''. Master's thesis, ISTY, 2005. * [Repp, 1992] B.H. Repp. Diversity and commonality in music performance: An analysis of timing microstructure in Schumann's "Träumerei". ''Journal of the Acoustical Society of America'', 95(5): 2546-2568, 1992. * [Soulez et al., 2003] Ferréol Soulez, Xavier Rodet, and Diemo Schwarz. Improving polyphonic and poly-instrumental music to score alignment. In ''Proceedings of the 4th International Conference on Music Information Retrieval'', Baltimore, Maryland, USA, 2003. * [Wang and Zhang, 2008] Y. Wang and B. Zhang. Application-specific music transcription for tutoring. ''IEEE MultiMedia'', 15(3): 70-€“74, 2008. * [Widmer et al., 2003] G. Widmer, S. Dixon, W. Goebl, E. Pampalk, and A. Tobudic. In search of the Horowitz factor. ''AI Magazine'', 24(3): 111-130, 2003. * [Zhao et al., 2010] Wei Zhao, , Xinxi Wang, and Ye Wang. Automated sleep quality measurement using EEG signal. In ''Proceedings of the ACM Multimedia'', Florence, Italy, 2010. == '''[[Other exploitation areas: Challenges|Challenges]]''' == {{:Other exploitation areas: Challenges}} <!--[[Other exploitation areas: Challenges|'''Follow the discussion and leave your comment here...''']]--> [[Roadmap#Exploitation perspective|Back to → ''Roadmap:Exploitation perspective'']]

Return to Other exploitation areas.