Knowledge-driven methodologies

From MIReS

You do not have permission to edit this page, for the following reason:

You can view and copy the source of this page:

<onlyinclude> For a long time, the MIR community has been focusing on a range of bottom-up approaches, addressing the kinds of data we use and the types of algorithms we apply to it. A major challenge is to complement this focus and explore other methodologies and fields of science which approach music in a more integrated way. After all, music information research is just one of many sciences that centre on and care about music, which include musicology, psychology, sociology and neuroscience. Over decades of research, each of these fields has aggregated knowledge concerning music which can inform the process of music information research. The focus here is on gaining domain knowledge from outside of MIR as opposed to borrowing methodologies or algorithms. This will require that researchers from different disciplines engage in a dialogue on all aspects of music. The potential impact is that all participating disciplines benefit from the diverse and differing views on the phenomenon of music, in all its aspects and forms. <!-- Our community has for a long time been focusing on a range of bottom-up approaches: starting with the kinds of data we use (see [[Musically relevant data]] and [[Music representations]]) to the types of algorithms we apply to it (see [[Data processing methodologies]]). This chapter is about transcending this focus and exploring other methodologies and fields of science which approach music in a more integral way. This includes musicology, psychology, sociology, neuroscience and human computer interaction. We ask ourselves what additional information about the process of music information research we can gain from these fields of science and how we can make the most of it. The focus is here on gaining domain knowledge from outside of MIR as opposed to borrowing methodologies or algorithms as discussed in [[Data processing methodologies]]. --> </onlyinclude> [[Roadmap#Technological perspective|Back to → ''Roadmap:Technological perspective'']] __FORCETOC__ == State of the art == In what follows, we briefly review the already existing and potential relations between MIR and musicology, psychology, sociology and neuroscience, which we identified as particularly relevant for our field of research. ====='''''Musicology'''''===== Musicology is fundamental to MIR, and building bridges between disciplines is at the very core of research in musicology [Dahlig-Turek et al., 2012]. Musicologists have taken an active role in the ISMIR community, for instance, musicology has always been considered a key topic in the ISMIR call for papers (see, e.g. research areas related to computational musicology, computational ethnomusicology explicitly considered at [http://ismir2012.ismir.net/authors/call-for-participation ISMIR 2012]). Moreover, the conference on Interdisciplinary Musicology ([http://www.uni-graz.at/~parncutt/cim CIM]) has included papers on computational modelling in the program, and there is a special edition of this conference on the topic of "Technology" that is planned for [http://www.sim.spk-berlin.de/cim14_919.html CIM 2014]. There are also some relevant journals in this intersection (e.g. [http://www.tandfonline.com/loi/tmam20 Journal of Mathematics and Music]) and the Special Issue on Computational Ethnomusicology in the [http://www.tandfonline.com/loi/nnmr20 Journal of New Music Research]. An overview on the relationship between MIR and musicology is provided in the Musicology [http://ismir2011.ismir.net/tutorials/ISMIR2011-Tutorial-Musicology.pdf tutorial] presented at ISMIR 2011 by [Volk and Wiering 2011], and a guide to the use of MIR technology in musicology is given in [Leech-Wilkinson, 2009]. Although musicological studies in MIR have traditionally focused on the symbolic domain, recent developments in music transcription and feature extraction technologies from audio signals have opened new research paths at the intersection of musicology and signal processing. Key research topics in this area have been, among others, melodic similarity, key estimation and chord tracking. Musicological and MIR research have been contrasted [http://ismir2011.ismir.net/tutorials/ISMIR2011-Tutorial-Musicology.pdf] in terms of, among others, data sources, repertoires and methodologies, and some opportunities for future research have been pointed out. MIR technologies can contribute with tools and data that are useful for musicological purposes, and Musicology can provide relevant research problems and use cases that can be addressed through MIR technologies. A mutual influence is starting to take place, although there is still a need for more collaboration between musicologists and technicians to create a truly interdisciplinary research area and contribute with truly music-rooted models and technologies. Only by this collaboration can we address the current gap between feature extractors and expert analyses and make significant contributions to existing application needs, e.g. version identification, plagiarism detection, music recommendation, and to study how the relationship between people and music changes with the use of technology (e.g. [http://www.elec.qmul.ac.uk/digitalmusic/m4m/ "Musicology for the Masses"] project). <!-- Musicology, understood as the academic study of music [Harper-Scott & Samson 2009], is a main discipline in Music Information Research, music being our main object of study. For that reason, musicologists have taken an active role in the ISMIR community. For instance, the 2010 ISMIR Edition was hosted by a musicology department (http://ismir2010.ismir.net) and musicology has been considered as a key topic in the ISMIR call for papers (see, e.g. research areas related to computational musicology, computational ethnomusicology explicitly considered at http://ismir2012.ismir.net/authors/call-for-participation). Moreover, the conference on Interdisciplinary Musicology (CIM http://www.uni-graz.at/~parncutt/cim) has included papers on computational modeling in the program, and there is a special edition of this conference on the topic of “Technology” that is planned for 2014 http://www.sim.spk-berlin.de/cim14_919.html). There are also some relevant journals in this intersection (e.g. Journal of Mathematics and Music http://www.tandfonline.com/toc/tmam20) and the Special Issue of Computational Ethnomusicology at the Journal of New Music Research (http://www.tandfonline.com/toc/nnmr20). An overview on the relationship between MIR and musicology is provided in the Musicology tutorial presented at ISMIR 2011 [Volk & Wiering, 2011]). Although musicological studies in the ISMIR area have traditionally focused on the symbolic domain (section [[Musically relevant data]]), recent developments in music transcription and feature extraction technologies from audio signals have opened new research paths on the intersection of musicology and signal processing. Key research topics in this area have been, among others, melodic similarity, key estimation and chord tracking. [Volk & Wiering 2011] contrasted (p. 46) musicological and MIR research in terms of, among others, data sources, repertoires and methodologies, and pointed out some opportunities for future research. MIR technologies can contribute with tools and data that are useful for musicological purposes, and Musicology can provide relevant research problems and use cases that can be addressed through MIR technologies. A mutual influence is starting to take place, although there is still a need for more collaboration between musicologists and technicians to create a truly interdisciplinary research area and contribute with truly music-rooted models and technologies. Only by this collaboration we can address the current gap between feature extractors and expert analyses and make significant contributions to existing application needs, e.g. version identification, plagiarisim detection [Cook & Sapp 2009], music recommendation, and to study how the relationship between people and music changes with the use of technology (e.g. “Musicology for the masses” project http://www.elec.qmul.ac.uk/digitalmusic/m4m/). --> ====='''''Psychology of music'''''===== Music is created and experienced by humans, and the ultimate goal of MIR is to produce results that are helpful and interesting for humans. Therefore it is only natural to care about how humans perceive and create music. Music psychology tries to explain both musical behavior and musical experience with psychological methods. Its main instrument therefore is careful experimentation involving human subjects engaged in some kind of musical activity. Research areas span the whole spectrum from perception to musical interaction in large groups. Research questions concern the perception of sound or sound patterns, as well as perception of more musically meaningful concepts like harmony, pitch, rhythm, melody and tonality. The emotions associated with personal music experience are a part of music psychology, as are personal musical preferences and how they are influenced through peer groups and family, and musical behaviors from dancing to instrument playing to the most sophisticated interaction within whole orchestras. Therefore music psychology should be able to provide valuable knowledge for MIR researchers in a whole range of sub-fields. Indeed there already is a certain exchange of knowledge between music psychology and MIR. Just to give a few examples, Carol L. Krumhansl, an eminent figure in music psychology, was an invited speaker at the 11th International Society for Music Information Retrieval Conference ([http://ismir2010.ismir.net/ ISMIR 2010]), in Utrecht, Netherlands talking about "Music and Cognition: Links at Many Levels". Her monograph on "cognitive foundations of musical pitch" [Krumhansl, 1990] is still seen as one of the standard texts on the subject. Gerhard Widmer, who has been an important contributor to MIR early on, was a keynote speaker at the 12th International Conference on Music Perception and Cognition ([http://icmpc-escom2012.web.auth.gr/ ICMPC]), which is one of the most important conferences in the field of music psychology. At last year's major conference in the MIR field ([http://ismir2012.ismir.net/ ISMIR 2012]) there was a joint presentation of an MIR researcher and a psychologist elaborating on the sometimes complicated dialog of the two disciplines [Aucouturier and Bigand, 2012, 2013]. <!-- Music is created and experienced by humans, and the ultimate goal of MIR is to produce results that are helpful and interesting for humans. Therefore it is only natural to care about how humans perceive and create music. Music psychology tries to explain both musical behavior and musical experience with psychological methods. Its main instrument therefore is careful experimentation involving human subjects engaged in some kind of musical activity. Research areas span the whole spectrum from perception to musical interaction in large groups. Research questions concern the perception of sound or sound patterns, as well as perception of more musically meaningful concepts like harmony, pitch, rhythm, melody and tonality. The emotions associated with personal music experience are a part of music psychology, as are personal musical preferences and how they are influenced through peer groups and family, and musical behaviors from dancing to instrument playing to the most sophisticated interaction within whole orchestras. Therefore music psychology should be able to provide valuable knowledge for MIR researchers in a whole range of sub-fields. Indeed there already is a certain exchange of knowledge between music psychology and MIR. Just to give a few examples, Carol L. Krumhansl, an eminent figure in music psychology, was an invited speaker at the Eleventh International Society for Music Information Retrieval Conference (ISMIR 2010), in Utrecht, Netherlands (http://ismir2010.ismir.net/) talking about "Music and Cognition: Links at Many Levels". Her monograph on "cognitive foundations of musical pitch" [Krumhansl 1990] is still seen as one of the standard texts on the subject. Gerhard Widmer, who has been an important contributor to MIR right from its start, will be a keynote speaker at the "12th International Conference on Music Perception and Cognition (ICMPC)" (http://icmpc-escom2012.web.auth.gr/), which is one of the most important conferences in the field of music psychology. And, last but not least, there are MIR researchers who contribute to both music psychology and MIR literature (e.g. Simon Dixon, http://www.eecs.qmul.ac.uk/~simond/). --> ====='''''Sociology of music'''''===== Social psychology and the sociology of music focus on individuals as members of groups and on how groups and shared cultural codes influence music-related attitudes and activities. This point of view allows one to ask and answer important questions like: How do individuals and groups use music? How is the collective production of music made possible? How does music relate to broader social distinctions, especially class, race, and gender? Although it is evident that such a sociology of music should be able to provide important insights not only for the field of MIR, many authors have suggested that research over recent decades has largely ignored the social functions of music at the expense of its cognitive and emotional functions (see e.g. [Hargreaves and North, 1997]). [Hargreaves and North, 1999] concluded that music serves three social functions: it is used by individuals to help manage their moods, self-identity [DeNora, 2000], and interpersonal relationships. [North et al., 2000] elaborated this idea, showing that a sample of 13- to 14-year-olds listened to music to portray a social image to others, and to fulfil their emotional needs. Similarly, [Tarrant et al., 2000] showed that American and English adolescents listened to music to satisfy both emotional and social needs, as well as for reasons of self-actualisation. [Lonsdale and North, 2011] remarked that listening to music was "a social activity", which offered an opportunity for participants "to socialise with friends" (e.g., dancing, sharing live music). Music has a stronger social component for teenagers and young people than for seniors but it still keeps some powers to strengthen social bonds and to provide memory aids when brain functions decline. In this respect, life-span and elderly-centred applications are yet to be fully explored and exploited [Magee et al., 2011]. How MIR can benefit from these and other results concerning the sociology of music is still a largely open question which opens up new and promising areas of research. <!-- Social psychology and the sociology of music focus on individuals as members of groups and on how groups and shared cultural codes influence music-related attitudes and activities. This point of view allows one to ask and answer important questions like: How do individuals and groups use music? How is the collective production of music made possible? How does music relate to broader social distinctions, especially class, race, and gender? Although it is evident that such a sociology of music should be able to provide important insights not only for the field of MIR, many authors have suggested that research over recent decades has largely ignored the social functions of music at the expense of its cognitive and emotional functions (see e.g. [Hargreaves & North 1997]). [Hargreaves & North 1999] concluded that music serves three social functions: it is used by individuals to help manage their moods, self-identity [DeNora 2000], and interpersonal relationships. [North et al. 2000] supported this idea, showing that a sample of 13- to 14-year-olds listened to music to portray a social image to others, and to fulfil their emotional needs. Similarly, [Tarrant et al 2000] showed that American and English adolescents listened to music to satisfy both emotional and social needs, as well as for reasons of self-actualization. [Londsdale & North 2011] remarked that listening to music was "a social activity", which offered an opportunity for participants "to socialize with friends" (e.g., dancing, sharing live music). Even though music has a stronger social component for teenagers and young people than for seniors, its powers to strengthen social bonds and provide memory aids when brain functions decline are yet to be explored and exploited. How MIR can benefit from these and other results concerning the sociology of music is still a largely open question which opens up new and promising areas of research. --> ====='''''Neuroscience'''''===== All music psychological questions raised above could of course also be examined with neuroscientific methods. Instead of measuring the subject's behavior in music psychological experiments or directly asking subjects about their experiences concerning music it is possible to measure various signals from the human brain during such experiments. Possible signals range from electro-encephalography (EEG) to magneto-encephalography (MEG) or functional magnetic resonance imaging (fMRI). Each of the signals has its own characteristic strengths and weaknesses. E.g. EEG has a very good temporal but poor spatial resolution where fMRI is just the opposite. No matter what brain signals are being used, the fundamental question is always what parts of the brain contribute in what way to a subject's experience or creation of music. It is not immediately clear what MIR could gain from such a knowledge about brain structures involved in perception and production of music that could go beyond knowledge obtained from psychological experiments not utilising neuroscientific methods. The biggest contribution might concern problems where humans have difficulty self-assessing their performance and experience. One example is the experience of emotions when listening to music. Neuroscientific methods might be able to provide a more quantitative and maybe more accurate picture than human self-assessment (see e.g. [Blood and Zatoore, 2001], [Schmidt and Trainor, 2001]). Differences in brain structure and function between skilled musicians and non-musicians is another well researched subject (see e.g. [Gaser and Schlaug, 2003], [Krings et al., 1999]). The same holds for the study of the neuronal processes during performance of music where the sensorimotor interplay is at the center of interest (see [Zatorre et al., 2007] for a recent review). <!-- All music psychological questions raised above could of course also be examined with neuroscientific methods. Instead of measuring the subject's behavior in music psychological experiments or directly asking subjects about their experiences concerning music it is possible to measure various signals from the human brain during such experiments. Possible signals range from electro-encephalography (EEG) to magneto-encephalography (MEG) or functional magnetic resonance imaging (fMRI). Each of the signals has their own characteristic strengths and weaknesses. E.g. EEG has a very good temporal but poor spatial resolution where fMRI is just the opposite. No matter what brain signals are being used, the fundamental question is always what parts of the brain contribute in what way to a subject's experience or creation of music. It is not immediately clear what MIR could gain from such a knowledge about brain structures involved in perception and production of music that could go beyond knowledge obtained from psychological experiments not utilizing neuroscientific methods. The biggest contribution might concern problems where humans have a hard time self-assessing their performance and experience. One example is the experience of emotions when listening to music. Neuroscientific methods might be able to provide a more quantitative and maybe more accurate picture than human self-assessment (see e.g. [Blood & Zatorre 2001], [Schmidt & Trainor 2001]). Differences in brain structure and function between skilled musicians and non-musicians is another well researched subject (see e.g. [Gaser & Schlaug 2003], [Krings et al. 1999]). The same holds for the study of the neuronal processes during performance of music where the sensorimotor interplay is at the center of interest (see [Zatorre et al. 2007] for a recent review). ====='''''Human computer interaction / interfaces'''''===== The Association for Computing Machinery defines human-computer interaction (HCI) as "a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them." (ACM SIGCHI Curricula for Human-Computer Interaction: http://old.sigchi.org/cdg/cdg2.html#2_1). HCI involves the study, planning, and design of the interaction between people (users) and computers. It is often regarded as the intersection of computer science, behavioral sciences, design and several other fields of study. Interaction between users and computers occurs at the interface, which includes both software and hardware. The basic and initial goal of HCI is to improve the interactions between users and computers by making computers more usable and receptive to the user's needs. In that sense, it is not surprising that for decades, HCI has mostly focused on making interaction more efficient, apparently not discovering until quite recently that not only efficiency is relevant, but perhaps also beauty and fun, and that the same perception or experience of interaction can have intrinsic values per se, independently of the outcomes it may produce [e.g. Norman 2004; McCarthy and Wright 2004]. Since, as we have already repeatedly stated, the human components of MIR are highly relevant, both from cultural, psychological or physiological perspectives, it comes as no surprise that MIR could strongly benefit from knowledge inherited from HCI and other related disciplines such as User Experience (UX), or Interface and Interaction Design studies. These methodologies could bring important benefits not only to the conception of MIR systems at earlier design stages, but also for the evaluation and subsequent iterative refinement of these systems. In that sense, whereas the evaluation of MIR systems conceived for providing univocally correct answers (e.g finding or identifying a known target song) seems quite straightforward, more open systems, with more open and thus more subjective outcomes, the inclusion of more subjective aspects such as the users’ emotions, perceptions and internal states [e.g. Hekkert 2006], the effort done by users in order to accomplish an assigned action, or the context or environment within which the interaction occurs [e.g. Hassenzahl & Tractinsky 2006], could also be taken into account2. Furthermore, beyond the evaluation of User Experience, another MIR component that could directly benefit from HCI-related knowledge would be the application of SOA interface and interaction technologies in the creation of MIR systems and tools. This topic is covered in more detail in section [[Interface and interaction aspects]]. --> ====='''''References'''''===== * [Aucouturier and Bigand, 2012] Jean-Julien Aucouturier and Emmanuel Bigand. Mel Cepstrum and Ann Ova: The Difficult Dialog Between MIR and Music Cognition. In: Fabien Gouyon, Perfecto Herrera, Luis Gustavo Martins, and Meinard Müller, editors, ''ISMIR 2012'', pp. 397-402. FEUP Edições, 2012. * [Aucouturier and Bigand, 2013] Jean-Julien Aucouturier and Emmanuel Bigand. Seven problems that keep MIR from attracting the interest of the natural sciences. ''Journal of Intelligent Information Systems'', in print, 2013. * [Blood and Zatoore, 2001] A.J. Blood, R.J. Zatorre. Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion. In ''Proceedings of the National Academy of Sciences of the United States of America'' 98(20): 11818-11823, 2001. * [Dahlig-Turek et al., 2012] R. Parncutt, F. Wiering, E. Dahlig-Turek, S. Klotz, editor. Musicology (Re-) Mapped. Standing Committee for the Humanities. European Science Foundation, 2012. * [DeNora, 2000] T. DeNora. Music as a Technology of Self. ''Poetics'', 27: 31-56, Elsevier, 1999. * [Gaser and Schlaug, 2003] C. Gaser and G. Schlaug. Brain structures differ between musicians and non-musicians. ''The Journal of Neuroscience'', 23(27): 9240-9245, 2003. * [Hargreaves and North, 1997] D.J. Hargreaves, A.C. North, A. C. ''The social psychology of music''. Oxford University Press, 1997. * [Hargreaves and North, 1999] D.J. Hargreaves, A.C. North. The functions of music in everyday life: Redefining the social in music psychology. ''Psychology of Music'', 27: 71-83, 1999. * [Krings et al., 1999] T. Krings, R. Topper, H. Foltys, S. Erberich, R. Sparing, K. Willmes, A. Thron. Cortical activation patterns during complex motor tasks in piano players and control subjects. A functional magnetic resonance imaging study. ''Neuroscience Letters'', 278 (3): 189-193, 1999. * [Krumhansl, 1990] C.L. Krumhansl. ''Cognitive Foundations of Musical Pitch''. Oxford University Press, USA, 1990. * [Leech-Wilkinson, 2009] D. Leech-Wilkinson. [http://www.charm.kcl.ac.uk/studies/chapters/intro.html ''The changing sound of music: Approaches to studying recorded musical performance''], CHARM project, London, 2009. * [Lonsdale and North, 2011] A.J. Londsdale, A.C. North, Why do we listen to music? A uses and gratifications analysis. ''British Journal of Psychology'', 102: 108-134, 2011. * [Magee et al., 2011] Wendy L. Magee, Michael Bertolami, Lorrie Kubicek, Marcia LaJoie, Lisa Martino, Adam Sankowski, Jennifer Townsend, Annette M. Whitehead-Pleaux, and Julie Buras Zigo. Using music technology in music therapy with populations across the life span in medical and educational programs. ''Music and Medicine'', 3(3): 146-153, 2011. * [North et al., 2000] A.C. North, D.J. Hargreaves, S.A. O'Neill. The importance of music to adolescents. ''British Journal of Educational Psychology'', 70(2): 255-272, 2000. * [Schmidt and Trainor, 2001] L.A. Schmidt, L.J. Trainor. Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. ''Cognition and Emotion'' 15(4): 487-500, 2001. * [Tarrant et al., 2000] M. Tarrant, A.C. North, D.J. Hargreaves, English and American adolescents' reasons for listening to music. ''Psychology of Music'', 28: 166-173, 2000. * [Zatorre et al., 2007] R.J. Zatorre, J.L. Chen, V.B. Penhune. When the brain plays music. Auditory-motor interactions in music perception and production. ''Nature Reviews Neuroscience'', 8: 547-558, 2007. <!-- * [Cook and Saap, 2009] N. Cook & C. Sapp. Purely coincidental? Joyce Hatto and Chopin's Mazurkas, [http://www.charm.kcl.ac.uk/projects/p2_3_2.html#d3419e8182 online resource]. * [Harper-Scott and Samson, 2009] . Harper-Scott, J. & Samson, J. (ed.) An Introduction to Music Studies., Cambridge and New York : Cambridge University Press, 2009. * [Hassenzahl and Tractinsky, 2006] Hassenzahl, M., Tractinsky, N. (2006) “User experience – a research agenda”. Behaviour & Information Technology. Volume 5, Issue 2. * [Hekkert, 2006] Hekkert, P (2006). “Design aesthetics: Principles of pleasure in product design”. Psychology Science, 48(2), 157-172 (2006). * [McCarthy and Wright, 2004] McCarthy, J. and Wright, P. (2004) “Technology as experience”, Interactions, Vol. 11, No. 5, 42-43. * [Norman, 2004] Norman, D. (2004). Emotional Design: Why We Love (Or Hate) Everyday Things, Basic Books. * [Volk and Wiering, 2011] A. Volk, F. Wiering. Musicology. Tutorial, ISMIR 2011. --> =='''[[Knowledge-driven methodologies: Challenges|Challenges]]'''== {{:Knowledge-driven methodologies: Challenges}} <!--[[Knowledge-driven methodologies: Challenges|'''Follow the discussion and leave your comment here...''']]--> [[Roadmap#Technological perspective|Back to → ''Roadmap:Technological perspective'']]

Return to Knowledge-driven methodologies.