Creative tools: Challenges

Jump to: navigation, search

Develop methodologies to take advantage of MIR for artistic applications in close collaboration with creators. New possibilities for music content manipulation resulting from MIR research have the power to transform music creation. The development of tools for artistic applications can only be done with the involvement of the creators in the whole research and development process.

Develop tools for sound processing based on high-level concepts. New approaches in MIR-related research should provide musicians and sound designers with a high-level content-based processing of sound and music related data. This entails furthering the integration of relevant cognitive models and representations in the creative tools, and also enabling users to implement their own categories by providing them with a direct access to machine learning and automatic classification features.

Enable tools for direct manipulation of sound and musical content. Significant enhancements are required in polyphonic audio analysis methods (e.g. audio-to-score, blind source separation) in order to build applications allowing content-based manipulation of sound. This is expected to have a major impact on professional and end-user markets.

Develop new computer languages for managing temporal processes. This will provide more adapted creative tools, not only for music composition and performance, but more generally for temporal media and interactive multimedia.

Improve integration of audio database management systems in standard audio production tools. These should combine online and offline access to audio materials, and feature content-based search.

Develop real-time MIR tools for performance. Research real-time issues beyond the "faster search engines" in the use of MIR technologies for music performance, addressing the design of specific algorithms and of potential applications in their entirety, in collaboration with the NIME community.

Performance modeling and spatial dimensions. The management of sound and music information in creative tools shall not be limited to basic music categories (such as pitch, intensity and timbre) and must integrate in particular the dimensions of performance modelling and sound space. Beyond direct sound/gesture mapping, the design of new electronic instruments requires a better understanding of the specific structures underlying gesture and performance and their relation to the sound content. As for the spatial dimension of sound, new research advances are expected in the automatic extraction of spatial features for mono- and multichannel recordings and the simulation of virtual acoustic scenes from high-level spatial descriptors, with applications in music production, audiovisual post-production, games and multimedia.

Develop MIR methods for soundscaping. Immersive music environments and virtual soundscaping are growth areas in the creative industries, particularly in relation to physical spaces. Research may involve knowledge gained from collaborations with specialists in building acoustics, architects, and installation artists.

Use artistic sound installation environments as MIR research test grounds. Immersive discovery experiences and physical explorations of music presented as art installations can contribute to a better understanding of the user’s Quality of Experience (QoE); the potential of using sound as an aid to narrative creation and as a non-linguistic means for communication; the use of the the human body as an instigator of the generation of sound; and active engagement of listeners with their environment.

Develop creative tools which include data useful to commerce. Research areas uncovered by consulting commercial and industry practices may include e.g.~sonic branding, personalisation, interactive media environments, social platforms, and marketing tools between artists and fans.

Improve data interoperability between devices An effort is required towards the standardisation of data protocols for a pan-European exchange of music software and hardware modalities. This is especially relevant for music, which is a paradigmatic example of multimodal media, with active communities of developers, working with a rich diversity of devices.

Develop automatic playlist generation and automatic mixing tools for commercial environments. Systems which deliver the appropriate atmosphere for purchase or entertainment require music information research in conjunction with consumer psychology. For example, high level descriptors may be developed to include relationships between music and certain types of product, and music psychology may include field work in commercial environments.