Mutations (megamix)

Exploring Notions of the “DJ set”, “Mashup” and “Remix” through Live Piano-based Performance

by Robert Ratcliffe, Jon Weinel and Zubin Kanga

Mutations (megamix) is an interactive work for piano and electronics, which explores notions of the “DJ set”, “mashup” 1[1. The term “mashup” is used within popular music to denote a specific compositional technique in which two or more existing records are superimposed to create a new track. This often involves the use of vocal a capella material, which is layered against other recordings. One of the most commonly cited examples is Freak Like Me byRichard X and the Sugababes (2002), which combines Adina Howard’s Freak Like Me with Are Friends Electric? by Gary Numan and the Tubeway Army.] and “remix” through live performance. Conceived by Robert Ratcliffe, this project has been developed through a variety of different approaches to performance, the most recent of which utilises a custom-designed software application constructed within Max/MSP. 2[2. This patch and further documentation are available from Weinel’s website.] The following article will trace the development of Mutations (megamix), featuring a description of the various components and compositional techniques employed, and a technical explanation of the custom electronics used to facilitate the performance. There will also be a consideration of performance practicalities and issues relating to the realisation of the piece, together with an evaluation (in terms of the æsthetic and technical goals), and the consequences for future work. The article is intended to stimulate thought for creative practitioners interested in the field of musical hybridism, and specifically hybrid performance practice which combines traditional components with elements sourced from electronic dance music 3[3. Electronic dance music is a collective term used to denote rhythmic-driven, dance floor-oriented styles of electronic music, such as drum ‘n’ bass, dubstep, house and techno.] and DJ culture. In addition, it is hoped that the article will highlight the practical advantages of custom-designed software to aid the realisation and accessibility of a performance.

Concept

Mutations (megamix) is an interactive work by Robert Ratcliffe, exploring notions of the DJ set, mashup and remix through the integration of various streams 4[4. Use of the term “streams” in this article reflects the blending of different types of source material in a manner comparable to the blending of two or more audio sources (for example, two record players) by a DJ, as commonly encountered in DJ culture.] of piano-based material in live performance. Integrating three core components: live piano (notated and improvised material), Disklavier / MIDI playback 5[5. Although the MIDI material was conceived and recorded using a Yamaha DC3E Disklavier, an alternative MIDI-controlled piano (for example, Bösendorfer’s CEUS) could be used for performances where a Disklavier is unavailable. Note: the MIDI-controlled system is separate from the piano used for the pianist’s live improvised performance.] and soundfile playback (with or without live electronics), Mutations (megamix) has the capacity to be reconfigured for various performance situations, with the components selected according to a realisation determined by the performer(s). 6[6. A realisation of the piece may involve one or two performers (pianist, electronic musician or both), depending on the configuration chosen for performance.]

Incorporating human and machine-generated material, a realisation of the piece involves the management of a pool of audio files, MIDI files, and score fragments, which are drawn upon during performance. In this way, the performers are required to control and shape the various streams of material in the same way that a DJ would select and combine records during the structuring of a live set.

The supply of audio files, MIDI files and score fragments used in the construction of the piece takes existing works from the piano repertoire as source material, both transformed and quoted intact, resulting in a spectrum of recognisability ranging from the easily identifiable to the ambiguous to the non-referential. The integration of this borrowed material within the three strands of the piece highlights various connections between traditional forms of musical borrowing, transformative imitation, improvisation, electroacoustic sound transformation and quotation, electronic dance music (EDM) sampling practices, remix practices and DJ performance.

Through the course of this project, a variety of methods have been trialled in order to achieve the combination of these three streams of sonic material in a live performance scenario. These have included at various stages: a range of live performance and recorded clips of Disklavier and piano, and the manipulation of these streams using MIDI controllers and Ableton Live. The most recent version of Mutations (megamix) utilises a specialised software patch developed in collaboration with Jon Weinel. The purpose of creating this software was to streamline the project into a neat and more manageable package. Although the patch exploits principles that can be achieved with other software applications (such as Ableton Live), the advantage is to enable the performer to almost instantaneously load the required audio material and perform with it in the way the composer intends, through careful attention to the design and usability of the graphical user interface. The result is intended to supply the most appropriate means to technically achieve Ratcliffe’s creative aims with the project, in a way that provides a suitable level of accessibility for the propagation of Mutations (megamix) in a variety of performance scenarios.

The Recombinant Aesthetic of EDM

Mutations (megamix) is a product of Ratcliffe’s ongoing compositional research in the area of musical hybridism, and in particular, the cross-fertilisation of art music and EDM. 7[7. Further examples of Ratcliffe’s hybrid compositional work are available on his MySpace and SoundCloud pages.] Mutations (megamix) was conceived as Ratcliffe explored different ways of mixing classical repertoire in performance 8[8. This idea was originally tested by Ratcliffe through the concept of the “mixed” recital programme (Ratcliffe 2008). This involved the performance of a continuous recital programme for solo piano, in which constituent works were integrated through techniques such as juxtaposition and superimposition, or linked through improvised transitional passages.], in a similar way to DJs who combine records during the construction of a set. In this capacity records are often used as raw material to be creatively deconstructed, edited and recombined during performance to produce original music. 9[9. Originality in some forms of EDM has more to do with a freshness, or an idiosyncratic or novel arrangement of existing materials.] This is a common æsthetic goal of the EDM DJ, producing what is known as the “third record” through the combination and manipulation of multiple elements in performance (Butler 2006, 94):

… the real goal of the DJ, or a good DJ, would be to create what they call the “third record”, and make it their own, by either scratching or manipulating songs that they’re using, or just basically sampling themselves… (DJ Impact quoted in Butler 2006, 243)

Within some EDM genres, tracks are conceived with this kind of recombination and transformation in mind, with records designed to function as modular components within the larger structure of the DJ set, as opposed to individual compositions. The form of such tracks will usually follow an established structural template that allows for the combination of multiple records in performance. At a basic level, this may involve the composition of an introductory percussion passage of expected duration (for example, 16 bars) to aid in the process of beat matching and the manual synchronisation of records, but may also extend to the design of larger structural components to facilitate more complex modes of integration. However, records that are not intended to function in this way may also be effectively combined through the use of specific transformational techniques, such as the application of live filtering or through the use of an alternative mixing approach (for example, eclectic cut ‘n’ paste mixing derived from the æsthetic of hip hop 10[10. For more on the compositional techniques of the hip hop DJ, see Smith 2000.]). Within this tradition, DJs have developed strategies for combining records that differ in tempo, metre, structure, instrumentation and stylistic content, but what is common throughout the various mixing approaches is that combined materials work together to produce a coherent musical result. By replacing EDM records with piano-based material derived from the classical repertoire, Mutations (megamix) explores whether the coherent attributes of DJ performance (æsthetics and performance practice) can be extracted as compositional paradigms and translated to other musical forms. 11[11. Although Mutations (megamix) uses piano-based material, this concept could be explored further using other musical instruments.]

Performance Materials

Performance materials for Mutations (megamix) consist of performance instructions (guidance notes) and a modular score containing notated fragments from the piano repertoire. These written elements are complemented by a collection of MIDI and audio files 12[12. It should be noted that the default bpm of the various audio and MIDI loops is considerably varied, requiring manual modification by the performer/s in order to create tempo-synchronised effects.], which the performer/s must trigger and integrate with the score fragments to produce a coherent performance. Although a set of general instructions is provided, these directions are not prescriptive and most decisions are left at the discretion of the performer/s, in keeping with the open form and improvisatory nature of the DJ set. However, there should always be a definite aim to integrate the components, as opposed to a random selection of elements. This requires the performer to make genuine choices, like a DJ who responds to their environment.

The first decision is to choose an appropriate configuration for performance, as the modular nature of Mutations (megamix) allows for the three components of the piece to be reconfigured in accordance with the performance situation. Below is a list of possible configurations that may be selected according to a realisation determined by the performer(s):

Individual components:

• Improvised 13[13. If required, a through-composed version of each component may be prepared in advance by the performer/s.] selection and manipulation of MIDI files;
• Improvised selection and manipulation of soundfiles;
• Improvised piano performance, whereby notated score fragments are chosen “on-the-fly”.

Possible configurations for performance:

• MIDI files + Soundfiles + Live piano
• MIDI files + Live piano
• MIDI files + Soundfiles
• Soundfiles + Live piano

MIDI

Mutations (megamix) was preceded by (and derived from) Mutations (Audio 1), which is a “fixed” work for Disklavier constructed through the transformation of existing material from the piano repertoire. During the performance of Mutations, “borrowed” material is assimilated within a hybrid musical language equally influenced by the rhythmic innovations of Stravinsky, the experimentation of Nancarrow’s player piano studies, the evolutionary processes of minimalism and the angular riffs and sequences of EDM genres, such as rave and techno. The various source works that were drawn from, together with the corresponding location of transformed material within Mutations can be seen below:

Upon completion, Mutations acted as a source of material for Mutations (megamix), with the MIDI information used to drive the Disklavier separated into individual loops, providing a pool of MIDI clips that could be triggered and manipulated during performance. This was initially realised through the use of a specially designed performance system within Ableton Live 15[15. Ratcliffe has used this system to generate a number of live improvised remixes and mashups, including Live Mashup [Excerpt 1] (Ratcliffe 2011) available on the Furthernoise compilation, Explorations in Sound, Vol. 4: The Sound of Live Performance.] to facilitate real-time mixing and manipulation of MIDI and audio. It is important to note, however, that whilst early performances of Mutations (megamix) were realised using Ableton Live, the use of this software is purely illustrative, and a similar performance system could be constructed using other sequencer platforms or custom-designed software (as demonstrated in the Max/MSP version).

Soundfiles

The supply of MIDI loops is accompanied by a collection of audio files, which were extracted from recorded piano performances given by Ratcliffe in December 2008. These recordings were made in the Keele University chapel using a Steinway grand, Logic Audio and a variety of microphones. The performances were captured by Andy Willy and Steve Bird, with the separate recorded signals blended together by Ratcliffe during the mixing stage to produce the desired sound.

Recorded works in the audio files for Mutations (megamix) include:

• Chopin — Berceuse, Op .57
• Chopin — Nocturne in D-flat major, Op. 27 No. 2
• Chopin — Nocturne in E-flat major, Op. 9 No. 2
• Debussy — Arabesque No. 1
• Debussy — Prelude from Suite Bergamasque
• Liszt — Consolation No. 3 in D-flat major
• Schumann — Des Abends, No. 1 from Phantasiestücke, Op. 12

As with Mutations, each of these works was separated into a collection of smaller loops that could be triggered and mixed live during performance. The content of these works provides a contrast to the mechanistic material generated by the Disklavier, allowing the performer(s) to navigate between human- and machine-produced musical expression.

Score Fragments

The score for Mutations (megamix) consists of “borrowed” score fragments sourced from the classical piano repertoire, which are presented as separate notated modules. 16[16. Modular works, such as Terry Riley’s In C, and the notation of open form structures, such as Stockhausen’s Klavierstucke XI, and the Third Piano Sonata of Boulez were influential in this respect.] These fragments may be looped or chained together to form different patterns, with the selection and ordering of modules left at the discretion of the performer, who chooses freely between the fragments, making selections in the same way that a DJ chooses records during the construction of a set. In addition, the score fragments may be subjected to basic transformational techniques, such as augmentation, diminution, inversion, retrograde and transposition, or used as a starting point for additional improvised material. 17[17. Improvised material may also be created in response to the MIDI and audio components.] Finally, the performer(s) may add additional score fragments to the pool of available material.

As with the open form of the DJ set, the duration and structure of the work are to be determined in performance. However, there are certain characteristic paradigms that could be considered when choosing an overall shape for the piece, and one strategy would be to structure the performance using an archetypal form used by DJs when constructing a set. In this respect, performers of Mutations (megamix) are directed towards archetypes such as the “rollercoaster” and the “work-out” 18[18. As Brewster and Broughton point out, DJs will use tempo changes during the course of a set to create characteristic formal shapes such as these. In the case of the “rollercoaster”, the overall speed will gradually increase to a peak before switching suddenly to a lower tempo. This fluctuation is then repeated throughout the course of the set, creating a repeated contour that resembles the rise and fall of a theme park ride (Brewster and Broughton 2002).], as identified by Brewster and Broughton (2002, 141), which may be used as structural templates. Again, coherence is the overriding concern.

Mutations (megamix): Max/MSP Software

As previously discussed, creating a specialised piece of software using the graphical programming environment Max/MSP was conceived as an effective way to realise the compositional aims of the performance, in an accessible and easily manageable self-contained package. In many situations it would not be possible to incorporate live Disklavier, for example. While the use of pre-recorded material circumvents this problem, due to the large number of soundfiles incorporated in the performance, an Ableton Live project file may be considered an overly complex method of performing the work. This is specifically the case when additional performers are used, who may not be familiar with the Ableton software and Ratcliffe’s intended use of it. The Max/MSP version of the project makes the process simpler and more accessible for performances that use the recorded soundfiles, by providing only the required functionality. The software can easily be distributed via the internet. Additionally, it provides a carefully considered mapping of controls (designed for a Korg NanoKontrol), and some unique DSP.

As discussed, the Mutations (megamix) software was conceived by Ratcliffe and built by Weinel, as a collaborative effort. Ratcliffe proposed the fundamental design and specification of the software, which was then constructed by Weinel. The software incorporates some feature suggestions from Weinel, such as DSP modules which were also used for his own projects such as Bass Drum, Saxophone and Laptop. 19[19. Discussed in Weinel’s article “Bass Drum, Saxophone and Laptop,” eContact! 12.4 — Perspectives on the Electroacoustic Work / Perspectives sur l'œuvre électroacoustique (August 2010). [Last accessed 10 January 2011]] Notably, Mutations (megamix) also incorporates a Doppler shift designed by Rajmil Fischman. Features such as these were considered appropriate for use in both pieces of software and were therefore adapted accordingly.

Figure 1 shows the graphical user interface of the Mutations (megamix) software. The following section shall detail the operation of each visible module.

MIDI Control Input

The most vital controls of the patch are accessible via a Korg NanoKontrol. The patch provides a graphical display (the top module seen in Fig. 1) that mirrors the appearance of the device itself. This makes the correlation between the physical surface and the patch as explicit as possible, with the function of each button and slider labelled. The controls provided each correspond to buttons on the five modules positioned beneath, which perform the actual sound processing (and provide a more detailed display than the image of the Korg NanoKontrol).

Mutations

The centre left module labelled “Mutations” provides general control over all modules of the patch. The provided buttons enable the user to simultaneously trigger both the “Performance Extracts” and “Mutations” modules. Additionally, there are controls to reset the various parameters of the patch. Finally, there is a facility to play a fixed performance that was produced with the patch. This function is provided for situations where a performer is not available to manipulate the patch using the Korg NanoKontrol, allowing a fixed performance of the electronics that may be combined with the live piano part.

1. Performance Extracts

This module enables the playback of the piano “performance extracts” (pre-recorded soundfiles). A range of controls enable manipulation over the playback of the 49 soundfiles 20[20. Although seven source works were originally used, only three of these (Arabesque No. 1, Berceuse, and Des Abends) were incorporated within the Max/MSP patch. This reduction was made in order to provide a more manageable set of materials, in keeping with the accessibility and streamlined nature and of the Max/MSP patch.], their loop status and the speed of playback (which affects the pitch).

2. Mutations

This module enables the playback of the Disklavier “Mutations” (pre-recorded soundfiles). As with the performance extracts module, the range of controls enable manipulation over the playback of the 49 soundfiles.

Audio Effects

The “Audio Effects” rack (Fig. 1, bottom left) provides a range of DSP, which can be applied to either the performance extracts or Mutations streams of audio. Effects include overdrive, Doppler (incorporating a design by Rajmil Fischman), filter, ring modulator, reverb and a gate. Parameter controls are available for each.

Audio Output

The “Audio Output” module (Fig. 1, bottom right) provides the performer with controls to mix the output amplitudes of the signals. The design is similar to the style of hardware mixers usually used by DJs. In addition to the amplitude controls for “extract” and “mutation”, a crossfader enables the two signals to be mixed as a DJ might do so. The crossfader incorporates a range of fader angles, for linear and non-linear amplitude transformations. This feature is often found on the type of mixers popular with scratch DJs. 21[21. Scratch DJs or turntablists manipulate records and turntables to create improvised rhythmic material in hip hop music. Variable and non-linear crossfader angles are often popular with scratch DJs, since they enable the user to splice and mute sounds rhythmically with the minimum of physical motion.] Providing these DJ mixer-style features enables the mixing of performance extracts and Mutations in a similar range of ways to those employed by a DJ.

Further controls on this module enable the selection of an output device and general DSP settings. In addition a “Play/Record” module allows performances created with the software to be recorded, either as a single stereo file or as a multi-track file. The provision to record multi-track files enables the material to be manipulated further and fine tuned, in the event that it is required for the creation of a fixed recording.

Performance: NIME 2010

Mutations (megamix) was performed at New Interfaces for Musical Expression (NIME 2010), Sydney, Australia, 17 June 2010. This performance put the accessibility of the software and performance to the test, and called for a re-evaluation of the performance model previously devised. Ratcliffe and Weinel were unable to attend the performance, and pianist Zubin Kanga stepped in to perform the work at short notice. Due to the circumstances, it was felt that performing both the piano part and using the Max/MSP software simultaneously would be impractical for Kanga as the solo performer of the piece. It was agreed that the ideal scenario would be for Kanga to perform the piano part, while an additional performer would be used to perform with the Mutations (megamix) software. Unfortunately, due to the limited time available to prepare for the performance, a second laptop performer was unavailable. Therefore, it was agreed that the best way to approach would be for Kanga to perform the live piano part in synchronisation with a fixed tape part, which Ratcliffe produced using the software in advance of the performance.

Ratcliffe provided Kanga with a fixed tape part, together with score fragments (“performance extracts” and “Mutations”). This enabled Kanga to develop improvised material to accompany the tape part. Kanga describes his approach:

Though I improvised in the final performance, I prepared excerpts from these works to match those used in the “tape”, blending into these interludes, while using more complex gestures at extremes of the keyboard to match the more abstract, MIDI-created sections of the tape. Though I could have resisted this obvious solution of matching the tape and my improvisation by style, I felt the combination of “found objects” alternating with more abstract, complex textures was enough in the tape part to provided adequate musical interest and contrast, and that my role in this context should be as an accompanist of the “tape” part. This preparation of gestures and excerpts to either blend or contrast with the different sections of the ‘tape’ allowed me to improvise within a framework, ensuring a balance of freedom and coherence. (Kanga 2011)

As Kanga highlights, although the performance contained a degree of improvisation, his familiarity with the fixed tape part inevitably influenced the performance towards a more planned structure than would have been the case if a second laptop performer had been involved. It can therefore be summarised that the performance model used was perhaps less than ideal, in that the spontaneous elements of this realisation were restricted. One might compare the tape part to the equivalent of a recorded DJ set: a reproduction of the real thing. Nonetheless, as with the mix CDs that are commercially released by popular EDM DJs, there is an advantage in that Ratcliffe was able to practice and fine-tune his own performance for the tape part, selecting the best realisation for Kanga.

Evaluation

Mutations (megamix) provides a novel concept for the integration of compositional hybrids that explore the approaches of EDM and DJ culture in piano-based music. The work offers a new strategy for the organisation of musical materials in real-time, drawing upon the idiomatic performance characteristics and recombinant aesthetic of EDM. Ratcliffe has explored this theme throughout his PhD research and continues to do so, demonstrating the effectiveness of such hybrid approaches through internationally received performances of Mutations (megamix) and other works. This article has discussed a possible approach to the design of such a composition and commented on some of the technical challenges for the realisation of such a work in a live performance situation. It is hoped that the article will stimulate thought for composers interested in exploring this exciting emerging area of composition and generate further discussions relating to the application of technology and accessibility of such projects for multiple performers.

The Mutations (megamix) software demonstrates the possibility to encapsulate a complex project of this kind into a streamlined and accessible package that can be disseminated via the internet. Through a design which incorporates only the desired functionality with a purpose built interface, the software is intended to offer compatibility, and provide a solution which enables Mutations (megamix) to be performed by as many people as possible. While the software also provides some unique DSP processing, such as Fischman’s Doppler shifter, it can be summarised that the advantage of the software is not in the audio processes used (which are mostly standard and could be achieved through numerous other means), but in the way that it streamlines these processes into a relatively neat and simple package.

However, the realisation of Mutations (megamix) at NIME 2010 highlights further restrictions that must be accounted for. In order for the piece to be performed using the software as a live instrument, it is essential for an experienced laptop performer to be present. This performer would need to be separate from the pianist, since the complexity of the software would make it difficult to control both components simultaneously. As a laptop performer was not available in this situation, the accessibility of the software to perform Mutations (megamix) was not utilised for this particular performance. Ratcliffe and Weinel are confident that the software makes such realisations possible, but it is also the case that further revisions to the software could be required in accordance with feedback from performers. This will likely be explored through future performances.

Mutations (megamix) is located within the constantly growing field of piano and electronics. 22[22. See Xenia Pestova’s “Piano and Live Electronics Repertoire List” in this issue.] There are many possible routes of further development for this project, in terms of expanding the concept and software design. Ratcliffe’s work with Disklavier and hybrids of DJ culture suggests many unexplored artistic avenues. Consider for example, the ability for a DJ to mix together live Disklavier material in synchronisation. This could be achieved by developing software that controls and manipulates multiple MIDI tracks, rather than recorded audio material. The sonic results which could be achieved with software such as this and prepared piano material could also yield interesting sounds, which explore the boundaries between techno and art music. These are future possible projects that would quite likely require the design of a separate piece of software, in order to maintain simplicity and ease of use.

Acknowledgements

This work would not have been possible without the support of the Arts and Humanities Research Council. Thanks also to Andrew Johnston for his assistance with the NIME performance and video recording.

Bibliography

Brewster, Bill and Frank Broughton. How to DJ (properly): The art and science of playing records. London: Bantam Press, 2002.

Butler, Mark. Unlocking the Groove: Rhythm, meter, and musical design in electronic dance music. Bloomington IN: Indiana University Press, 2006.

Kanga, Zubin. Private correspondence. 6 January 2011.

Ratcliffe, Robert. Live Mashup [Excerpt 1]. Published on Explorations in Sound, Vol. 4: The Sound of Live Performance. Furthernoise 2011 [FNCD007].

Ratcliffe, Robert. Mutations (2009). Available on the author’s MySpace page. http://www.myspace.com/visionfugitive [Last accessed 11 January 2011]

Ratcliffe, Robert and Andy Willy. Mutations (2009). Video recording posted by user “RobRatcliffeComposer” (13 July 2010, dur. 5:46). http://www.youtube.com/watch?v=6ZYC_O8jC74 [Last accessed 11 January 2011]

Smith, Sophy. “Compositional Strategies of the Hip-Hop Turntablist”. Organised Sound 5/2 (August 2000) “Definition of Genre,” pp. 75–79.

Weinel, Jonathan. “Bass Drum, Saxophone & Laptop: Real-time psychedelic performance software.” eContact! 12.4 — Perspectives on the Electroacoustic Work / Perspectives sur l’œuvre électroacoustique (August 2010). https://econtact.ca/12_4/weinel_psychedelic.html [Last accessed 10 January 2011]

Weinel, Jon and Robert Ratcliffe. Mutations (megamix). Software. 2010. Available on Weinel’s website. http://www.jonweinel.com [Last accessed 10 January 2011].