My Cyber Chamber Music Partner

A Project for Piano, Electronics and Video

by Rei Nakamura

The question is: why piano and electronics? Morton Feldman, for one, preferred the piano as it had always been used, which is to say, on eighty-eight keys. Conversely, John Cage never stopped his search for new sonic possibilities, not only on the keys of the piano, but also by striking the inside of the piano on the body or the strings, or by preparing the piano by placing materials such as rubber or bolts between the strings of the piano and then playing on the keys. But my aim is hardly to discuss the various ways in which composers have tried to revolutionize the piano as an instrument, nor how electronics came to play a part in this process. Rather, I intend to stay with a much more personal approach, looking at my relationship with the piano as a performer and how electronics and video work in this situation as my “chamber music partners”.

My discussion has four discrete parts. First, I introduce my own initial encounter with video and electronics, which led me to start my own project. Second, I consider various different uses I have made of live electronics, drawing on specific compositional examples. Third, I demonstrate the particular role video plays in my project and what this might say about what performance means to me. Fourth, I describe the process of working with Annesley Black in the development of her piece, 4238 de Bullion, to show a specific example of how one might work with a composer from the birth through the development of an individual piece. I am particularly interested here in how solutions can be found to deal with the unexpected obstacles — not only technical ones — which arise in the developmental process. Lastly, I suggest what I believe to be the advantages and disadvantages of working with electronics and other new technologies from a performer’s perspective.

Encounter with Live Electronics and Live Video

I had the pleasure to encounter live electronics and live video through the Canadian composer Annesley Black. After performing a solo recital in Freiburg in 2006, Annesley Black approached me and asked if I would be interested in collaborating on a project. She hadn’t considered specific details, but thought it might combine the piano with live video and live electronics. Watching me perform, she said she had been engaged by the physical movements I made as I performed and that she had in mind a piece with a live video, emphasising my movements as a performer, which would be both composed and choreographed. The piece which resulted was genuinely unique and, moreover, guaranteed its newness through the fact that the technology needed to execute live video processing, with practically no delay, has only recently become easily available.

During rehearsals for Black’s piece, it became clear that setting up a piece involving both video and electronics is an extremely complex, time-consuming process. For that reason, I decided it would be more efficient (and more exciting in performance) to have a programme devoted to just this set-up: piano, video and electronics. Even though the set-up was the same, the first programme I created in this way was received with excitement and interest by the public, who recognized both the newness of each piece, but also that each piece was completely different from each other on the programme. It was through the experience of working with composers on this first programme that I realised myself that the restrictions of the set-up actually brought greater interest and contrast. Since this first programme in 2008, I have decided to start a project, Movement2sound, sound2Movement — a piano solo program with electronics and / or video. Since then, I have realized a programme of new pieces written specifically for me and for the project every year. The only condition for the composers is that it has to be for piano and at least electronics or video (or both) and that those elements may be either live or pre-recorded.

Different Types of Live Electronics

Playing with electronics can be like playing with a chamber music partner. In my experience, there are four ways in which this relationship with live electronics takes place:

• The electronic patch is completely fixed, triggered automatically by a computer at specific points. In this situation, it is only possible to play with a click track;
• The composer triggers pre-recorded sounds and/or sounds recorded live in performance by hand;
• The patch is defined, but is controlled through the interaction between the performer and the computer. What results is then dependent to some degree upon chance and there is a tendency toward improvisation with certain defined materials;
• Free improvisation with a live partner, who entirely controls sounds recorded live in performance.

It is probably clearest to give examples of each of these types of situation.

Annesley Black’s 4238 de Bullion is an example of the first type. It has an entirely fixed patch and, similarly, an entirely fixed video. I will return to this piece below in the section specifically on video.

Alistair Zaldua’s inner and outer mind (2010), prèmiered at the ZKM Piano+ Festival, was not planned as, but ended up being, an example of the second type. Shortly before the concert we had difficulties with the prepared patch and, in order to perform the piece, the only option was for Zaldua to trigger the sound manually. Though this was not how the piece was supposed to have taken place, in truth I felt extremely comfortable that the composer was performing and interacting with me.

An example of the third type of situation might be Gerhard Winkler’s Hybrid VIII (PiAnimaux) (2010) for Disklavier (a MIDI-controlled piano), piano and live electronics (Fig. 1 and Audio 1), which, like Zaldua’s inner and outer mind, was prèmiered at the ZKM Piano+ Festival. Here the pianist has a range of roles to play. First, the pianist is to act as the Disklavier’s “pilot”: certain keys pressed on the piano run through a pitch-detector and trigger keys on the Disklavier (there are eight different “worlds” which determine the reactions of the Disklavier; the pianist decides, too, which of these worlds should apply). Second, the particular musical motives available to the Disklavier within a particular “world” can be changed by moving a point displayed on a monitor placed on the right-hand side of the piano. Third, since not all the notes of the piano directly affect the Disklavier, the pianist can improvise freely with all those notes that do not act as triggers. Moreover, whenever middle C is played, it prevents the Disklavier from responding to any fresh input from the pianist, continuing to play whatever it had previously been told to for a given number of seconds. Within this time period, the pianist can improvise entirely freely. In certain “worlds”, live electronics are also activated. Given the degree of possible combinations of activity and material in the piece, even though the Disklavier is restricted to a certain number of materials in any particular “world”, the music which results is always different. The pianist must act, listen and re-act, in just the same way as when improvising with an ensemble.

Since 2003, I have worked in an improvisation duo with the artist Peter Vogel. This is an example of the fourth case: I play the piano, Vogel performs with live electronics (Fig. 2). Ever since our first encounter, I have been fascinated by his work in general, and in specific pieces like Klangwand or Shadow Orchestra. What particularly caught my attention was the interrelation in his work between sonic and visual elements: sound objects which react to light or to shadow and light objects which react to particular sonic frequencies are fundamental to his work. I discovered that he had some old recordings of his own music on magnetic tape and that, moreover, he had constructed a 7-channel delay, which could record and save a maximum of 8 seconds of loops; I asked him if he would be interested in improvising with me, not with the intention of performing in concert, simply for us to find out what would happen. With Vogel’s machine modulation, echo, dislocation of phrasing and tempo / pitch shifting were all possible. The advantage of the machine, over a laptop, say, really lay in the fact that it was possible to control many parameters simultaneously via the knobs (Audio 2). Because the loops the machine was capable of creating were between 2 and 8 seconds long, the improvisations that resulted tended to become very minimalistic. Working with a laptop, in contrast, made it possible to work with significantly longer loops, to have more flexibility in selecting which parts of recorded material would be used for looping and to have access to a much more diverse range of effects. Moreover, the laptop is much lighter and more portable! For these reasons, Vogel’s part of our collaboration has moved entirely to laptop since 2009 (Audio 3). For me, this project is something totally different from other collaborations with composers. Rather than reading scores, attending to the wishes and desires of the composer, it is about playing and listening. More than anything else, working in this way has caused me to reflect upon how narrow-minded I was with regard to contemporary music at the point when Vogel and I started working together. We exchange our different points of view — even though we work in sound, he is an artist, I am a musician — letting the ideas and preconceptions we draw from our respective fields combine and intermingle in order to improvise. We listen. I have learned a lot from him, about freeing myself from constraint, letting myself hear afresh; only by being free does it become possible to improvise.

Performance

How the Video Complements the Performer

The relationship between movement and performance has always been central to my interests, most especially when I am on stage. This is not to make the music into theatre, but rather to recognise that performance of music is always already visual. The moment a performer steps onto a stage, people are watching (unfortunately, sometimes focusing more on watching than listening). Thus, stepping onto to a stage has to be done in an awareness of what it means to step onto a stage. A musical performer hardly has to think of themselves as a performing body in the way that a dancer might, but musicians too often, I believe, forget or ignore this aspect of performance.

For me, movement and sound are independent, mutually supportive. While many performers do link movement and sound, the aim of my project is to investigate deeply the fundamental potentials of this interrelation in order to discover an entirely new type of performance, both visually and aurally.

In Michael Beil’s Mach Sieben (2000) a pre-recorded tape and video both play the material of the piece backwards, while the pianist performs it forwards. The pianist looks as if they are playing in a piano duo with their other self in the projection on the right-hand side of the stage (Video 1).

Annesley Black’s 4238 de Bullion (2007), premièred at the Musikhochschule Freiburg, is written for piano, live electronics and live video. A video camera is placed on the left-hand end of the piano, with the image from the camera projected in real-time on the screen above the piano. Sometimes the image shows exactly what the pianist is doing “live”, but at other times, for instance, the image from five seconds earlier might be reprocessed in slow motion and projected (Video 2). Since this has all been pre-programmed in the computer program Pd, the pianist requires extreme accuracy, playing with a click track. Personally I feel restricted by the click track and prefer playing without one, even though in this case it is unavoidable.

In die Haut Anderer (2008), premièred at the Klangwerkstatt Festival in Berlin, Mark Barden employs an extremely rare combination: piano with video playback (Fig. 3). An homage to Emma Hauck, a patient at the Heidelberg Psychiatric Clinic at the beginning of the twentieth century, Barden’s piece succeeds in linking these disparate media by providing a strong conceptual focus: obsessive repetition. In the video, two words — Herzensschatzi komm — are written repeatedly with pencil on paper; once the page is full, we see the pencil rewrite the same words on top of themselves, retracing but slightly obscuring the lines (Fig. 3). This process repeats until the page is black. Mirroring this process of compulsive overwriting, the musical material presents many species of repetition: single note repetitions with different dynamics and registers; additive transformations; repeated figures coupled with increases in velocity which intentionally force the player to lose control; silent key depressions of material which occurs later with the full sound; and finally sequence repetitions, which are notes organized in the same order, but with varying combinations, chordal densities and durations. The material remains the same, but sounds different every time, in effect using repetition to transcend repetition itself (Audio 4).

Christoph Ogiermann’s zig gesetze angewendet auf rei nakamura / umpteen laws, as applied to rei nakamura (2009, premièred in ZKM, Piano+ Festival) focuses on the overtly theatrical elements of the project. Around the audience members, who are seated in the centre of the performance space, are four small stages, where the pianist performs with the “assistance” of the composer. On each stage are between one and several instruments, such as MIDI-grand piano, grand piano, keyboard, piano, CD player, sampler keyboard, etc. The performer is required to play on one stage and then run to another, where she must play, or speak, or both. It is hardly possible for the audience member to capture everything that happens in the piece, since so many events are occurring at once: when the performer is at the front of the performance space, whispering into a microphone (Fig. 4), a pre-recorded video is also running on the screen; the pianist also performs with the composers on top of a tape part, simultaneously speaking an incomprehensible text. The composer gives the audience members a space in which to experience what it might mean to attempt to absorb and make sense of an excess of information, reflecting contemporary society, where even multiplying oneself may not be sufficient to survive (Video 3).

As the examples show, video has an essential role in the project. The image gives a visual counterpoint and adds a further dimension to the performance.

Working with Composers

The Process of Working with Annesley Black on 4238 de Bullion

I have had the opportunity to work closely with each of these composers: Annesley Black, Mark Barden, Hannes Seidl, Robin Hoffmann, Orm Finnendahl, Jeffrey Trevino, Christoph Ogiermann, Sven-Ingo Koch, Eduardo Flores Abad, Alistair Zaldua, Gerhard Winkler and Michael Beil, and am looking forward to working very soon with Martin Iddon, Johannes Kreidler, Thomas Bruttger, Bernd Asmus and Fredrik Wallberg.

In this part, I would like to take Annesley Black’s piece as an example of such collaborations, in order to show how this piece began and developed through time. I will describe in detail the difficulties we encountered and the solutions we discovered through a work-intensive process.

4238 de Bullion was a piece that underwent many transformations during this (at times trying) process, as did we. In 2006, after a performance of mine in Freiburg, Black approached me and the idea of a collaboration took root. In 2007, Black began conceptualizing and writing the piece. Despite this lengthy development process, the very first performance in April of 2008 was also the very first rehearsal with the complete audio-video set up, since it was previously, due to technical limitations, only possible to rehearse with the sound component of the live-electronics.

The piece had been initiated as a university project. Professor Michael Beil, a composer and video artist was invited to Musikhochschule Freiburg to lead a workshop with live video for Professor Orm Finnendahl’s digital studio composition class. 1[1. See Beil’s article “AV — Music and Video” in this issue.] At this point, I had never worked with live electronics, apart from my improvisation project with Peter Vogel, in which he took complete responsibility of the electronics. Black did have some experience with electronics, but not with video. Other student works were created for this project, all using a Pd patch designed by Orm Finnendahl. For the concert each piece required a different stage constellation, some of which were so intricate that the complete set-up of the whole event was not available for rehearsals until the day of the concert. It was therefore impossible to see and hear the piece in its full audio-visual configuration and there was no time for Black to change anything or revise the visual part of the piece until after the première. Much rehearsal time in the concert space was spent working out details of the computer program designed for the project, or waiting nervously for the video signal to recognized by the software.

Just as pianists are accustomed to practise right hand and left hand separately, or extract two voices out of four, it is necessary to practice the choreographed movements as a separate voice in this piece. The movements in the score are notated graphically (Fig. 5). To decipher the score was not my main challenge nor was it a particularly unfamiliar task for me (although doubtless Black’s presence, explanations and corrections throughout the learning process helped a great deal); after understanding the notation I had to isolate and practise the movements of my fingers, wrist and arm (a strange business, even to a pianist who regularly premières new works).

Then came the added (acquiescently shared) assignment, which inspired regular visits to local toy, kitchen appliance and fishing stores, of finding small oddities, such as fishing weights, magnetic marbles and the infamous wind-up pecking chicken (Fig. 6). These objects are integral to all levels of the piece (visual, acoustic and choreographic). The fishing weights, for example, serve several functions: first, they act as a visible sustain pedal, depressing specific keys; second, they are objects acted upon by the hands and therefore play a thematic role in the choreographed movements as well as in the general visual material; and, third, they provide acoustic material: the “guerro” sound of the weights dragging across the keys becomes a recurring musical element, which is (acoustically and electronically) compositionally developed. Knowing that the dragging of the iron weights could be detrimental to the keys, I suggested we protect the keys by covering the weights with slips or bags. So an additional trip was made to a fabric store to find material to make bright pink bags. I remember the late-night sewing session with Annesley Black and some wholly indispensable pizzas, in the middle of our most intensive rehearsal phase, exhausted, labouring over the six tiny bags. Only once all accessories were collected and completed, and after the notation had been discussed and I was familiar with my part, were we ready to rehearse with the electronics.

The next step was to rehearse with the click track, which is not only used as a rehearsal aid, but also for the performance as a means of synchronising the pianist with the live electronics. I had some problems with the click track sounds. They were uncomfortably high-pitched, but the particular click sound, when transposed too low became inaudible in certain parts of the piece. However, Black was flexible and attempted to rectify any problems of a technical nature so that it would be more comfortable for me. We practised with the click track whenever we could and Black checked the synchronicity between the click track and the live electronics carefully. The Pd patch worked for the most part, but there were some situations in rehearsal when it failed to deliver. In these cases, I rehearsed my parts alone while Black tried to resolve the technical difficulties. Since I had plenty of material to practice on my own, it was not such a problem for me. When the computer did not respond at all, we worked on the movements and choreography of the fingers, which also rehearse well without electronics.

The only aspect in which we were not making any progress at all was the video part. For the first version of the piece, the camera was placed on a stand about 3 meters above the piano. The image projected on the screen was from a bird’s-eye view, displaying the keys and some of the inside of the piano. This angle posed various problems. First of all, when I played inside the piano, my head was visible and prominent on the screen. Regrettably, it was neither pleasant nor interesting to see somebody’s head moving back and forth across the screen. Second, since the camera shot had to include the entire keyboard, the image as a whole lacked clarity and detail. The fine finger movements became difficult to see and thus the viewer lost interest in them. Unfortunately, at the time of the première both the performer and composer were somewhat overwhelmed by the number of unforeseen details and problems; we could not find an immediate solution for the video image. For my part, I was more than happy that I could play the music and simultaneously and “dance” the choreography well with my fingers and Black was simply relieved that the patch was working at all. We were too absorbed and somewhat overpowered by the complexity and poly-dimensionality of the piece.

After the première we planned a handful of concerts, beginning a couple of months later  —  Festival Klangwerkstatt (Berlin), Piano+ Festival ZKM (Karlsruhe), Black’s final composition exam / recital (Musikhochschule Freiburg), Hochschule für Medien und Kunst (Köln) and at the Frankfurter Gesellschaft für Neue Musik (Frankfurt). Black went through the piece and revised carefully and we met frequently to rehearse and discuss what we could both improve. The first thing we agreed that had to be changed was the angle of the video. After testing different positions, we decided on the final solution: to place it on the left-hand side of the piano. This way, it was able to see the exact movements of the hand and fingers enlarged on the screen and unnecessary images would not disturb the piece. To adjust to the new camera angle, Black adapted the piece, keeping the sections which involved the inside of the piano to a minimum. Some changes in the audio-electronics were also made, for instance, in a section where I pluck stopped harmonics with a guitar plectrum. In total, about 8 minutes of the piece were cut and very little of what remained was left unchanged.

Between each performance listed above, there was between a one week and a one month gap, during which we were able to work and revise the piece. For the third performance, there were still some adjustments made to the score. We even made changes right up to the last performance of our tour. Instead of having a technician trigger the live-electronics (most sections are automatically triggered by the patch, but in three places it is necessary to control the start of a new section directly), Black set up a MIDI pedal, so that I could command the triggers. These were minor changes, all requiring some getting used to, but they simplified the setup and allowed for the piece to flow more smoothly.

To understand and interpret a piece musically is hard enough, but to go one step further and undertake a project with visual elements certainly required more time and energy than usual. As a performer, I was challenged to articulate the precise movements that are notated in the score but at the same time to maintain an understanding and interpretation of the musical context. Black, as a (sonic) composer, faced the challenge of formulating not only her auditory, but also her visual material in a way that allowed that which she imagined and conceptualized to reach not only the ears but also the eyes of the audience. I do think, though, that the energy put into this work was not in vain. Black achieved a multifaceted piece, in which each component has its own role and importance, approaching the newest technology in a refreshingly unique way.

Advantages and Disadvantages of a Programme Incorporating Electronics and Video

Putting together a rich and varied programme incorporating different combinations — piano and tape, piano and video, piano and live electronics, or even piano, live video and live electronics — is exciting, but risky too. The more complex the set-up (and the change of set-up from one piece to the next), the greater the risk of technical problems. Live video is, it seems, especially capricious. Black and I found, on several occasions, that the computer did not want to recognize the existence of the camera we had just plugged into it.

Another disadvantage is similarly pragmatic: one cannot offer the programme to every concert hall. It is important to have the right combination of infrastructure, instruments, equipment and, not least, highly qualified technicians. A few places — I am thinking in particular of ZKM in Karlsruhe or the SWR Experimentalstudio in Freiburg — are ideal for this sort of work. In contrast, a programme involving both live electronics and live video is unsuitable for the majority of concert halls. Naturally, compromise is necessary: a programme with only tape, or with only video, makes matters simpler.

The time spent waiting is inevitably much greater than when working with purely acoustic instruments. It often feels more like preparing a theatre project, waiting until the lights have been set up. It would not be unusual to have spent half a day working on the technical set up, before even beginning to play. It should go without saying that patience is needed and given the length of time technical preparations can take, it is vital for the performer to be fully prepared to rehearse on arrival at the hall, since these technical difficulties can mean that time for rehearsal is scarce. The risk of a concert being cancelled is always present (though it’s never happened to me, it once got unnervingly close). As a performer, it remains vital to trust the technicians and the composer to do their jobs; we can then stay calm and do ours.

Even though, on paper at least, a programme like this is for solo piano, the reality is quite different. The video and electronics support the piano as if they were ensemble members and make possible an extremely diverse range of musics. Even without other musicians, the situation that arises is one in which there is extremely intensive work between the composer and performer.

Seen from the perspective I just described, it might sound like I feel such a programme has more disadvantages than advantages. Truthfully, the reasons for working on this kind of repertoire have little to do with enumerating positives and negatives; the most important reason is that the project feels unique, interesting and new. It is something, too, which could only have been achieved by my generation of musicians. The computer, television, electronic sound and electronic music have become increasingly integrated into our lives since the 1990s. They are, undeniably, an integral part of what it means to live now. They reflect our society and using them within composition might lead, I believe, to new modes of expression within contemporary music, new ways of performing and being a performer. I want, then, to work with composers of the twenty-first century who feel the same, who are still searching for something new. This does not need to mean abandoning the past: I want to continue studying, researching and learning from what has gone before. At the same time, I want to look forward, to work with today’s composers, today’s techniques and technologies, in search of a performance of and for my generation.

Acknowledgements

The author is grateful to Mark Barden, Annesley Black and Martin Iddon, all of whom helped in the formulation of the present article.