Performing Proprioception and Effort in “Hypo Chrysos”, an Action Art Piece for the Xth Sense

by Marco Donnarumma

A “Media Gallery” featuring works and performances by the author is also published in this issue of eContact!

Hypo Chrysos is a work of action art for vexed body and biophysical media. Here the term biophysical addresses audio and visual media that emerge as a joint result of biological and physical mechanisms of the body. Hypo Chrysos is based on the Xth Sense (XS), a biophysical musical instrument I have been developing and performing with since March 2011.

As opposed to other biosensing controllers such as the Biomuse (Knapp and Lusted 1990), that deploy electrical signals released by the body, the XS uses a microphone that picks up subcutaneous mechanical vibrations, or better, sounds that originate within the muscle fibres of the performer’s body (Fig. 1). The XS uses these sonic vibrations as sound material to be processed using the same data stream. The performer controls the live sampling and spatialization of the muscle sounds, which the computer diffuses through the loudspeakers. This a model I refer to as visceral embodiment. 1[1. For the interested reader I recommend my related discussion included in a recent paper presented at the conference on New Interface for Musical Expression (NIME) in May 2012.] Hypo Chrysos exasperates the viscerality of the body, so as to to critically explore the territory in between biosciences, music technology, performance art and human affect. Whereas in previous works for the XS I explored the body as a source of musical meaning, with silence and sonic events constructing an aural architecture, this piece has a different scope: to vex my own body and materialize its strain in the outer world. There is no score, and musicianship is purposely deterred by the condition of the performance.

During this twenty-minute action I pull two concrete blocks in a circle. The increasing strain of my corporeal tissues produces continuous bioacoustic signals. I wear two XS microphone sensors placed on my arms and veins that capture blood flow, muscle sound bursts and bone crackles. These sounds are amplified, distorted and played back through eight loudspeakers. The bioacoustic data stream excites an OpenGL generated swarm of virtual entities, lights and organic forms diffused by a video projector — a window into the subcutaneous tissues of my body (Fig. 2). The performance is exhausting. It requires a fair amount of training and concentration. To design and perform Hypo Chrysos meant to inform my current practice by linking self-perception, effort and physicality to the notion of visceral embodiment. In this article I unfold this linkage in the practical context of Hypo Chrysos.

Proprioception, Affect and the Sonic Body

While learning to master the XS, it became clear that playing it required a greater concentration than I was used to. A traditional musician is used to receiving direct tactile feedback from her instrument, which she can use to refine her performance skill. In most cases, a performer using a biotechnological musical instrument lacks this sense of touch; instead of plucking a chord or pressing a key, she either executes an empty-handed gesture or rests in a static position. This is the main reason why I began to perform with my eyes completely closed. Performing as if blindfolded I can better control my muscle tension.

This is a sense known as proprioception. It uses information from the sensory receptors 2[2. A sensory receptor consists of the ending of a sensory nerve. It records an internal or external stimulus and transduces the related information in electrical impulses received by the central nervous system.] of muscles, joints and inner ear to determine limb position and strength of the effort. The muscle sensory receptors are called muscle spindles and mechanically record the changes in the muscle length. Proprioception is best understood when compared with exteroception (the tactile sensitivity) and interoception (the sense of movement of the internal organs). As illustrated by Brian Massumi:

Tactility is the sensibility of the skin as surface of contact between the perceiving subject and the perceived object. Proprioception folds tactility into the body, enveloping the skin’s contact with the external world in a dimension of medium depth: between epidermis and viscera. The muscles and ligaments register as conditions of movement what the skin internalizes as qualities… (Massumi 2002, 58–59)

After about sixteen months performing with the XS, I felt as if my body had produced an acute enhancement of the proprioceptive sense and that I had acquired improved flexibility of my joints. At the same time, I had become unconfident when performing with sound systems that could not provide a Sound Pressure Level (SPL) between 100–120 dB. Low frequency sounds at such a high SPL make the body vibrate and I had noticed that the lack of this feeling would somehow compromise my performance. Could there have been something worth investigating? Very recently, I began reviewing some neurophysiology articles and started to “connect the dots”. Evidence exists which proves that vibration exercise (VbX) enhances physiological responses and muscle functions (Cardinale and Bosco 2003; Cochrane 2011). VbX consists of body stimulation by means of low frequency mechanical vibrations. By externally applying vibrations to the muscle or to the tendon, the muscle spindles activate to dampen the tissue displacement. Contextual user studies reported that the continuous activation of the muscle spindles by means of a chronic VbX treatment improve neuromuscular performance (Cardinale and Bosco 2003).

Drawing on these observations I came to think that performing with the XS may have the potential to alter proprioception and neuromuscular activity.

In a performance using the XS, the player focuses mainly on her sense of proprioception. Because the MMG loudness increases with increasing nerve stimulation (Bolton 1989), the information about the changes in the performer’s muscle length become paramount to the musical articulation of muscle sounds. But each time a contraction activates the player’s sensory nerves, the amplified muscle sound emitted by the loudspeakers feeds back into her body. At this point, the player’s sensory fibers activate again to dampen the sound wave displacement of the tissues. As the process happens in real time, the body becomes the subject of a multi-modal sensory feedback. Given that the VbX can alone improve neuromuscular performance, it is reasonable to suggest that the sensory feedback loop enacted by the XS may be capable of altering proprioceptive mechanisms in the user. Although time constraints have for the moment impeded the factual validation of my argument, this scenario denotes the implication of neurosensorial mechanisms in the performance of biophysical music that are worth further research.

Being a Body Affecting Bodies

When my inner body vibrations become tangible sound breaching into the outer world, they invade the audience members’ bodies through their ears, skin and muscle sensory receptors. The tension of my inner body resonates the their muscles, establishing a nexus between player and audience. The listeners’ bodies, my own body and the performance space resonate synchronously. The flesh vibrational force becomes a vector of affect; my proprioceptive dimension has been magnified and now embraces the bodies of the audience members. Here, the term affect refers to Deleuze and Guattari’s definition of a body potential to affect and be affected. It is a proprioceptive potential of interaction among bodies (Deleuze and Guattari 1987). Because of its position in between cognition and viscera, affect is autonomous and unactualized. As Massumi puts it, affect is not an object relegated within the body tissues, rather it “escapes confinement in the particular body whose vitality, or potential for interaction, it is” (Massumi 1995, 96). In Hypo Chrysos, affect expands beyond the boundaries of my tissues in which it originates and modulates the audience sensory system by activating resonances in their flesh.

A Wetware without Musicianship

In this work, my role is certainly not that of a musician in the strict sense of the term. Rather I am wetware; a body with no creative cognition, but with an inherent expressive indeterminacy. Although the tension of my body produces the emerging soundscape and the variables to process it, I have little time to think about playing, for my only aim is to resist to the strain. There are no musical gestures, as commonly understood, rather it is the proprioceptive agency which directs the organization of the sonic experience. The challenge therefore was to find a balance between the apparent agency of the performer and the outwarding sound forms. In this sense, the interaction with my instrument in Hypo Chrysos presents evident differences if compared to the execution of my previous piece, Music for Flesh II, for which I also use the Xth Sense 3[3. See the “Media Gallery” for videos of Marco Donnarumma performing his works Music for Flesh II and Hypo Chrysos.] In the latter work, the design of the sound gestures is accurate, and it provokes the expectation of a given sound form. In Hypo Chrysos, however, the sound gestures cannot be precisely designed, because the goal here is not to create or play music, but to simply pull a weight. In a completely idiomatic manner, the sound gestures in Hypo Chrysos remain integral to the sound world I create, but they lack a musical scope.

Effort and Physicality

In traditional music and dance effort implies physicality. When a high degree of technicality is matched by a brave imagination in exploring new sounds and body movements, the experience of a performance can be breathtaking. Imagine a double bass player that unnaturally bends the whole body to resonate a hidden part of his instrument 4[4. This image refers to an inspiring concert by John Eckhardt which I attended in June 2012 at the Dialogues festival in Edinburgh, UK.], or a dancer forced to use very short crutches to frantically move along the stage. 5[5. Here I refer to a scene from the dance piece bODY_rEMIX / gOLDBERG_vARIATIONS (2005), by the Compagnie Marie Chouinard. A video (10:14) of the performance can be viewed on YouTube. Posted by user “2beersbetterthan1” on 2 April 2010.] It is arguable that the performance of a computer musician is rarely associated with the idea of effort and physicality. There is a modernist preconception that the computer’s most prominent feature is that of providing great results with effortless human input. Although this deceptive notion is somewhat somatized by the mainstream, we should retain from applying it to the arts. It is a reflection of the passive approach to technology that the market has introduced in the past years, and that we have consciously accepted; as Ryan puts it:

Effortlessness in fact is one of the cardinal virtues in the mythology of the computer. It is the spell of “something for nothing” which brightly colors most people’s computer expectations. (Ryan 1991)

I share Ryan’s feeling in that this is a rather dangerous viewpoint when carelessly applied to music and performance. Ultimately, the effort exhibited by a performer on stage “is the element of energy and desire, of attraction and repulsion in the movement of music” (Ibid.).

Being passionate about action art I challenged my practice and tried to incorporate the notions of musical effort and physicality in that field. In an attempt to outline the expressive qualities of bodily effort, I looked for a performance strategy that would minimize the apparent musicianship, yet offer a sensuous and impactive sonic experience. The choice to realize an action art piece pushed me to move out of the comfort zone of a fixed concert setting. By changing the performative context, I could conceive new ways of playing the XS.

Next, I will describe the path that led me from the idea of musical effort to broaden the application scope of the XS. Before reading the following sections the reader is invited to view the performance video of Hypo Chrysos in the Media Gallery, also published in this issue of eContact!

Performance and Strain

Hypo Chrysos is freely inspired by the sixth Bolgia of Dante’s Inferno, located in one of the lowest of the circles of hell. Here the poet encounters the hypocrites walking along wearing gilded cloaks filled with lead. It was Dante’s punishment for the falsity hidden behind their behaviour; a malicious use of reason which he considered unique to human beings.

Using my arms to pull two ropes tied to concrete blocks, I struggle to walk along the stage. The ropes are short, forcing me to curve my torso toward the floor throughout the performance. I walk in a circle for twenty minutes in that position; I cannot give up until the action is completed. The overall weight of the blocks is 30 kg (Fig. 4); initially they are not extremely difficult to pull, but on the long run the resistance of my (very thin) body is truly stretched to the limits. First, I feel the abrasion caused by the rope on my hands; after about ten minutes the tension in my arm becomes painful, and a few minutes later the spinal column feels like burning due to the continuous friction of the vertebra.

The first performance resulted in much more strain than I expected, so I included a section in which I could perform sitting on the floor for few minutes and partially recover. Usually the day after the performance I suffer of delayed onset muscle soreness (DOMS), also known as muscle fever. Stiffness is a natural reaction of the muscles that after a strenuous exercise adapt rapidly to prevent damage to the tissues; in this condition every kind of movement is fairly painful, so I have to rehearse the piece two days before the public performance.

Mapping Textural Richness through Adaptivity

Hypo Chrysos is a work that aims to bypass the player’s cognitive mechanisms and materialize the biological body field of affection. Neither the music nor the moving images are controlled by my body, they rather emerge from within it. I am not a player, but only a body.

The XS sensitivity is increased to the maximum, so that any tension bursts of the tissues can be captured. The resistance of my whole body is crystallized in acoustical vibrations. The sounds are manipulated by means of a two-stage DSP system, consisting of a stack of feedback delay lines and distortion effects (fuzz and all-pass). 6[6. This text does not offer enough room to get into the details of the audio processing techniques which I refer to. However, those processes are documented in common literature and the XS audio effects mentioned in here are documented in their respective help files. These are available in the “Downloads” section of the Xth Sense documentation page.] Multiple sonic instances of the signal are stored, delayed, distorted and played back. In contrast with Music for Flesh II, in which the XS sensitivity is low enough to mute ancillary gestures, here the XS captures any sonic event from within my body. The sound input is therefore continuous and the feedback loop is virtually endless. A wall of sound slowly emerges from an octophonic array of loudspeakers, steadily increasing in loudness and frequency response. Initially the soundscape consists of low frequency sonic bursts, during each section further frequency bands are added, until the human auditive threshold is reached in the closing scene.

The mapping system is illustrated in the next section. It consists mostly of a small array of continuous events; this way I could overcome the impossibility of articulating musical gestures, and let the XS adapt to my body. The obvious drawback of using a limited set of control features is less sonic richness. I needed a strategy that would diversify the sonic outcome in the absence of musical intentions. The mapping of the features does not change significantly throughout the piece. Instead, I addressed the issue by creating subtle changes of the same mapping. For this purpose I coded a new XS mapping feature. The objects are fairly simple, but they opened up interesting possibilities. Before the actual mapping takes place, the incoming data stream is processed by customizable logarithmic or exponential curves. Then the stream can be offset by setting a custom bottom and top value. Finally, the stream can be reversed.

Another computational idiom that aptly exhibits the adaptivity of the XS is a rhythm tracking algorithm I coded for Hypo Chrysos. The rhythm I refer to is the cadence of my muscle contractions. The algorithm measures the amount of effort in a temporal and linear dimension. Namely, it monitors a given feature threshold and counts how many times that threshold is reached. When the feature peaks Y times over the threshold, the algorithm generates a trigger. This starts the playback of a pre-recorded sound sample. The Y value can be either set by the user, randomly generated or autonomously defined by the XS according to the player’s current energy. The idiom is used throughout the piece to trigger signals 7[7. The term here is intended as in the soundscape studies glossary.], such as single reverberated percussion hits, recalling a primitive ritual, or masking sound effects that indicate the transition from one scene to another.

Feature Extractions and their Application

The computer learns about the emergent physiology of my body by extracting discrete and continuous features from the MMG signal. Each sensor produces one analog signal output; this is digitized and passed through an array of algorithmic functions designed to meaningfully shape the incoming biosignal into diverse control features (Fig. 6), namely: Natural (N), Soft (S), Linear (L), Tanh (T) and Maximum Running Average (MRA).

The N value is computed in two steps. First, the initial data that describe the dynamics of the muscular motion are generated; the XS software tracks the root mean square (RMS) of the MMG output using a Hanning window of 512 samples. This could be thought of as a useful event on its own, however, my experience with the system showed that it can prove to be far too linear to constitute an expressive and detailed control event. Therefore, the RMS is passed through a custom function so as to sculpt a subsequent continuous event that imitates the elastic, and sometimes jittery contraction of the muscular tissues. It can be compared to the bending of a rubber band: when the muscle is flat, N is equal to 0; at the onset of the first contraction, N increases proportionally to the amount of energy released. However, when the contraction ceases, N does not fall back immediately to 0, but it bounces back and forth as the muscle tissue recovers to its resting position.

The S feature is obtained by passing N through a single exponential smoothing (SES) function (NIST/SEMATECH 2003/2012), which results in a much softer continuous event. S is generally used to drive subtle textural permutations of the control variables.

L is calculated by converting the direct MMG audio signal to a control value every 20 ms; this time interval proved to be the best compromise between high resolution representation of the biodata and computer processor performance. L is then passed through a SES function to obtain T. This feature presents minimal variations, it is therefore used to delicately control musical processes that require a careful and minimal lead, such as brief glissando and minimal sound spatialization.

Eventually, L is reiterated through a sub-process which produces the RMA. The computation consists of four steps: first, L is observed in order to identify the running average; then, the last maximum (LM) of the stream is extracted every two seconds; finally, the discrete LM value is normalized and interpolated with its previous instance to generate the RMA. This is a continuous event that moves away from the micro level of the single gesture and reflects the average amount of energy that is being released by my body in a wider time window.

The variations in the soundscape of Hypo Chrysos depend on the S and MRA features of my arms, which provide the less jittery data. The features are mapped to the wet mix of delay and distortion effects, to the amount of delay lines feedback, and to the input of a cosine panner. By using a minimal one-to-many feature mapping, but varying the curve scaling and range offset of the data stream, the XS can produce a uniform soundscape in which sonic richness is experienced through manifold microtonal variations. Moreover, the physiological state of my body before playing and the exhaustion I accumulate throughout the performance drastically affect the control features that my body produces. This is how textural variances of the soundscape exist, albeit neither actually planned beforehand, nor consciously enacted.

Bibliography

Bolton, Charles F., Anthony Parkes, Terry R. Thompson, Michael R. Clark and Chris J. Sterne. “Recording Sound From Human Skeletal Muscle: Technical and physiological aspects.” Muscle & Nerve 12/2 (February 1989), pp. 126–134.

Cardinale, Marco and Carmelo Bosco. “The Use of Vibration as an Exercise Intervention.” Exercise and Sport Sciences Reviews 31/1 (January 2003), pp. 3–7.

Cochrane, D.J. 2011. “Vibration Exercise: The potential benefits.” International Journal of Sports Medicine 32/2 (February 2011), pp. 75–99.

Deleuze, Gilles and Félix Guattari. A Thousand Plateaus: Capitalism and Schizophrenia. Trans. Brian Massumi. Minneapolis MN: University of Minnesota Press, 1987.

Donnarumma, Marco. “Xth Sense: A Study of muscle sounds for an experimental paradigm of musical performance.” ICMC 2011: “innovation : interaction : imagination”. Proceedings of the International Computer Music Conference (Huddersfield, UK: Centre for Research in New Music (CeReNeM) at the University of Huddersfield, 31 July – 5 August 2011).

_____. “Music for Flesh II: Informing interactive music performance with the viscerality of the body system.” NIME 2012. Proceedings of the 12th Conference on New Interfaces for Musical Expression (Ann Arbor MI, USA: University of Michigan at Ann Arbor, 21–23 May 2012). http://www.eecs.umich.edu/nime2012

Goodman, Steve. Sonic Warfare: Sound, Affect, and the Ecology of Fear. MIT Press, 2009.

Massumi, Brian. “The Autonomy of Affect.” Cultural Critique 31 (September 1995) “The Politics of Systems and Environments (2),” pp. 83–110.

_____. Parables for the Virtual: Movement, Affect, Sensation. Duke University Press, 2002.

Knapp, R. Benjamin and Hugh S. Lusted. “A Bioelectric Controller for Computer Music Applications.” Computer Music Journal 14/1 (Spring 1990) “New Performance Interfaces (1),” pp. 42–47.

NIST/SEMATECH. e-Handbook of Statistical Methods. June 2003, significantly updated in 2012. Available online at http://www.itl.nist.gov/div898/handbook [last accessed 9 July 2012]

Ryan, Joel. “Some Remarks on Musical Instrument Design at STEIM.” Contemporary Music Review 6/1 (1991) “Live Electronics,” pp. 3–17.