An Agent Approach to Working With Modular Synths

by Michael Spicer

Over the past few years, I have been creating electronic music with an approach based on the concept of autonomous agents. 1[1. See, for example the author’s contribution to eContact! 16.2 — Sound Art and Interactivity in Singapore, “Interactive Composing with Autonomous Agents.”] I find that this approach is especially suitable for creating performance-oriented patches on modular analogue synthesizers. The agent concept provides a high-level abstraction that provides a clear way to conceptually group modules together; this approach aids in managing complexity of performing with such an unstructured instrument.

The Agent Concept

This approach to working with modular synthesizers is based on the concept of autonomous artificial agents, which has been used to unify various concepts from Artificial Intelligence for many years (Russell and Norvig 2003). An autonomous agent can be thought of as an entity that can perceive its operating environment in some way — it receives “percepts” (in this case via control voltage inputs) — and has some ability to change or interact in that environment — it has and can make use of “effectors” (in this case by outputting either an audio signal or a control voltage). An agent has some internal mechanism or process that determines exactly how it will interact with the environment. This part of the agent, referred to as the “agent function”, can be implemented in a wide variety of ways, and it is this flexibility that makes the agent model such a useful abstraction. It provides a neat way to encapsulate many different compositional algorithms or decision-making strategies in one unifying design.

The agent concept helps us to organize complex modular synthesizer patches by mapping the component modules’ input and output sockets into percepts (inputs), the agent function (various voltage generators and processors) and effectors (outputs). A performance system would typically consist of the available modules deployed so as to constitute a number of agents that operate in parallel and can interact in various ways. Creating this type of performance system is analogous to creating an environment consisting of a number of performer agents and providing some way for a human performer to influence their output.

The Agent Categories of Modular Patches

I group agent designs for modular synthesizers into categories that are differentiated by their degree of autonomy and their predictability. A common attribute of all of the agents is they receive input (percepts) from their environment via control voltage inputs, knobs, switches or external controllers, and they all contribute to their environment via their various control voltage or audio outputs (effectors). The fact that analogue synthesizer modules provide so many inputs and outputs means that it is very easy to experiment with the many ways agents can interact. The core of the agent is the agent function, which decides how the agent responds to its environment and largely determines which category an agent design would fall into.

The agent designs in my work with modular synthesizers can be broadly classified into two high-level categories: standalone generative patches and interactive patches. These in turn can be further subdivided. Generative patches can be grouped into those that involve random processes and those that are cyclic. Interactive patches can be similarly divided into those that feature a random element and those that are largely deterministic. This combination of strategies allows a composer / performer to create a collection of agents that exhibit distinct and complementary musical characteristics. This can help create intelligibility in the resulting musical output, allowing each agent to carve its own space in the sonic spectrum.

Agent Built Around Generative Patches With Random Elements

Generative synthesizer patches autonomously generate musical material without necessarily being explicitly controlled by a human performer. The only control available to the performer is their output level. This category of generative patches is built around random processes and they exhibit a degree of unpredictability in their output, although the musical output could exhibit clear tendencies. Most of these agents are built upon noise generators. These can be easily created with analogue electronic circuits and they produce truly random signals, unlike the pseudo-random generators commonly implemented in software synthesis systems.

Probably the most common way to create a random generative patch involves combining a noise generator with a sample and hold module. Every time a new random value is required, a trigger signal is sent to the sample and hold module, which measures the instantaneous noise signal at that time and outputs this value. Shaping the noise generator’s spectral density with a filter can produce various types of random control voltages and this provides an intuitive way for a performer to intuitively manipulate the shape of the resultant probability density function (Strange 1983). Various manufacturers have packaged specific modules that implement variations of this idea in distinct modules since the 1960s. Perhaps the most influential of these is the Buchla 266 Source Of Uncertainty (SOU) module, which was released around 1970. Three modern modules inspired by the Buchla 266 are the Doepfer A-149-1 Quantized / Stored Random Voltages, the Richter Noisering and the Make Noise Wiard Wogglebug. These modules, along with basic noise generator / sample-and-hold modules and shift registers, form the basis of the agent functions in my realizations of this agent category.

A very simple example of a generative patch that generates its pitch material by periodically sampling filtered noise is demonstrated in Video 1. In this case, the random voltages are quantized to restrict the pitch material to a melodic minor scale and this voltage controls a digital oscillator that implements the Karplus-Strong plucked string algorithm. Details of the sound produced, such as its attack characteristic, are altered by percepts, in this case via the DIG RND VOL section of the Doepfer A-149-1 module. The resulting sounds are then shaped into phrases, via a slowly rising then decaying envelope, which is initiated by an external percept, derived in this case from a random value from the Stored Random Voltages module.

Agent Built Around Cyclic Generative Patches

It is possible to create autonomous generative agents that produce a repetitive output. These can be constructed in a number of ways and are useful where cyclic patterns form the foundation of the music. With the modular synthesizers, I use two approaches to create cyclic agents. One approach is built upon a step-time sequencer module. In this type of module, a limited number of discrete voltages are pre-set, and are sequentially stepped through when triggered to create the output. When the last voltage is read, the system loops back and then reads the first one again. There are many variations of this technique, and modern sequencers often have a number of options that allow the performer to easily modify the behaviour of the module. The other approach involves creating a cyclic control voltage from one or more low frequency oscillators. The outputs of the oscillators may be combined and further processed in many ways. They may be scaled, inverted, waveshaped, sampled and quantized. The resulting control voltage is then applied to control some aspect of the musical output, typically pitch and note timing. Quite complex results can be obtained, and these have the advantage that they can easily be modified to produce new patterns simply by altering an oscillator frequency.

Video 2 shows an extremely simple cyclic agent. Two LFOs in the Maths module are mixed together, quantized and then applied to a simple synth voice, consisting of the Dixie II oscillator and the Optomix Low Pass Gate. The controller inputs of the LFOs and the controls affecting how they are combined can be controlled by precepts to create variation. This process is normally accomplished by processing the percept control voltages using a combination of voltage mixers and voltage controlled amplifiers, but for clarity, in this example, these are simply adjusted by hand to illustrate some of the variations possible.

Interactive Agents With Random Elements

Agents may be constructed in such a way that the performer can alter the output of generative patches in real time. This type of agent normally exhibits some degree of predictability and allows the performer to effect high-level control over some perceivable aspect of the musical output, such as determining the pitch range or average duration, while other details exhibit some unpredictability. This performance element can vary in complexity, from simply adjusting the available knobs, sliders and switches to the possibility of applying a performance of controller to any parameter or group of parameters. This results in an agent that is an interactive performance system, where the musical output is the result of collaboration between the generative system and the performer. The level of interactive control can vary between working at the intra-note level of detail, providing a means of imparting performance expression, up to shaping high-level musical structural elements where a performer may approximately specify elements such as tessitura, pitch contour or note rate in real time. A huge range of controller options is available because the synthesizer modules simply need a control voltage. This means that virtually any controller can be useful, such as simple homemade battery-powered devices that directly generate control voltages, which can easily be constructed. Alternatively, it is quite simple to adapt commercially available MIDI controllers and/or build sophisticated arrays of sensors that are processed to produce control voltages. I have found that a good way to transform controller output information into control voltages is with an Arduino microcontroller.

A very simple example of an interactive agent with random elements is shown in Video 3. The core of the agent is the Stored Random Voltage module and an oscillator, enhanced here with a quantizer that enables the pitch material to be constrained to scale or chord tones; this can be useful when working in a tonal music context. In this case the DPO dual oscillator can constitute the entire synth voice, because its second oscillator implements the wave folding technique. This enables output level and timbre to be shaped by a percept that delivers a control voltage to its Fold input. The output of the Pressure Points’ touchplates supplies this control voltage, with the result that the patch produces an output whose amplitude and timbre are determined by the player’s finger pressure. The pressure control voltage also applies a small amount of vibrato from the DPO first oscillator, whose frequency varies slightly with each note. The main random element in this example is the basic pitch of each note. The Pressure Points gate output is used as a clock for the Stored Random Voltage module, so each time a pressure sensor is touched, a new pitch control voltage is generated. Using three pressure sensors enhances this example, allowing the player to slightly bias the system towards producing low-, middle- or high-pitched output.

Deterministic Interactive Agents

Deterministic agents exhibit a low degree of autonomy and provide a predictable output for a given performance input. Most common synthesizer patches can be included in this category. The output of the agent is largely determined by the precepts derived by the human performer, such as playing a particular note on a keyboard at a particular time, and the sound evolves in a predictable way. It is possible that there are distinctive aspects of the sound that are autonomously generated by the patch, but they all have an element of predictability.

“Momentary Diversions”

A musical example created with a modular analogue synthesizer and the agent-based approach is the ongoing series of pieces Momentary Diversions. Each Momentary Diversion is an improvisation environment for modular analogue synthesizer. Each instance of the piece is created in real time, and there are no overdubs allowed. They are mainly realised in a studio, but I have also performed the work in public on several occasions, with more performances planned. The available synthesizer modules are patched so as to instantiate at least six autonomous agents, and are set up so that the musical contribution of each agent exhibits a particular characteristic that is complementary to the compositional strategies implemented in the other agents. Each agent contributes an individual sonic layer to the musical texture. The final musical output that is heard by the audience is determined in real time by controlling the output level of each agent via a mixer.

Some examples of the compositional strategies in a typical instance of Momentary Diversions include:

• Creating low drones that exhibit complex movement through alterations of the harmonic content (Audio 2 at 1:22; Audio 3 at 0:44);
• Creating a number of short sounds that collectively exhibit sense of unity via some random process controlling a perceivable element (opening of Audio 1, 2 and 3);
• Creating layers that exhibit cyclic properties manifest at two or more hierarchal levels (Audio 2 at 3:30, clicking sound; Audio 3 at 1:53, siren-like sound; Audio 4 at 1:50, foreground sound; Audio 4 at 4:00, marimba-like line);
• Creating interlocking rhythm layers (Audio 1 at 3:33, two layers of echoed percussive sounds; Audio 4 at 0:42 and 3:15, percussive sounds);
• Creating one or more layers of sustained gliding tones (opening of Audio 4).

In addition, at least one melodic, or foreground layer is created by an interactive agent patch that has a significant random element used in conjunction with a continuous controller of some kind, thereby providing a two-way interaction between the agent and the human performer. The human will not know precisely how the agent will respond before the gesture takes place. I often use a pressure-sensitive touch sensor system, similar to those developed by Serge and Buchla in the 1970s. This produces a gate signal every time it is touched, plus a control voltage that is proportional to the surface area of the finger in contact with the surface (Video 3). The gate signal triggers the generation of two new random control voltages that control the frequencies of two oscillators. The outputs of these two oscillators modulate each other in some way, producing a different complex timbre for each gesture. The pressure sensor controls the volume of this signal and the result is an expressive system whose output is a collaboration between the agent and the human performer. Typically additional melodic material will be added using whatever instruments I have available. This can be a conventional instrument played by another musician, creating an ensemble improvisation context.

Conclusion

Adapting an autonomous agent approach to performing with modular synthesizers helps to manage the inherent complexity of the task. By conceiving the synthesizer as a distinct ensemble of autonomous entities, each made up of several physical synthesizer modules, it is easy to set up complex patches that are intelligible to the performer, have a degree of autonomy and provide complimentary musical material. Because each agent has the same a common basic structure (percepts, agent function and actuators) it is easier to remember what has been set up, and there is less chance of being overwhelmed by the tangle of patch chords. This framework allows a performer to offload some of the detail and help keep the focus on creating a dynamic and coherent performance.

Bibliography

Russell, Stuart and Peter Norvig. Artificial Intelligence. New Jersey: Prentice Hall, 2003.

Strange, Allen. Electronic Music Systems, Techniques and Controls. 2nd edition. Dubuque IA: Wm. C. Brown Company Publishers, 1983.