'Composing the now' - notes for a lecture -

on engagement with sonic time------------------------------------------------------------

through sensors, electronica, loudspeakers and ears--------------------------------


Michel Waisvisz, Amsterdam

for the occasion of 40 years IPEM symposium in Gent, october 2003




40 years ago the composing of electronic music was still done ahead of 'music time'.
The building of the sounds was done outside, or more precise: before 'music time'.
Its creation occupied a multitude of it's performance time.
Manual interaction was mainly setting switches, controlling sliding en rotating potentiometers, recording, mixing, cutting and splicing tapes and performance was an exhibition of a frozen stream of time.
Later when technology allowed for faster sound synthesis and processing we started calling the previous approach non-real-time.
Today we start to forget what is real-time; because we do not experience most computer processes as non-real-time anymore.
We think that we think and act in the now.

About the style of this text

This lecture reflects on composing the now' in today's electronic music practice.
Similar to many of the electronic music performances nowadays the lecture itself will be composed in the now at the time of the delivery.
So instead of writing this paper as a reflection of the thoughts at the time of writing I feel it is more appropriate to publish a list of selected and prepared thought bits, being a non- chronological collection of the cue sentences that will feed my thoughts during the lecture.
Some thoughts published here will possibly not be part of the lecture.
While others, not published here, might pop up during the lecture instead.
Traditional notes for lectures usually consist of a plan to organize linear time.
This cue list contains material that allows jump-connections in variable directions and allow for new alliances and flows during the lecture performance.
Recombine, grow, connect, mutate, explore, discover, intertwine, mate, feed, connect, grow more, decay, die, decay, feed, become food for new thought, compost, re-compose, compost.
As a reflection on composing electronic music 40 years ago and today: this is what composing is now.

Composing the now

Composing electronic music beyond the illusion of control.
Composing electronic music on stage in the now, with prepared computers that contain sounds, algorithms hooked up with sensors and the presence of a dedicated audience.
Involved performative composing of music with new integrated composition and performance instruments.
Illusion of control versus rules of engagement.
Algorithmic machine composing nudged and steered by the performing composer.
Sound communication instead of sonification of communication.
Composing by ear.
Composing is meta steering of performance actions.
Preparing a performance is not the same as composing, neither is it studying the piece and/or rehearsing; rather it is designing behavior and providing sounds and sound processing tool set ups; it is like preparing an operation; a lot will be known in advance but most of it can be different at the theatre.
We will see composers who re-compose their work every time they perform and some who try repeat their performance every time.
We will see new performers that act and engage fully bionically merged with their instruments.
This is not unlike traditional music instrument practice in a metaphorical sense.

Being in the now:

Another paradox like Zeno's.
The exact now will never be reached.
When we think about it we are always too soon, too late.
Happily being caught in a stream of unconsciousness?
Augmented awareness of music reality?
Hyperawareness of the sonic space and the moves one can make towards the sounds and the audience?
Tunneled awareness perceived as concentrated?
In terms of trying to de-construct moments closely around the now the loop can be a valuable tool.
We have experienced how it also can become the lazy mans composition tool.
Several times in music history of the 20th century there have been huge polemics about 'cyclic' versus 'non-cyclic'. Almost as severe as the discussions about 'particles' versus waves' in physics
Today in the looped-scene flow (everything non-loopy) is called ambient' and this term is made to contain many different categories.
In the 'electro acoustique' scene 'loop based pop' is often considered as 'composed by the software'.
I'm worried that if the more dogmatic approaches amongst these parties persist it can slow down interesting cross fertilization.
The loop is a very good microscope to study time and particularly the now. Its a fantastic manner to deconstruct the associated meaning of sound and reveal, or create other or new ones
Surprisingly repetition either puts our minds asleep, or heightens our awareness.
Cyclism makes us assume we do not need to anticipate change.
Put our scanning of the data stream to rest. This is sometimes perceived as a nice state of mind; that has room for 'other' thoughts and observations.
Technology invented the perfect loop in time: via clockworks, player piano, film-, tape-, computer memory loops, etc.
Many of us still have to deal with creating the perfect musical loop sequence.
The more flow oriented music often contains slow hidden loops.
People use the same technologies but with different pace and sometimes different ways to hide their making process.
Hopefully in the near future this subject of loops will be looked at in a more creative and less dogmatic way by all involved parties because it deals so much with the musical now in it's own special way!


There is an opinion that the absence of direct manual intervention creates machine music with a quality more closely related or even elevated to our 'mind processes' and 'nature' and even the 'cosmos'.
Others insist that the interaction of our physical body with electronic music instruments adds a musicality that goes beyond machine music; some even speak about the occurrence of musical magic caused by this physical interaction.
In my vision the magic lays in the engagement and the convergence of both our mind and body with electronic/physical instruments while interacting with other musicians preferably in the presence of an audience!
Physical engagement - touch - adds more data streams, back and forth between the performer and the instrument.
We do not understand the meaning of all these data streams and leaving out some of these streams has been empirically shown to lessen the perceived musical quality.
In my personal vision for electronic music instrument design I have almost always pragmatically opened as many as possible data channels and their feedback between my body and the instruments.
In the early eighties I formulated thoughts about the importance of forcing the performer to apply physical effort when playing sensor instruments. I assumed that also this effort factor was crucial in the transmission of musicality through electronic instruments.
Now I think the crucial aspect of perceived musicality is not the notion of effort itself, but what we feel and perceive of how the physical effort is managed by the performer.
This is also why laptop performance - where the performer is sort of hidden behind the screen is so un-engaging to the audience when played outside of a dance context.

A tiny bit of history

In the beginning of electronic music development there was a consistent presence of dedicated physical interfaces. When the music industry started to mass-produce synthesizers with the traditional organ keyboard as its main interface, these developments became largely ignored.
Small scale initiatives of individuals and smaller institutes / companies continued to develop gestural controllers.
Those who, at the time of IPEM's 20th birthday, advocated gestural control encountered resistance from the computer music community, but less from the electro-acoustic scene.
There was an almost 'classic' disdain for manual labor as if it would lower the quality of the musical aesthetics and infect the clarity of clean computed data streams with the subjective data of shaky finger movements.
Now scientific and industrial efforts have allowed the computer technology to get miniaturized to such a degree that the interface will soon 'swallow' its computer. Ergonomic and metaphoric motives will dominate interface design considerations totally.
In many branches of science like in space-, the war- and the medical theatres manual override of process control systems has been rediscovered as a necessary and more reliable way to control electro-mechanical instruments and tools.
Now gestural control is on the agenda of many electronic or computer music institutes.
However unlike in the industry a lot of the work is aimed at trying to embed the human 'system' into the musical system.
Like the slightly incalculable agent that the electronic music score needs to follow.
Or as the conductor who can mainly trigger the next preset and only control a limited amount of the parameters of the 'orchestra'.
Or as the amateur juggler that is given a virtual instrument a physical model that allows for refined virtual gestures and feed back but that produces music merely as a 'side effect'.
Or the dancer whose moves produce a heavy sound shadow that constantly follows each and every dancers move, without ever leaving the dancer space to be silent or giving the dancer room to respond or dialogue with the sound monster that is locked into the dancers motoric system by motion tracking.
Or the performer that with lots of dynamic movements plays back multitrack files and moves like every note has been crafted with intense effort and sacrifice at that specific moment of performance by his own hands, live on stage deeply dedicated and generously catering its audience.

Intention space

I propose 'intention space' as a notion that creates a calculable and logical model of the relationship between the performers musical intentions expressed through possible motoric movements. It's a model that works on creating sets of connected and sometimes overlapping sensing spaces that allow for defining combined physiologically and psychologically weighted layers and boundaries.
Instead of working with sensors as linear displacement analysts, sensors can be set up in various combinations in order to define the space which the performer occupies and modifies.
The data interpretation is based on correlating the analyzed data changes of the defined sensing spaces with musical synthesis or processing procedures that have been constructed in a symmetrical way.
The notion of symmetry between intention space and sonic space is crucial. Here we define artistically and technically what effort, gesture trajectory, gesture rhythm, gesture presence will mean in musical sense.
An example of musical gesture interpretation:
Throwing sound: Sensors detect during a short amount of time direction and speed of a hand gesture.
Lets say the system is defined to analyze this into one of 16 different possible outcomes.
Each possible outcome triggers the sending of a specific package of control data to the sound synthesis/processing system.
This can be for instance the triggering of a specific sound evolution
After this initial throwing of the sound the gesture is not interpreted unless it moves faster than a set threshold value.
Movement is then consistently analyzed in terms of speed and direction and this data is compared with reference values pre-designed to form conditional sets. If a conditional set is met then this will modify the ongoing sound process
Specific gestures can be programmed to stop the sound evolution or to trigger other sound processes
On a meta level while analyzing the data-stream and doing translations of the gesture flow one can also set-up the system to look at recognizable patterns in the gestures and use these for meta control.
Meta control can be setup for influencing more generic musical values, or even non-musical values:
Groove steering
Dynamic characterization
Spatial characterization
Stimulation of timbral unity of the various sound streams
Sound density control
Sound family modulation
In genetic sound synthesis algorithms: changes in the seeding of these processes
Opening/closing to other musicians audio or steering channels
Opening/closing audience response channels
Influencing sound projection characteristic
Light / image steering

Exciting future of gesture based instruments using EAP

The recent progress in development of electro active polymers (EAP) will drastically improve the physical qualities of gestural instruments.
Instruments that vibrate through the sound.
Instruments that are sensors, processors and loudspeakers at the same time.
Control surfaces that change shape in coherence with the change of the sound it plays.
Think of instrument that vibrates electro-mechanically when you touch it.
Physical response of the instrument and resulting sound is inspiring the performer to touch and engage with touch/music.
Trapped in the sounding and haptic perceivable groove.
Deliberately trapped.
Excited by the excitement.
Exiting the excitement.
Performer and instrument merge together into one electro-mechanical circuit.
Like with the crackle box in the seventies!

Loudspeakers all over

I expect that it will be possible to make EAP surfaces that vibrate rapidly and intense and thus can replace our present days loudspeakers.
Instead of the tunnels of sound that are beamed under high pressure towards us from today's concert stages we will be able to design spaces or tents with vibrating surfaces that sound all around us and that have distinct movable areas for specific sounds.
We will be able to move around in a vibrating environment bathing our ears in sound and sometimes laying down on surfaces that musically excite all our senses.


For next steps it is of primary importance to gather practical experience in a variety of music practices.
In order to be able to establish the minimum diversity needed for research it is conditional that the community working with gestural instruments should expand into as many directions as possible.
In order to stimulate this development we first must provide the communities with affordable sensitive intelligent toolkits for building instruments. So virtuosity can be developed by the community of new performers at home and on the road.
The developing of initiatives in an independent way is crucial for cultural diversity.
STEIM is introducing the OIK toolkit.
OIK is a low-cost and easy follow up to the high definition gestural input instruments that STEIM has developed over the years. OIK is a result of STEIM's research and development into ways of opening up and modifying game controllers and allow them to be changed into refined and sensitive gestural instruments to fulfill the growing demands by laptop performers. Whereas STEIM has traditionally developed platforms for complex multi-sensor data the OIK project allows for the development of simple and effective musical controllers using as little as one sensor. The ultimate goal of the OIK project is to provide artists with information and do-it-yourself kits that will allow them to create their own gestural performance instruments by hacking widely available game controllers in a technically simple and low cost process.


Images of early and recent developments will be shown during the lecture
This part of the lecture very often eats most of the lectures time
That's why this text is hopefully useful in providing info about what was not part of the lecture...

Ears: audience

Audience response is measurable in many senses
Traditional performers hear and include in their performance every move of an audience consciously or unconsciously. In the new performance halls with loud PA systems we could create instruments that augment the situational awareness of the performer. In other words amplify the presence of the audience to have their presence become more integrated in the performance. - Of course playing less loud will probably also be one of the new inventions of the near future.
These days I think about:
Also designing instruments that allow an engaged audience to participate in the performance.
Allowing the audience to seed and control algorithmic music entertainment machines.
Providing a circle of handheld gestural instruments so audience members play and play and become performers!



return to crackle