The following text is Waisvisz response to questions formulated by the gestural controllers group at the IRCAM. The same questions were also sent to: Max Mathews, Jean-Claude Risset, Chris Chafe, William Buxton, Tod Machover, Don Buchla, Robert Moog andLaetitia Sonami.

The following text, and those of the others, will soon be published on the WWW and a CD-rom. The URL of that site will be added here as soon it's available.

Amsterdam september 1999


>Subject: Round Table Gestural Controllers




>1.1. Could you shortly summarize when and why you first became

>interested in using electronic controllers for performance/composition?

It started at home, together with my brother, I was four years old, playing with my fathers short-wave radios, tuning into the telexzooms, the squeaks and squeals and distorted voices from afar.

We used to play warplane flights and space journeys with these instruments, for hours and hours, and fabricated bigger dials to tune in more precisely. Also the antennas were manipulated, metal objects moved near the spools, we blew air from the end of the vacuum cleaner towards the loudspeakers and moved bottles and pipes in front of the speakers in order to create some kind of ‘flanging’.

My father built a Theremin when I was 14. At that time I was listening, to, amongst others, early records of electronic music; also with my brother we explored record players; not only did we discover ‘scratching’ and looping but we also were changing the surface of the record with washing powder, syrup, glue and were carving and filling it with fine metallic grit in order to obtain ‘granulated’ and chopped versions of the original recordings.

I started giving concerts at 16. By that time I played with an old cither that was prepared with all kind of objects. The sounds were picked up with electro-magnetic guitar microphones. I also played bass in a ‘happening’ band by hitting and pulling a heavily amplified knitting needle through a wooden frame.

At 18 composer Dick Raaijmakers let me literally in through the back door of the electronic music studio of the Royal Conservatory in The Hague.

I started making tape music. But instead of playing them back in the normal way I developed a tape performance instrument: ‘The TapePuller’ - two stands with tape heads for two identical tapeloops and the performer sitting in the middle and ‘rowing’ the tapes; and synchronised loudness control with two footpedals.

The studio had an early Moog cabinet synthesiser. I used to hang the connecting wires in tea and other wet objects and experimented controlling the sounds by pouring the tea on wired clothes. I ended up buying this wonderful Putney VC3 synthesiser (Zinoviefs design). Here I opened the back and started touching and connecting the wires with my bare hands. This gave subtle control over wild timbres and was the inspiration for what later became the CrackleBoxes. Philosophically it was interesting as a human being to become a variable electronic conductor/resistor, and a reasoning and sensing part of the instrument one plays.

In the early seventies, based on this integrated principle, I started working on the development of Crackle boxes.These were finally built and sold through STEIM in the mid seventies.

Also in the mid seventies, I created the first ‘playable exhibitions’ with all sort of Crackle objects and created many music theatre performances with these objects.

In 1984 I gave the first concert with what is now known as ‘The Hands’ in The Concertgebouw in Amsterdam.

In 1988 I started working on The Web (the first Electro-mechanical computer for the programmable distribution of simple finger movements into a networked set of generators for complex music control signals) .

Since 1985 I have been developing software instruments, in collaboration with Frank Baldé: the Lick Machine, Sam, LiSa.

At present I enjoy not modifying The Hands too much, and concentrate as much as possible on the creation of music in various collaborations and solo-projects.

At STEIM we are preparing a big travelling ‘playable exhibition’ with many new instruments. Some of them were presented already at the Touch festival late 1998.



>1.2. What kind of performance/composition using gestural controllers are

>you doing today?

Mainly The Hands and occasionally The Web. These are used for composing and performing a series of compositions, theatrical pieces and dance pieces.

Spin offs and variations of these instruments are being developed for an exhibition of instrumental objects that will travel the years to come and give the audience a possibility to play and perform en discover these new, and some by now ‘old’, approaches. With these instruments I have also started to give workshops for music, theatre and puppeteer professionals and students.



>1.3. In your opinion, would you consider the current state of these

>gestural controllers satisfactory. If not, what are the shortcomings?

I like to answer this question in a generic way first.

A growing number of researchers/composers/performers work with gestural controllers but to my astonishment I hardly see a consistent development of systematic thought on the interpretation of gesture into music, and the notion of musical feed-back into gesture.

When I lecture about the work we have done at STEIM in this field I'm regularly confronted with responses like: 'but that is all very personal'.

Fortunately in music a lot is 'personal', but ‘the personal’ is not an analytically impenetrable romantic chaos of emotions, feelings and ghosts. One can analyse and create distinct relationships between the character changes of a gesture, and the change of musical content - and context - in a way that ones musical intentions are clearly grasped by listeners. This is of course limited to the context of a certain cultural scene, as is the case with the new music scene anyway. But interestingly enough in recent years I have seen the gestural approach in the independent music scene increasingly reaching across cultural borderlines.

Many of the institutional research efforts however still focus on the recycling of traditional instrumental metaphors and are reluctant to ‘touch the real thing’.

As with most of the new digital media technologies it seems that its creators cannot envision more than a meager recreation of existing concepts, imitation of analogue worlds. New Media, but with old content, in even older contexts.

I do not ignore the fact that the analogue world has a lot to offer, but in our field with the help of new technologies (digital and analogue !) you have the freedom to create any gestural relationship with a vast area of sound. New instrumental objects can be envisioned. One can go beyond the recreation of a sort of clarinet or bongo or baton or a supercharged cello or even joystick.

At present the music instrument industry also shows no guts at all.

I met wonderful researchers from several companies, but they never seem to get their ideas through the filter of the marketing people.

About my own experiences with gestural controllers I can only say that I fight with them most of the time. That’s something that almost every instrumentalist will tell. But if you are in the position to be able to design and build your own instruments, and so many interesting technologies pop up almost weekly, you are tempted to change/improve your instrument all the time. This adds another conflict: you never get to master your instrument perfectly even though the instrument gets better (?) all the time.

The only solution that worked for me is to freeze tech development for a period of sometimes nearly two years, and than exclusively compose, perform and explore/exploit its limits.

I suppose that in the years to come the widespread belief in the benefits of unlimited technological progress will slow down. The fact that technology does not really improve our wisdom and does not solve the basic problems that humanity will see itself confronted with will be recognised by many. In pop music, even in the hard core underground of new techno, there is already a longing to return to folk or folk emulation. In the new media culture the quest for new technologies as an inspiration for our living culture will probably lose its present (highly virtual) aura, and people will, as has happened before, crystallize progress into ‘standards’… whereafter some completely new, or very old, medium will capture our imagination.





>2.1. In your opinion, what makes an electronic controller expressive?

I have a problem to associate a gestural controller in the first place with ‘more expression’. A gestural controller is not just an instrument with more subtle control and therefore more ‘expressive’.

Expression is a notion that historically strongly is related to melody. It is not precisely defined. It is often seen as what the performer adds to the melody, a ‘personal touch from the hart’ to use the romantic description. It’s as if it adds an emotional context to melody, sometimes almost as a comment. Some simply say: ‘It is what makes music musical’ - to avoid further discussion!

This century however has seen an immense rise in the notion of ‘sound’. Electro acoustic music has defined itself often as ‘timbral music’, or a ‘sonic continuum’. In jazz and pop it is the ‘sound’ of a musician that counts. ‘Sample’ culture has turned working with ‘sounds’, and the notion of sound as visions, into a highly popular practice.

I think that the main quality of gestural controllers lies in being able to deal with the expression of these new approaches of sound. The gestural controller can be made sufficiently complex in its control diversity, and therefore can optimally deal with expressive timbral control. And with this I mean it can provide the translation of physical intentions of the composer/performer, ranging form utmost fragility to outstanding trance, into a set of related timbral trajectories.

The translation of parameters is often done in a crude one-to-one way. However it is possible to create a network of relationships between a group of control signal generators and distribute a simple finger movement in a complex set of control signals to the music synthesis system. This area is often considered as a technical area. I believe the algorithm for the translation of sensor data into music control data is a major artistic area; the definition of these relationships is part of the composition of a piece. Here is where one defines the expression field for the performer, which is of great influence on how the piece will be perceived.

One of the freedoms in gestural control design is the fact that one can design a controller without being restricted by physical characteristics of the sound synthesis system. Unlike in most of the traditional instruments there is not an unbreakable mechanical relationship between the ‘keyboard’ and the sound producing ‘mechanism’ anymore. Any object filled with sensors can do the job. One is free to implement physical effort at a place in the system where this is not needed at all. Constructing a sequence of changing relationships between the physical object and the musical result can become a composition in itself. The instrument that presents itself toward the performer as a ‘dynamically mapped keyboard’ becomes a character with its own rules, and as a new form of music theatre: ready to be mastered, partnered, domesticated or let free by the performer.

In my work I have always designed instruments that demand a considerable degree of physical effort to be played and at the same time they are able to convey the slightest trembling of the hand. These tremblings are, during a concentrated performance, not just errors, but an integral part of the muscular/mental effort pattern that lays at the base what is perceived as musical expression. Music in a pure conceptual format is only understandable by the ones who know the concepts. Music that contains the physical expression of a performer is recognisable by a larger group through that expressive mediation of the performer.

During inspired performances I have experienced that a mental/physical state can emerge where a fast closed loop establishes itself between the musical intention, the muscular effort and actions, the mechanical response and the sonic feed back and the perception of this whole loop. This happens so fast that one seems to act immediately in sound and not in ‘terms of sound’ and not in terms of ‘control’. Composition/performance melt into a single state of emerging - timbral - expression .

Finally I would like to mention the intriguing addition of ‘expression’ to meta control through the use of gestural controllers.

Meta control is commonly implemented in algorithmical composition, semi-automated conductor systems and other complex control area’s such as: theatre lighting or the manipulation of multi-media robots.

To connect gestural controllers to meta control algorithms in music performance is relatively new. To use the composers/performers, traditionally low-level, motoric behaviour to control higher levels of musical organisation (composition, phrasing etc) will wildly expand the notion of expression.

Whether this will lead to the equivalent playing a harpsichord with a shovel or an angel conducting the Crystal Orchestra of the Spheres with the twinkling tip of a fine finger is to be seen.



>2.2. What would it take for a new controller to become a ‘standard’

>device so that a repertoire could be developed for it (like the Theremin

>and Martenot in the past)?

I'm cannot relate to the assumption that lies at the base of this question.

Standards are shared illusions with temporary value within confined groups of people in which the persons who hold power have great influence on what is a standard or not.

Standards are not values, but trendy subjectivities that quite often occur at the end of periods of progression.

I do not think that the Theremin or the Martenot can be considered standards. Do we tend to call them so because more than one person performed written melodic scores with them? If yes: many more instruments should apply: a sine wave generator in the first place.

I like the situation in which there are no standard instruments. I encourage musicians to develop their own sets, or their own variations. I like to encourage composers to write for persons and their instruments, instead for instruments exclusively; or better to become performers as well.

It’s needed to compose for performance, and not for the ‘soul’ of the score.

I have been amused to hear young composition students mutter about the ‘unstable and short life cycles’ of new gestural controllers. They hadn’t yet made a single piece for a present day audience, but were already worried about the hereafter.

I enjoy the fact that our presence in culture is just temporary. To work a whole life in order to be remembered by others is not my cup of tea. ‘Ars longa, vita brevis’ is for carved stone and other slow declining materials, but not for human communication. It helps me tremendously to be biased and short-sighted about this.



>2.3. Can new electronic controllers replace traditional instruments as

>viable performance tools? Should they?

One just has to study the conflicts Niklaus Harnoncourt encountered when using what he considered ‘authentic’ instruments to realise that the notion of 'traditional instruments’ is not a very stable one. Traditional instruments have been recreated all the time and changed according to trends that depict what is ‘authentic’ or ‘just’ at that time. Even the grand piano underwent changes not long ago. The concepts of traditional instruments have always been, and are, dynamic.

Many of the instruments that we now call traditional are known from the culture of the symphony orchestra. The symphony orchestra for a while became a metaphor of a well organised, stable, society. The instruments all had their place and the instrumentalists had specific roles in that organisation.

Society is not that stable anymore and definitely not organised as the symphony orchestra still is.

There are now many new ways, beyond the symphony orchestra, in which a composer/musician can independently develop his/her position in the musical world.

The musician of today, equipped with gestural controllers and sound synthesis instruments, and good amplifiers, can produce the sound impact of a symphony orchestra with the wave of a hand. The musician is now free to change musical roles all the time. An ensemble of musicians can play one instrument together, musicians can re-use each others sounds, and compose/perform networked on stage or in different places in the world at the same time.

Instead of becoming ‘viable performance tools’ in the traditional sense, the new tools do not need to prove themselves in terms of the cultures of the past. They should be judged by there function in our present days new cultures.



>2.4. Do you ever use gestural controllers outside of performance

>situation, for instance as compositional tools in the studio?

I use them as:

* Compositional tools; to play and organise, save and reselect and order sound sequences.

* Navigation tools; to find my way in soundlibraries on stage and in the studio

* Programming tools for editing sounds in MIDI synthesizers and changing the parameters of for example the LiSa software

* I experimented with them for writing emails while in hotel rooms (using macro’s).

* I have used software to mix-up the key relationship with the MIDI commands and to play without knowing what the result will be. This I used in the process of: ‘Oracling’; just playing, without knowing the exact output beforehand and recording and using the good bits as an inspiration for new pieces.

* At STEIM we also created instruments for disabled people and for a wide variety of therapeutic projects.

* I find specially conceived low-threshold controllers extremely helpful in education: the web, the midi conductor are successfully being used to help students through a hands-on approach; intuitively find their way in the world of sounds.



>2.5. How would you characterise their role in this context?

We were able to develop a more natural ‘feeling’ This creates a more intimate relationship with the instrument. People experienced this as ‘warm machines’

The controllers that I use also liberate me from the usual chair/desktop setting. It is great to leave the ‘desktop world’ and to be able to walk while you think and work.

Also: being able to change the functionality of a tool gives you a surprising and inspiring way to reflect on your work methods in general. Changing old gestures can be liberating, confusing and forces one to review its habits. Lazy as I can be, this has for a long time proven to be a good strategy to wake up and rethink the strategies of work and life.




>3.1. Regarding the development of input devices, what can gestural

>control gain from specific fields such as HCI, ergonomics, experimental

>psychology, cognition?

With some exceptions it has struck me in conversations with specialists from some of the sciences mentioned above that they often merely verbalise a number of aspects of what I already knew ‘physically’. It was understandably the knowledge of the spectator that they conveyed.

In recent years I have had the desire to learn about the inside of performance, to deepen the knowledge of the maker, I wanted to concentrate on the understanding of the multiple forms of conciousness during performance. This is an inside journey. The vision of the spectator on this acitivity can be analytic - in the wrong way by pulling the performer back to much to the outside appearance of performance. For this purpose the immediate spontaneous response from the spectator is more valid, more connected with the experience the performer just had. Scientists, (for good scientific reasons) very often have lost that ability in their professional role. Friends in the HCI sector however have followed our work with great interest, notwithstanding my relatively negative attitude.

These remarks here are not meant as generic as they may sound; they are to be understood in the context of the work I undertake these years. More into the music, integrating technology in an intuitive preparation of a new series of works.

About ergonomics: ergonomics for musical instruments is a paradox. We need instruments as obstacles, as challenges. Mentally and physically mastering these is one of the main ingredients of musical tension in performance.

No virtuoso violin without a pain in the neck.

I cannot be other than very conservative at this point. I do not get any excitement from a performer numbly toddling a mouse or occasionally pushing a key awkwardly posted in a chair. Neither do I enjoy the by now traditional bunch of DJ's jumping around and at regular intervals just pushing a button or hitting a slider.

I love to see a performer work him/herself up into that decisive set of gestures; like a billiard master after a long sequence of preparations successfully hitting a whole field of conditions with refined, complex but unidirectional effort.

I’m happy to see that in the world of electronic music performance the practice is improving through the growing use, and sophistication, of gestural controllers.

If some institute will foster the development of ergonomics towards ‘the ergonomics of desirable effort in musical obstacles’, I will generously contribute my experiences.



>3.2. Is there any sense in adapting scientific methodologies from other

>disciplines like HCI, etc., for developing controllers, or do musical

>controllers require idiosyncratic approaches?

The people that I learned most from were: puppeteers and some sports people.

Puppeteers deal with gesture as a life-giving force all the time. Modern puppeteers are able to give life to the simplest objects by finding ways to let the objects work themselves and just interfering with gesture where it is needed. I have always unconsciously treated my sounds as being my puppets, until two years ago at the Institute International de La Marionette at Charleville-Mezierre (Fr) puppeteers made me aware of our mutual interests. Now I know my sounds are my puppets; and I have to share my life with them. I will start collaborating with puppeteers and am looking forward to the results of this collaboration.

Some sportspeople have given me insight in concentration matters. Especially the effects of concentration and over-concentration (temporary stress) on muscle control. I have loved the old golfer who explained me that in order to make a natural swing (unobstructed by muscular stress) he used to sing while hitting the balls. This led to a distraction and through that: the lowering of unconsious and disturbing muscle activity. This gives such a nice insight in the effects of the vocal side-acitivites during performance of for instance Glen Gould and Charles Mingus (he used to talk a lot while playing).

I tend to avoid the scientific methodologies in periods where my main activity is composing/performing because of the way they translate in practical advice. Scientific advice has often come in the form of lists of facts and derived rule-like strategies. Preparing oneself for work in the mental/physical performance area demands a method of communication that is geared towards improving musical awareness and building up sonic and interhuman conciousness. The old greek priest running the business at Delphi would have been the right coach. But if he would be still alive he would probabably have been bought up, cloned, ‘scientifically’ reprogrammed with and sold to the masses by a company in Redmond.





>4.1. Are there any points that we haven’t covered? If so, please address

>at your leisure.

The more real interactivity is added to the systems that we live with in daily life the more we have to deal with the notions of performance. Every ‘user’ becomes a performer somewhere, whether it is dealing with coffee machines, installations in museums, setting the clockradio before falling asleep, buying tickets or browsing the internet on a mobile phone in public.

The music community has built up a tremendous amount of knowledge about performance and the relationship composing/performing, especially the transferring of emotion through sensitive instruments. Many of the music research institutes and smaller organisations are increasingly confronted with bad financial support and/or budget cuts. Some have to fear for their existence.

I think this is a tragic paradox and I hope an awareness about the value of this knowledge will come in time.


I like the emergence of a new generation of performers. At STEIM 4 out of 10 of the new applications for research en development projects come out of this group, somewhere growing between academic/electro acoustic and DJ-culture. With a strategy to be authentic, while maintaining a connection to a certain ‘scene’. Searching without dogmas in the whole gamut of electronic music instruments in order to stick together their own set.

They deal with attitude; conceptualism is over. Their goal is to convey intention and simply use the tool that connects with their attitude and intention. Almost like wearing the right gyms.

So: gone is the systematic analytical approach towards the development of generic instruments. There is historical awareness, but preferably not beyond 40 years back. The instruments can be new, or deadly old. They may even not work properly. They might be merged with another instrument; they can be cheap mass production gadget instruments, etc.

Amongst them is a growing interest in gestural controllers. They want to move in front of the (DJ) table; stand in the light. And add sweat to the performance like some of their dancing audiences do.

And they want to move also to the concert situation; middle size halls in non-art buildings and friends who listen. Becoming international stars for small audiences through mediation via the Internet. Have the appropriate gear, not necessarily the newest, fastest or most expensive gear.

It's as if the music finally starts to count.



Michel Waisvisz, Amsterdam September 1999