Piano+: An Approach towards a Performance System Used within Free Improvisation more

Piano+: An Approach towards a Performance System Used within Free Improvisation Sebastian Lexer ABSTRACT PRENOTE This article primarily references non-idiomatic improvisations as described by Derek Bailey [1]. On a technical level I will not only refer to the terminology of “convergent” and “divergent” parameter mapping introduced in Wanderley (2000) [2] but also distinguish between two main categories of controller sources: (1) “direct controls,” usually MIDI controllers, etc.—within this category the intent of data input can be implied, therefore specific values are set and data is usually only generated when changes occur (event-driven controls); and (2) “indirect controls,” which can be generated from data and signal streams and are usually continuously clocked, that is, gestural information extracted from sensors and utilizing analysis techniques (usually audio analysis such as amplitude and pitch analysis) [3]. PHILOSOPHICAL CONSIDERATIONS The idea of the “absolute” as described by Hegel has continued to govern and influence musical performance over the last century. Musicians strive in general for an absolute rendition and execution in performance, an approach that has survived during the past century, despite substantial scrutiny of and challenges to the definition of music and its structures. Audio technology during the same period helped push performance to its extremes, facilitating the generation (rather than the preservation) of performances for a posterity in which virtuosic and technical precision and interpretative finesse excel. Mirroring concerns and concepts discussed in several writings on improvisation [4], Giorgio Agamben’s [5] discourse on potentiality provides an additional philosophical avenue away from the idea of the perfectly rendered and executed performance: The “absolute” is a firm reference to the past (“like never before”) and simultaneously implies a claim for future validity (“like never again”). For Agamben, this “corresponds to [an] attempt to absolve the subject [the performer] of [his/her] necessary relation to the event” [6]. Following Heidegger’s focus on the “event” (Ereignis) that diminishes references outside the present, a performance is placed firmly into the here and now and involves a reflexive form, “a departure from the self and a return to the self” [7]. Agamben also stresses that a relation to “what is proper to the subject” is an essential part of “grasp[ing] the very movement of pure temporality and pure Being beyond what is temporalized and said in actual discourse” [8]. his article explores the author’s strategy for developing a computer performance system designed for free improvisation with acoustic instruments following a nonidiomatic approach. Philosophical considerations on potentiality and personal and social space and research into the psychology of motivation and behavior have inspired and enabled a different approach to integrating technology with improvisation. The technical realization of a parameter space, in particular utilizing contingent behavior emerging from the convergent mapping of a mixture of controller types, has proven effective for the spontaneous creative decision making required to extend the sonic potential of an acoustic piano while minimizing direct computer operation, as applied regularly in practice by the author. T POTENTIALITY— POTENTIALITY SPACE Potentiality is defined through an awareness of incompletion, in other words, potentiality maintains itself in relation to its own privation. Actualization erases potentiality. A finished performance and a recording are the testimony of the actualization. To escape uniformity through ceaseless investigation, to preserve progress through continuous exploration and to Sebastian Lexer (musician, lecturer, student), Goldsmiths, University of London, Lewisham Way, New Cross, London SE14 6NW, U.K. E-mail: <s.lexer@gold.ac.uk>. Fig. 1. Standard set. C2 and C3 (i.e. MIDI faders) manipulate the minimum and maximum of continuous data stream C1 (i.e. acceleration sensor). (© S. Lexer) ©2010 ISAST LEONARDO MUSIC JOURNAL, Vol. 20, pp. 41–46, 2010 41 Fig. 2. Double controller set. C1 can be used to switch and mix the data of two continuous controller streams C2 (i.e. a Continuous Time Recurrent Neural Network (Ctrnn) [27]) and C3 (i.e. amplitude envelop). (© S. Lexer) feel the freedom to strive to be different is to be capable of one’s own impotentiality. These qualities are fundamental to improvisation that goes beyond the freedom “to simply have the power to do this or that thing [or to] simply to have the power to refuse to do this or that thing” according to one’s moods and intuitive responses. In contrast, an intrinsic quality of human nature is to be “capable of their own impotentiality. The greatness of human potentiality is measured in the abyss of human impotentiality” [9]. One is reminded of Prévost’s concept of self-invention [10]: Performance as an all-encompassing and inclusive event emerging from the sonic potential of the instrument via the performer’s approaches, processes and responses through which s/he develops a sense of what may be possible. This potential is embedded and constituted within the relationship between the performer and the instrument. Thus, it can be argued that the performer and instrument form an intrinsic unit within which previously acquired skills, positive and negative experiences, intuitively felt possibilities and limitations are manifested through the performer’s personality, motivation and creativity. This highly complex constellation of human imagination and instrumental possibilities is conceptualized within a potentiality space: A space populated with potential approaches, processes and responses. The metaphor of spheres enables us to consider the continuously changing relationships of the content of this space, because any relationship can be seen within a continuum and is in constant flux. This metaphor avoids the objectification of human activity into an algorithmic construct, which would only attempt to model a specific response. The concept of potential space retains a fundamental flexibility as it does not suggest linear hierarchies and orders, but rather intertwined constellations of varying degrees. The entire activity of improvisation is within a continuous “training instead of rehearsal” [11], where the individual has gained experiences that have altered the person in some way, although quite obviously the person remains physically the same. The use of a spherical metaphor allows a unifying terminology and approach in language for theoretical, technical and practical considerations. Such a metaphor is capable of describing relevant concerns and issues emerging from combining improvised performance activities and electronics. It appears to withstand the traps of linear descriptions and mathematically inspired approximations of processes. This approach owes much to recent philosophy developed by Peter Sloterdijk [12], in particular his metaphor of bubbles, to describe the creation of a personal “inner space” (Innenraum) encompassing the individual. Human motivation, intent, experiences, education, traditions and culture are described as furnishing our inner space. APPROACH TO IMPROVISATION— NON-LINEARITY As a consequence, improvisations are considered as emerging from non-linear thought processes, contrary to various improvisation models described over recent years that rely heavily on a serial chain of sensory, cognitive and motor loops. Theories of improvisation create direct links between listening and a resulting activity. Simplified models [13] reduce the activity into causalities within serial chains of sensory, cognitive and motor loops. Any approximation of a theoretical model requires the establishment of a network of influences and feedback (i.e. culture, tradition, education and memory). With such an attempt to create a network of influences, the reductionist discourse fails to convince, as too many facets of experiences within the practice of improvisation are marginalized or even ignored. Cognitive processes appear to be far too complex to allow themselves to be simplified by such models. Sarath offers interesting additions to introducing different modes of listening: ordinary and heightened consciousness [14]. Furthermore “the human ‘reflexive capacity’ in Conscious Processes through Self-reference” and differentiation of “temporalities” [15] bear some similarities to ideas found in Agamben’s discourse. Most interesting is Sarath’s argument that an improvisation remains restricted when relying on “patterns encoded in [the performer’s] physical-conceptual apparatus which shroud his or her creative and interactive potential.” This occurs when the performer has “lost access to the dual qualities of freedom from and access to . . . heightened consciousness” [16]. Sarath shows a clear awareness of the need to allow more complex “routines” to account for creative potential. Agamben’s discourse on potentiality offers a different angle. Rather than finding a model of the activity itself, it is more useful to look into the meaning of working within a range of outcomes—to move between different possibilities, to explore the potential of performance during an improvisation, to engage in activities that encompass the “potential thought,” the possibility of “thinking a thought” and “thinking of a potentiality” [17]. Attempting to translate this discourse on potentiality into terminology more obviously related to musical performance, we can consider the following: “we have the ‘potentiality of thinking about an activity in response to what we listen to’ and the ‘actuality of the activity.’” This in itself appears to be a very suitable definition of being prepared for creative activities in a state of complete attentiveness and heightened consciousness. Potentiality within the performance activity is defined in terms of “to do” 42 Lexer, Piano+ and “not to do.” Seeing the potential of responding and choosing not to do anything is a fundamental freedom within which alternatives can emerge. The evaluative layer of thinking about a response within each moment and being prone to “get it wrong” has the potential of slowing the reaction time, but it increases the responsibility and integrity of the activity. Such processes occur before the activity is executed, and several of such processes can occur simultaneously. NON-LINEARITY IN THOUGHT The Personality Systems Interaction (PSI) theory of human personality and motivation by Julius Kuhl [18] gives valuable insights into the cognitive and creative processes that can be convincingly transferred to the discussion of improvisatory approaches. Furthermore, his theory elaborates the organization and processes of our thinking. Most relevant is the categorization of brain activity into four parts: intention memory (IM), extension memory (EM), intuitive behavior system (IBS) and object recognition system (ORS). The parallel processing ability of the brain is largely attributed to the EM and forms the basis of our creative thought as a multitude of experiences form the potential for activities, such as quick responses to holistic observations. The IM, by contrast, only processes thoughts serially and is therefore mostly responsible for the conscious planning and forming of intentions. Perception and motor controls, constituted within IBS and ORS, are closely networked to the EM and IM, and the entire system is regulated by what Kuhl describes as positive and negative effects. Most interesting in this research is the confirmation of the non-linearity of creative thought and the continuously changing relationships and evaluation between the different parts. Persistent and frequent exchange of information and processes is fundamental to improvisation, in particular if performances involve conscious strategies to involve aspects of inventive approaches—a continuous “self-invention.” Improvisation stands in stark contrast to performances that resort to a previously developed and trained repertoire of responses utilizing creativity that emerges from holistic processes within defined musical idioms. The ability described by Kuhl to transfer “unfinished thoughts” from the IM to the EM facilitates a continuous exchange between intentions and “parallel holistic processing.” This describes the practical experiences gained in free improvisation more accurately. The responses of improvisers are not reducible to a single possible cognitive response; actual responses can be based on several individual ideas within different time scales. At any moment several different ideas can emerge that find “actualization” through additional layers of evaluation, within conscious compound constructs, fragmented collages, etc. During an improvisation, cognitive processes involving intuitive and conscious levels, as well as degrees of self-reflection on significance, importance, attention, etc. are in continuous flux. For example, moments arise where the dialogue might need to develop (where it feels right to move) into a more parallel double monologue in order to give the musical structure its significance in the moment of “coming together” again [19]. This is arguably considered differently within genrespecific functional and structural blueprints of idiomatic [20] improvisations. These considerations have proven to be immensely valuable in conceiving a performance system, currently presented as the piano+ [21]. POTENTIALITY SPACE I propose that constructing and populating an imaginary space with the relevant items of cultural affiliation, personality, memory, skill and approaches allow a more cohesive approach, as any linearity and hierarchy of performance/improvisation techniques can be conceptually eliminated. In this manner, a means to embrace spontaneity, adaptability, flexibility and shifting concerns can be established. Furthermore, rearranging items in this mental construct can resemble the experience of non-linear time perception, as events in one’s memory might feel closer or further away. It is also important to note that hierarchies, associations, time-scales and inclinations within each individual are created that mirror the individual’s personality and motivation. Musical and performance experiences are taken metaphorically as areas in the potentiality space. Any activity, whether it emerges intentionally or is found accidentally, can be seen as a note or picture in this “inner room,” or in Kuhl’s terms, as placed in the EM. Revisiting and reengaging with previous experiences can reveal the holistic processes associated with the EM. Individual experiences can emerge as a synthesis of several concurrent approaches. They may be “reloaded” and intentionally processed in the IM. This exchange is described by Kuhl as being motivated by an increase of negative affect, for example, dissatisfaction with the current situation/state or the wish to find further possibilities, variations and meanings within the musical Fig. 3. Contingent emergence. Three continuous controllers show very contingent behavior (i.e. C1 acceleration sensor, C2 smoothened amplitude envelope and C3 IR distance sensor). (© S. Lexer) Lexer, Piano+ 43 context [22]. Therefore, what had been a small initial discovery can develop into a more detailed picture through an increase in understanding of the approach, properties and results, much like a closer investigation reveals the contours of its parts. Continuing this metaphorical discourse, the picture can also reveal itself as a window indicating the existence of a new space and eventually even a doorway into it. Detailed areas can, however, blur over time, vanish or become concealed within other experiences, to re-emerge at any time directly recognizable or disguised within new perspectives. PARAMETER SPACE The potentiality space described above can be divided into two main subspheres—the personal and the instrumental spaces. The instrumental space is the sum of all imaginable instrumental techniques, musical interests and acoustic understanding. For instance, the instrumental space might include the piano, its physicality, strings, mechanism and materials in relation to conventional playing techniques (the core), combined with a layer of extended acoustic playing techniques and additional objects from various materials being used to alter and excite sonic events. The computer system [23] is considered to be a further layer of expansion that resorts to an entirely different paradigm—the computer as an essentially empty shell, filled only with potential through software. Thus, the potentiality of a computerized extension is defined in the act of programming. The requirement of predefining processes in the course of programming is an unavoidable compositional aspect of decisions taken to fix suitable parameter routings and is considered in terms of their placement in the instrumental space. Each individually defined and implemented process constitutes an area within the overall parameter space. During a performance the space between these distinct processes and their audio and gesture-dependent parameter controls is explored in a highly heuristic process, which matches those the performer encounters when engaging with the acoustic sources. A control structure of sensors, audio analysis and suitable algorithmic and generative tools establishes the flexibility and adaptability of the performance system; this has proven to be a Fig. 4. 3D color mediated parameter space: The grey line shows plotted movement over time. The color information retrieved at any current position is used to control process parameters. (© S. Lexer) step towards getting truly contingent results matching the subtleties of acoustic sound production. Computer processes appear to be in a continuum of full control and random response, adding an organic feel to the processes. The ability to “provoke” the computer processes with physical gestures has proven most suitable for musical practice within free improvisations demanding high degrees of complex social interaction and allowing a highly personal voice within the combined acoustic and electroacoustic world [24]. It also has created a greater aesthetic independence from technical interfaces and preplanned musical material. Overall one of the most important practical aspects for me is the establishment of a direct link and sense of immediacy to the musical outcome by implementing qualities of a similar contingency as described by the pianist John Tilbury in relation to touch and uncontrollable sound development after a hammer strikes a string. Tilbury also highlights the pianist’s dialectic of determined and undetermined responses from an instrument “on the one hand, the extreme fingertip sensitivity and control—embodying the notion of intention—and on the other hand the recognition, through an awareness of the contingent, of the ultimate impossibility, indeed the undesirability of control” [25]. In relation to electroacoustic augmentation of an instrument, this idea can be rephrased: The use of technical interfaces gives the performer the ability to direct and provoke particular processes within a contingent range of possibilities through performative gestures. TECHNICAL IMPLEMENTATION: PIANO+ The technical implementation of the piano+ can be looked at as the combination of the instrumental space of the piano, extended acoustic techniques and electroacoustic processes. Real-time processes are arranged, to continuously process audio input captured from the acoustic part of the instrument, in a complex control system, utilizing audio analysis, algorithms (i.e. stochastic and neural networks), a variety of sensors and direct controls that interact and influence one another. The result is a very flexible and adaptive performance system most suitable for use within the non-idiomatic approaches essential to free improvisation. Additionally it has to be noted that the performer controls the computer processes through performance activity 44 Lexer, Piano+ her- or himself, ensuring preservation of adaptive and flexible qualities. This has been achieved without the need to rely on a series of preset settings and operational tasks of the computer system [26]. Convergent control streams are used for most of the parameters of processes implemented in the overall instrument design. In a static setting two direct controllers control the range of a control stream from the audio or gestural analysis. Figure 1 shows a standard scenario— C2 and C3 (i.e. MIDI faders) manipulate the minimum and maximum of the data stream from sensor C1 (i.e. sensor reading) allowing for the possibility to invert the range. Figure 2 shows a scenario where C1 can be used to switch and mix between the data values C2 (i.e. a Continuous Time Recurrent Neural Network (Ctrnn) [27]) and C3 (i.e. amplitude envelope). When all three controllers are set to continuous data streams, very contingent behavior emerges (Fig. 3: i.e. C1 acceleration sensor, C2 smoothened amplitude envelope and C3 IR distance sensor). Although accurate, precisely predictable results during performance are rarely possible, tendencies are clearly perceivable. Because resulting values have a direct relation to the actual performance activity on the acoustic instrument, the indirect computer control has a clear heuristic character. The variety of loosely interconnected controller streams outlines the possibility of using the controllers in different combinations to modulate several processes simultaneously. However, sufficient contingent sonic results overall are retained to avoid merely “parallel” parameter changes. data streams, that is, not the position but the color of the objects in the color-mediated parameter space determines the parameter values. The overall complexity of the controls has multiplied; however, the heuristic quality of use has been preserved. The performer can investigate the defined instrument (acoustic instrument, implemented electroacoustic processes, controller structure) in an experimental manner, but also can continuously build on previous results and experiences. Future implementations will include generative manipulation of these parameter spaces in relation to current and past performance activity. References and Notes 1. D. Bailey, Improvisation: Its Nature and Practice (Cambridge, MA: Da Capo Press, 1993) p. xii. 2. M. Wanderley and M. Battier, Trends in Gestural Control of Music (Paris: IRCAM, 2000) p. 234. 3. M. Young and S. Lexer, “FFT Analysis as a Creative Tool in Live Performance,” Proceedings DAFx-03 (2003). 4. Cornelius Cardew, Treatise Handbook (London: Edition Peters, 1971) reprinted in Cornelius Cardew: A Reader (Matching Tye: Copula, 2006); Bailey [1]; E. Prévost, No Sound Is Innocent (Matching Tye: CopulaMatchless Recordings and Publishing, 1995); C. Dell, Prinzip Improvisation (Köln: Verlag der Buchhandlung Walther König, 2002). 5. Most relevant is Giorgio Agamben, Potentialities (Stanford, CA: Stanford Univ. Press, 1999). 6. Agamben [5] p. 121. CONCLUSION Various fields of research and philosophy have outlined my approach for a practical take on technology applied to live performances of freely improvised music. Semi-emergent qualities derived from layering different types of controllers and suitable analytical and generative algorithms have been utilized in the composition of a performance system. A contingent quality of controls reflects those found in acoustic instruments and is very typical to the piano. Furthermore, a highly flexible and adaptive system minimizes direct operational computer control. This has helped retain a pianism in sound production involving conventional, as well as extended, playing technique and electroacoustic real-time processes. Practical applications over the past 3 years have shown that the system is highly heuristic. Investigative and experimental performance strategies, an openness for sonic results and a performance practice dedicated to an exploratory and sensitive nature are required to develop a close and intrinsic relationship between electronics and acoustics. Writings on potentiality and Sloterdijk’s novel socio-philosophical discourse have proven to be highly relevant for the continuation of research into improvisation, in general, and suitable software design, in particular. It appears doubtful that the act of improvisation can ever be explained by a linear progression of a single algorithm since Kuhl’s PSI theory in psychology shows how parts of the brain work in independent, but inter-regulated parts. Acknowledgment The author owes much to the continuing support and inspiration of friends and colleagues, in particular John Tilbury, Eddie Prévost, Seymour Wright, Heinz and Marion Meinhardt-Giesen, Elke Schwarz, and Ian Stonehouse. 7. Agamben [5] p. 116. 8. Agamben [5] p. 121. 9. Agamben [5] p. 182. 10. Prévost [4] p. 20. 11. Cardew [4] p. 126. 12. P. Sloterdijk, Sphären I–III (Frankfurt a.M.: Edition Suhrkamp, 1998–2004). 13. For example, Jeff Pressing, “Improvisation: Methods and Models,” published in John A. Sloboda, ed., Generative Processes in Music: the Psychology of Performance, Improvisation and Composition (London: Oxford Univ. Press, 1988). 14. E. Sarath, “A New Look at Improvisation,” Journal of Music Theory 40, No. 1, 1–38 (1996). 15. Sarath [14] p. 8.; also Dell [4] p. 71. 16. Sarath [14] p. 14. 17. Agamben [5] p. 250. 18. J. Kuhl, Motivation und Persönlichkeit (Göttingen: Hogrefe-Verlag, 2001). 19. Rowe/Tilbury, “Duos for Doris,” Erstwhile, 2003. 20. Bailey [1] p. xi. 21. The author’s musical examples include: Lexer/ Wright, Blasen, Another Timbre, 2008; Lexer, Dazwischen, Matchless Recordings, 2009. 22. A direct link between the EM and the IBS could serve to explain that idiomatic improvisation appears to be more virtuosic in the conventional sense, as the slower Intention Memory (IM) is bypassed. 23. Developed in Max/MSP. 24. George Lewis highlights the importance of sound as a carrier of personality. “‘Sound’ ought to become identifiable, not with timbre alone, but with the expression of personality, the assertion of agency, the assumption of responsibility and an encounter with history, memory and identity.” See G. Lewis, “Too Many Notes: Computers, Complexity and Culture in Voyager,” Leonardo Music Journal 10 (2000) p. 37. Eddie Prévost raises the question of how different people playing the same kind of instrument manage to sound so individually different. See Prévost [4] p. 56. 25. J. Tilbury, “Feldman and the Piano: The Art of Touch and Celebration of Contingency,” Second Biennial International Conference On TwentiethCentury Music, Goldsmiths College, University of London, 2001. 26. A distinction between “performance activity” and “operative task” revealed itself as very important in this discourse. “Operative tasks” are not restricted to the computer operation per se. However it is proposed that there is still a very obvious difference COLOR-MEDIATED PARAMETER SPACE Further complexities have been achieved by applying the concept described above to a virtual 3D color mediated parameter space. To reflect the conceptual similarities of equipping one’s inner space, the parameter values of the processes are stored as color information within the virtual space. Figure 4 indicates possible settings to control the density of two processes in relation to position within the space. The actual positions {x, y, z} in the space are dependent on three triple controller sets described above. The position of the current movement (C1[x, y, z]) can be restricted through manipulation of C2[x, y, z] and C3[x, y, z], which define the possible areas. The color information V[a, r, g, b] retrieved at the current position is used as the resulting Lexer, Piano+ 45 between picking up an object (e.g. bow) and operating a computer mouse to prepare a musical process, as the former supplies a visual clue to what might occur in the continuation of the performance. 27. Controls utilizing Ctrnn (Continuous Time Recurrent Neural Network) have been explored in a collaboration with Ollie Bown. See O. Bown and S. Lexer, “Continuous Time Recurrent Neural Networks for Generative and Interactive Musical Performance,” Proceedings EvoMusArt (2006). Manuscript received 1 January 2010. Sebastian Lexer is a freelance musician, programmer and lecturer specializing in the use, development and research of interactive technologies for performance. His performance system piano+ has developed equally from persistent interests in free improvisation and contemporary experimental music. Lexer’s piano studies and continuing collaborations with John Tilbury and an association with Eddie Prévost’s improvisation workshop were influential to the development of his musical style: a musical journey through the spaces “in between,” relationships to the instrument, sonorities and textures, musical processes, technology and fellow musicians. He has been active in London as a performer and promoter of free improvised music over the past 9 years, with numerous collaborations resulting in concerts and radio broadcasts in the U.K., France, Belgium, Germany, Hungary, Slovenia, Switzerland, Canada and the U.S.A. 46 Lexer, Piano+