Real-Time Graphic Notation Based on Gesture Data
The graphic notation for this work is generated by extracting bowing gestures data of two instruments: cello and a self-built augmented violin bow. This means that the graphics do not merely symbolize gestures, but show the actual gestures.
I used a Myo armband to record cello bowing data, and the augmented bow’s own position tracking module to record its gesture data. After processing the data, I used it to recreate the gestures on the screen, as a form of real-time graphic notation based on imitation. This notational approach allows for an immediate, intuitive interpretation on the spot, a crucial feature for the practice of real-time scores.
Myo EMG data of heavy and light bow pressure
The augmented violin bow is a hybrid instrument, that consists of a tangible part (a violin bow) and a digital part (a Max patch). Its purpose is to afford a physically engaging electronic performance, focused on bowing gestures. I wanted to express the bowing idea not only in terms of movement, but also in terms of sound. Therefore, the sound of the instrument is the sound of the bow itself, amplified and processed (according to movement and position data).
Augmented bow motion tracking as it appears in the Max patch
The graphic notation, in the form of gestures that the performers imitate, looks differently for each instrument. The cello part shows the front of the instrument, including the lower part of the fingerboard and the upper part of the tailpiece. A virtual bow moves on the screen, driven by the gesture data, instructing the cellist what to play. The augmented bow part shows a white dot, representing the LED diode mounted on the bow. It shows the bow’s movements as they appear in the motion tracking window in the Max patch, instructing the actions to the performer.
I have been working on this real-time graphic score as an artist in residence at Phonos Foundation, Barcelona. Capturing motion data, analyzing it, and using it in the context of the graphic score have been done with the help of David Cabrera and Rafael Ramirez (Music and Machine Learning Lab at Music Technology Group, Universitat Pompeu Fabra). The cello bowing gestures data was recorded with cellist Leo Morello. The graphic designer Nir Bitton advises with the score design.
Timbre-Centric Composition Method for Saxophone
I have been working together with saxophonist Jonathan Chazan on an artistic research project that explores a new way of composing for saxophone, based on timbral qualities. The overall, future goal of this project is to develop a method for composers and performers to indicate specific timbres and interpret them precisely, aided by technology. Eventually, we want to expand it into a quantitative notational language that allows to express timbres efficiently and unambiguously.
Our special interest in developing this method comes from the unique mental state a timbral-centric system creates in performing and listening to music, since it calls to pay attention to timbres in a more profound way. And, because it is not necessarily intuitive, it brings the composer and performer to novel, interesting, and unpredicted musical places.
So far, the project has yielded a 28-minute composition Siete Dolores, for saxophone and live electronics (audio and video/lights). The piece was designed to showcase this approach to composing with timbres, similarly to an etude that focuses solely on that timbral technique. The timbre analysis and identification in the piece is done with Sonic Print by Jean-Francois Charles, and with Zsa.descriptors by Mikhail Malt.
Siete Dolores (excerpts) – Jonathan Chazan, Saxophone
Our goals for the next stages of the project are: to improve our spectral analysis and timbre recognition system; to gain a better understanding of the perception of different saxophone timbres; to compose a new, shorter piece; and to present our work in academic settings.