Project History

July 5, 2007 on 4:27 pm | In Project History

Speech Guitar MaxMSP Patch and Interface Development

(Pen Partner Ukulele) 2007

Pen Partner Ukulele

(Speech Guitar Mabach Labhair) 2006 - 2007

The Full active area of the Wacom tablet is mapped to the pitch and rate parameters of the speech synthesis framework via MaxMSP interface. The tablet’s vertical Y axis is mapped to frequency with a range between 48.999428Hz and 739.988831Hz. The horizontal X axis is mapped to speech rate and also an MSP delay line with a VST effect.

Touching the Intuos pen on the Wacom tablet’s surface or pressing the pen’s button triggers the formation of an instant text string ready for MacinTalk synthesiser speech generation. This text comes from a buffer containing a limited vocabulary of Gaelic words with a 100 words representing each letter from A through to X. An abstraction can be included that dynamically adds embedded speech command syntax to fine-tune give emphasis to the pronunciation by either pen position on the tablet or the velocity of the gesture.

The axis 1 data stream of a USB Sprocket device is interfaced via the max object Insprock it is mapped to the low and high range of a dynamic signal envelope that controls multiple sound grain playback events based on a single buffer~. The buffer is replaced by holding down the sprocket device potentiometer button, along with a tablet touch which generates an audio format signal from the speech synthesiser.

The patch also uses the axis 3 data stream from the same USB Sprocket device to switch between the Wacom tablet’s continuous output to the parameters of the speech synthesiser, and that of the Percolate physical modal MaxMSP object flute~. The tablet’s output is also gated depending on which side of the tablet receives a pen touch. Drawing on the left side triggers an abstraction that forms coherent text to speech strings, the right side produces sustained and low rate abstracted speech formats.

A simple MSP abstraction was added containing record~ and an array of buffer~, sig~, and groove~ objects. These are used to build additional looped layers over the main speech synthesis events each capable of recording six to ten seconds worth of signal generated from the speech synthesiser. Sample duration and loop point is controlled by timing the drawing gesture made upon the Wacom surface, also by holding down the Sprocket device Joystick potentiometer button. This abstraction samples the MacInTalk speech voice “Bad News”, voice index 22. It’s effect gives an impression of a Canon, a form of structured hymn. It is an attempt to begin to explore ideas of structure, Meter, and poetry, with real-time speech synthesis.
Download Listen to a sound clip 1 from the Speech Guitar Patch

Download Listen to a sound clip 2 from the Speech Guitar Patch

(Speech Guitar Flite X) 2006

(SG to go Patch) 2006

(Wacom Rock Simple) 2006

Download Listen to a sound clip from Wacom Rock Simple

(Amp 600) 2006 Find under Categories

(Berlin School Patch) 2005

This patch was partly inspired by what is described as the Berlin school genera. During the applications development the sound it produced started to resemble the style of three classic 1970’s line up Tangerine dream albums. It was used to busk Wacom Tablet driven Speech and electric guitar Synth improvisations on the streets of Nottingham.

The patch incorporates similar functionality to that of the Wacom Rock Jam (see below) but it’s main feature is an 8 step sequencer abstraction and interface that drives an electric guitar synthesis modal which also can be routed through the Speech Synth MSP buffer, and Apple Quicktime GM percussion. Four active areas of the tablet are mapped to the sequencer making it possible to shift the guitar like bass lines around in real-time. It is then possible to flip the interface and switch the tablet’s output towards creating an accompanying lead solo of just the guitar synthesis.

Download Listen to sound clip 1 from the Berlin School Patch

Download Listen to sound clip 2 from the Berlin School Patch

(Amp) 2005

Amp is a patch created for a presentation that was given during the Cybermusic event at the Dana Centre Science Museum London. It is designed to provide a simplistic version of a theme-and-variation style performance, and was used to demonstrate some of the capabilities of controlling the MacInTalk speech synth with the Wacom Tablet or in this case the Wacom Rock Guitar. The speech synth input is routed through an MSP convolution abstraction that produces an effect similar to a Vocoder which could be seen as a bit of a paradox.

There is a choice of three male (TTS) voices each with their unique characteristic when controlled by the mapped and divided areas of the Wacom. The text buffers included in this patch are formatted in a way that gets the speech synth to produce an incomprehensible rhythmical babble of numbers, vowels, and made up words. There are seven set patterns and one generative which harmonically work together to create a tune that can be triggered, looped and modified by drawing gestures on the tablet.

Download Listen to a sound clip from the Amp Patch

(Choris) 2005

The active area of the Wacom Intuos is mapped into six independent sections that capture and record data produced from drawing gesture. This data controls the speech syntheses parameters such as rate, pitch and triggering text to speech symbols. Velocity and execution time of a drawing gesture influences MSP processes upon it’s speech syntheses buffer in each section. This produces multiple looped layers of sound of abstracted MacInTalk speech syntheses that can be written to disk in real-time. A recording called Gossip part 1 made with the Choris patch can be found here.

(Wacom Rock Jam) 2004

With the application Wacom Rock Jam the active area of the Wacom Intuos tablet is mapped to two main sections defined in the main patcher. The left section is simply defined as an area to trigger an Apple Quicktime synthesiser General midi beatbox abstraction, and sequences that drive a synthesis modal that comes from the Percolate suite of objects by Dan Trueman and R. Luke DuBois. The right section is used for making guitar strumming gestures where the interface is mapped to a note range from C2 E3 giving a frequency range of 65.406 to 164 hertz. This influences the notes played in each of the pre-structured pattern sequences.

This patch came from an early experiment with the flute~ physical modal object, by changing it’s parameters of breath noise, vibrato gain and frequency it was possible to produce quite an interesting noise that resembled an electric guitar. Initially it was not the intension to include literal guitar sounds, but i finally gave in some way into the project and included it as an optional alternative to reinforce the juxtaposition’s in the project’s aesthetic.

There are twelve different pattern sequences in this patch each influenced by ether listening to a particular band or adopting styles from a musical genera.

ZZ Top StatusQuo Motorik Slayer FatBoy Slim Glam Rock

Electro The Soft Machine TD TG Terry Riley

Download Listen to a sound clip of pattern 6 from Wacom Rock Jam

(Speech Patch) 2004

Download Listen to a sound clip from the Speech Patch

(Speech Harp) 2004

Download Listen to a sound clip from the Speech Harp Patch

(Death Metal Speech Synth) 2004

Download Listen to a sound clip from the Death Metal Speech Synth Patch

(QT Harp) 2004

Download Listen to a sound clip from the QT Harp Patch

(Developmental Test Patch) 2004

(Wacom Rock Guitar) 2004

The Wacom Rock Guitar was built in August 2004 in preparation for a performance at the upcoming Ars Electronica Timeshift festival. The materials used to build this controller/instrument have been recycled from a once existing broken sculpture that used to automatically give water and artificial light to a well known herbal plant while playing back to back trip hop CDs, the other from a couple appropriated, modified USB devices namely the Wacom Tablet. The ornamentation comes from using adapted templates designed originally for an artwork titled Auto Confessional 2 (see categories). Some of the wood veneer used and a blue cold cathode fluorescent lamp are utilised leftover components from that sculpture. The Controller’s strap was bought from a guitar shop in Denmark street London a few days before leaving for the Festival.

(Wacom Rock) 2004

The Wacom Rock patch was developed at IAMAS in Japan during a Masters Exchange program with Ravensbourne college. It was one of the first applications in the history of this project designed specifically to utilise Wacom Tablet control and map it to Speech Synthesis, MSP, and Quicktime Midi.

The concept for creating the patch was to design a more intuitive controller/interface for an experimental text to speech synthesis project that attempted to produce (TTS) Sprechgesang. Creating an early version of the patch involved working with the sound artist Jim Wood who recorded a library of samples from a session of playing a prepared electric guitar using an extended technique. The patch used these samples and processed them with MSP to produce a filter convolution with the speech synthesis. This patch was used for the projects debut performance at the Quadra Hac event during the Ogaki Biennale in Gifu Japan 2004.

(Gossip) 2003

Download Listen to a sound clip from the Gossip Patch