Apr 2012

Four Dimensions

A technology showcase blending Live Orchestra, Live computer graphics, 6 channel electronic music and EWI

            (above, rehearsal photo courtesy of Jim Shelton, Winter Park)

Notes from the Composer

“Four Dimensions” was created to lead an audience through an imaginary exploration of the dimensions through the music of a full orchestra that included an EWI, a surround sound playback system, and moving images utilizing a 48 foot video LED wall, and moving lights. The brilliant real-time computer graphics were created by Visual Artist, Nathan Selikoff, supporting the music story and evolution. It is a single movement work lasting under seven minutes.

Full Sail Live Venue is a hi-tech multipurpose space


In the fall of 2011, Christopher Wilkins, the Music Director of the Orlando Philharmonic, asked me to write a new work for the April 21, 2012 fund-raiser gala concert called “Symphony in HD” at new multi-purpose “Live” venue at my long-time employer, Full Sail University. This was the first time a full orchestra performed in the new facility. The concert consisted of 12 pieces from the standard symphonic repertoire, pops, opera and jazz; all synced to visuals specifically created for each work by Full Sail students and staff. 3 large HD video screens, a host of the latest rock concert technology in lighting, and an very large LED billboard video screen (48 foot) would bring a multi-sensory experience to cherished music. Billed as a kind of art concert of the future, my commission would ‘pull out all of the stops’ in technology to create a totally immersive audio visual, high powered experience. The music needed to be short, fun and entertaining. I decided to employ the full range flying surround sound system, asked digital artist Nathan Selikoff to create live visuals and requested that the orchestra hire local electronic wind instrumentalist (EWI), Brian Smithers – a colleague at Full Sail. Sean McKeown provided the excellent concert lighting for the event, and worked with Nathan to support his visuals.


The orchestra was conducted by Maestro Dirk Meyer, the Associate Conductor of the Sarasota Orchestra (who is now the Music Director of the Duluth Superior Symphony Orchestra). He was open and generous to helping us develop the technology of Four Dimensions. Not only did he prepare all of the other works on the technically complicated concert program, he prepared this difficult new score. Beyond those huge tasks he also acted as a musical performer at the same time with the use of the Nintendo Wii-mote. Further complicating his role, the orchestra only allowed a mere 20 minutes for rehearsal before the piece was premiered!

Akai 4000 EWI

I treated the EWI as a featured wind in the orchestra via a single Bose ‘stick’ line-array loudspeaker to imitate the natural acoustic spread of the woodwinds. (The Bose L1 is the tall black object behind Mr. Smithers house right). The EWI also provided a direct out mono signal. Instruments were miked in a variety of ways: high strings used mini microphones clipped to each instrument, low strings shared a mic per stand. Brass shared a total of 8 mikes. Percussion shared two, timpani had two of his own; woodwinds shared 1 per stand (4 total); the harp used its own mono DI, and the digital piano sent audio via a direct stereo output. The sound of this performance was wonderful – credit to Mike Stewart for a great film-style live mix made possible through the use of the 60 odd microphones on stage. All of the audio signals were also sent via fiber to the nearby “Audio Temple” studio and recorded to ProTools through a 72 channel SSL Duality console.

The prepared surround sound electronic music in “Four Dimensions” is presented as a new orchestral section and functions like one: it doubles other orchestral material for color variation , strength and emphasis, it often presents new material, and it sometimes serves as accompaniment to other sections. It provides both unpitched sonic textures as well as pitched ones. The 6 channel electronic score was built in Apple’s Logic Pro 9 with sounds created with MetaSynth 5, Logic ES2, Logic Sculpture and a Moog Series 55. It was performed at 96kHz/24bit resolution through three overlapping 6 channel audio engines in Max6 using a MacBookPro laptop and an Apogee Ensemble audio interface.


In my experience with electro-acoustic music, the use of any loudspeaker system in a concert space for electronic music rarely combines with the natural acoustic sounds in a natural way. Carefully crafted details and textures from the electronics are too often lost or dimensionally retarded. I designed sounds that emphasized an organic beauty – not the buzz click squeal of so many overly synthetic works – to blend with the orchestra like another orchestral section. My multimedia ‘pocket opera’ concept, “Starboy” (2004), demonstrated the potential to deliver a sonically unified experience to the audience with pre-produced, electronically generated surround sound tracks and the human voice. But the major flaw was the lack of tempo flexibility that is so crucial to opera: the performers were essentially singing to a click track.

Conductor held wii-mote

To allow my electronic score of “Four Dimensions” to adapt to the conductor’s musicality in timings, I asked Full Sail colleague and Max engineer Marc Pinsky to create an application by which information from a Nintendo Wii-mote would be used by the conductor like a baton to trigger pre-produced sound events via bluetooth wireless. (for an in-depth development history of our use of the device, see “Four Dimensions Development Updates” in the Journal section of this website)


First, the wii-mote sent beat-to-beat tempo data points culled out of accelerometer and orientation data control the speed of playback of the 6 channel pre-produced sound file. This proved untenable because there was no way to correct playback if it wandered off, and no exact synchronization could be guaranteed to match. Marc then explored better ways to find the beat by mounting the wii-mote in front of the conductor, and using the infa-red camera on the front of the wii-mote to track specific colors at the tip of the baton – even an illuminated one – but the camera quality was too low to sufficiently define the baton’s icti. By February, Marc had been able to define wii-mote data well enough with accelerometer data to provide good beat identification – if the conductor provided traditional conducting icti. We decided to number each beat in the work, and the software displayed the beat number to the conductor. To simplify the conductor’s tasks, I wrote the work’s tempo to be ‘locked in’132bpm – whether or not the music was rhythmically inactive or active. This proved too complex. Working with our conductor, Dirk Meyer was wonderful. Dirk was open to the ideas and willing to take the time to work with Marc and I to develop the Wii-mote baton concept. The final version allows the conductor to use their personal baton as normal, while holding a wii-mote in their other hand to physically trigger the files to play. The maestro is free do do what he/she wants with their left hand. But the previous incarnations of the Wiimote baton still proved useful: accelerometer data was sent to visual artist Nathan Selikoff’s computer to be used in the real time rendering on the 48 foot LED billboard. To further simplify the work for the conductor, I produced the 74 5.1 surround electronics track segments to begin their playback strictly on downbeats in each bar that contained electronic sounds. Each trigger event is notated in the score with diamond-shaped notehead on a single-line staff above the strings.(see the score, below)  While many segments’ durations last for many bars and overlap other events in ensuing bars, ‘hard’ events that must exactly synchronize with the orchestra are always started within their bar with a new trigger event to improve synchronization. The sound files slightly overlap to avoid gaps – yet allow flexibility in tempo.

This is a portable piece that can be performed by any orchestra a multimedia hall with little added investment. The “Four Dimensions” application interface comes with the score and will mount on any modern mac laptop. Beyond the score and parts, it requires a wii-mote (about $40), and a downloaded freeware application called “Osculator“. Osculator wirelessly connects the wii-mote via Bluetooth to the computer. An included Osculator file has all the settings that are required. It’s a snap to set up, requiring no knowledge of Max or Osculator. I created discreet 6 channel surround version in both high resolution (24bit/96kHz) audio, and standard resolution audio (24bit/48kHz). A Stereo version for concert halls which are not equipped to handle Surround audio is available in both high resolution and standard resolution (16bit/44.1kHz). Mr. Pinsky also created a full functioning version of the standard resolution Stereo application that includes a MIDI realization of the orchestral parts for a conductor to practice his wiimote technique. Plus it helps the conductors familiarize themselves with the soundfiles while simultaneously hearing the orchestra.

The Music of “Four Dimensions”

Our concept of “Four Dimensions” arose out of my decision to keep the form of the work simple and easy to absorb: that of growth. The music and video work together to create memorable experience of five dimension based universes and derives its form (and rehearsal points) from the Zero, First, Second, Third and Fourth Dimensions. The instrumentation was determined by the orchestra (2-2-2-2 4-3-3-1 Timp, 3perc, harp,pno, strgs). I chose to have the 2nd flute use piccolo throughout and the 2nd clarinet to play bass clarinet throughout, and added the EWI to the wind section.

Dimension Zero brass chord open

The “Zero Dimension” isn’t technically a dimension at all (hence the name “Four Dimensions”), but, simply the state of being and non-being (silence), singularity, point, on/off . To represent the Zero or Non-Dimension is an opening fortissimo major 13th +11 chord from the brass – a declaration of existence. The chord is restated in the electronics, then the woodwinds and then the strings over which the EWI appears with the main theme:

The same sequence becomes a round-robin, rhythmically shortening in duration and rapidity to reach a fortissimo early, brilliant sounding climax of two simultaneous major chords a tritone apart . Those elements: the eight note theme, the ‘being’ block chord, the polytonal harmonic palette, the idea of moving back and forth between instrument sections of the orchestra and electronic sounds provide the developmental content throughout the entire work.

A loud bit-crunched, electronic line emphasizing the tritone characterizes “First Dimension” – where singularities connect to form lines. The orchestra provides a background texture with string effects, gong, bass drum and brass doubling. A bass clarinet solo emerges out of the last note of the electronic line to transition into the next section.

towards the end of the 2nd Dimension

To create the Second Dimension, lines pair up and briefly connect into planes. It is a primarily a contrapuntal orchestral section with electronics providing doubling and accompaniment that support restatements of the eight note theme by the EWI. The polytonal harmonic palette expands to the cyclable pattern notated below. The EWI line, electronics and orchestra builds in intensity reaching a climax into the “Third Dimension, driven by a repeating triplet figure in the woodwinds and stri


The “Third Dimension” provides rhythmic relief with an EWI solo over sustained harmonies derived from those introduced in the Second Dimension. The voicing of using the woodwinds to emphasize the brass/strings harmonics was inspired by Japanese composer,  Toru Takemitsu. The section finds a quiet, stable point with the piano – repeating a closely voiced extended chord in D minor.

The loudest moment within the work interrupts the solace as the 3rd dimension disrupts into the “Fourth Dimension”. Nathan Selikoff described this to be encompassing our dimensional space at the same time as being beyond it. I recapitulated the chords from the opening, and forced them against those same chords realized in Sibelius with a long, slow downward glissandi created in MetaSynth. The interruption gives way to a-tempo 132 percussionists playing rhythms based upon earlier counterpoint on 3 sets of castenets. The “First Dimension” theme re-emerges with low brass and noisy electronics – this time, the first tritone is moved upward to a more consonant perfect 5th. The altered theme now better matches the constellation of pitches that have provided most of the melodic material – providing a kind of sonata-form resolution – are taken up by the orchestra and climb to the final statement. There, the EWI and strings take the main theme, the woodwinds repeat the earlier triplet figure, and big brass chords build to the final climax with the harmonic scheme from the Second Dimension. Upon the final polytonal chord, the electronics continue, ending with a loud, shaped noise sequence from MetaSynth.

Nathan Selikoff’s real-time art is essential to the music’s impact. This was not video, but rather a series of pre-designed computer programs coded by the artist to tell the story of and the transition to each dimension. Though the music stands on its own fine – the use of another sense in the enjoyment of sensory input (how great it is to be alive! to be able to hear so high and low so far and near! to be able to see such intense color and form!) to tell the story is powerful. My collaboration with Nathan was joyous. These photos, taken during the rehearsal by a Friend of the Philharmonic, Jim Shelton, attest to Nathan’s creativity and brilliance by capturing his visual mathematic art mid-frame:

The work finished with a stunned silence followed by a resounding standing ovation from the audience – thank you, supporters of the Orlando Phil! I hope “Four Dimensions” catches on with other orchestras looking for something different – something new – that their audiences would love to hear – with this brilliantly colored immersive multimedia art ‘concert of the future’ symphonic concert approach.

Thank you Christopher Wilkins, Dirk Meyer, David Schillhammer and the OPO, Ed and Edye Haddock, Garry and Isis Jones, Jay Noble, Dana Roun, Darren Schnieder, Keith Andrews, Dave Arias and all of our superb students and staff who made the “Symphony in HD” concert a success.

More rehearsal pics from John Shelton:


Your email address will not be published. Required fields are marked *