For the Orlando Philharmonic Orchestra‘s (OPO) May 31st Gala fundraiser for 2013, again at Full Sail University‘s Live venue, the orchestra decided to create a program of familiar repertory that was to be highlighted with striking visuals. The theme of the show was “Music of the Night“; a series of pieces beginning with dusk through early evening for part I; midnight for part 2; and pre-dawn late night through dawn for part 3. The music of the first 2/3rds of the program consisted of audience favorites from  orchestral literature. The music of the last 1/3 featured the Broadway Phantom of the Opera star Davis Gaines.

A collaboration between the Orlando Phil and Full Sail, the show’s production creative meetings were led by Full Sail’s Jay Noble. Students from different visual departments, led by their teachers, contributed to designing and producing the visual elements. The majority of these visuals came from Digital Arts and Design students under Eric Rosenfeld. Concert lighting was again designed by the Concert Lighting Dept. Chair Sean McKeown and the sound was mixed by Show Prodution’s Course Director, Mike Stewart. Visuals employed many techniques, including of static stage lighting, spots, motion lighting, smoke, fog, strobes, film cinematography, video, computer graphics, computer animation and projection mapping.

The previous year’s symphony in HD utilized animation sequences that did not have a way to ‘sync up’ with the live orchestra. Our method a year past was to make the media fit the live music. The video material for each song was chopped into smaller pieces that were reassembled live and cued by a professional score reader. To ensure that enough video footage would be available to cover slower tempos, the end of each segment was extended with repeating material. If the orchestra played a faster tempo, a dissolve provided an acceptable video transition to the next segment, all easy work with a Hippotizer. Though the show cruised along fine with this approach, video ‘hits’ did not lock in with the musical performances as designed – especially towards the ends of each asset. For the 2013 Gala, we wanted to improve visual synch by finding a way to lock the animation/video assets with the live performance for three pieces “Flight of the Bumblebee”, “Night On Bald Mountain” and “Hall of the Mountain King”.

The OPO again provided a recording of the tracks to the production crew to use as a timing template. Track by track, the videos and films were edited, the animations built and rendered, the house lighting was designed  – all upon the timings within the musical events provided in these templates.

 

Hall of the Mountain King (excerpt)

My proposal was to make the orchestra the ‘slave’ to the video through the conductor, Maestro Christopher Wilkins for these three works. This was done through the creation a prepared, “straightened”, click track to be heard only by him. He would deliver the proper tempi to the orchestra by following these tracks through a personal ear bud.

Many of the show’s template tracks were not conducted by Maestro Wilkins. Therein lay the challenge: every conductor’s interpretation of even well-known works contains their own stamp of individuality and character which includes a unique tempi fingerprint. It would be impossible for a conductor to exactly imitate another conductor’s micro/macro ebb and flow “DNA”. Therefore, the new template had to remove the minutae of individualistic ebb and flow and only deliver the larger scale tempo variations. The animation and film crews also received these new versions (without the click). This way, the ‘canned’ timings of visual movements of the video images would synchronize with the live music performance. This idea was new – an experiment – new even for the seasoned Maestro Wilkins – and he was excited to try it.

Hall of the Mountain King (Straightened)

The “straightened’ template contained  the same basic tempi and had the same overall length as the original, but was more conduct-able than the original because of the lack of micro-tempo changes.  It contained an audible click, a voice-over announcing tempi changes and rehearsal letters as they approached. A low level audio file of the  ‘straightened’ music was  included in the track for practice, as well as one without music. The Maestro preferred the track with music to simplify practicing.

Hall of the Mountain King (Conductor’s Ear)

Here are the steps to creating a ‘Straightened Track” for orchestra synchronization

  1. Score: Some pieces were shortened in duration for programming requirements from the very beginning. The OPO supplied a set of recordings as templates for all of the pieces performed, complete with any score edits, and the edited orchestral scores.
  2. Mono: The template original was bounced down to mono a version since the conductor would only be listening with one ear Example: “Hall of the Mountain King” Original recording
  3. DAW: Import the work into two separate tracks within a DAW that has compression and expansion tools. The second track must remain unchanged in order to provide a comparison to the processed track. Name the tracks accordingly (i.e. “straightened” and “original”). Mute the original track. 
  4. First Tempo and Meter: Set the opening tempo and Meter
  5. Analyze/Find Beats: As the DAW to ‘find’ the beats in the top track (“straightened”). This will be accomplished by the machine by looking for regularly timed transients. Usually this process will miss beats, but provide a good starting point
  6. Locate and Identify each beat: through listening to the onboard metronome and the track. Place a beat marker at each beat in the work. 
  7. Straightening: Move each beat to the corresponding beat on the DAW playback grid
  8. Meter Changes: Place meter changes called for within the score on the appropriate downbeats
  9. Bar Numbers: Check that all bar numbers match the score. Only the opening bars of the music will have the right tempo – but that will be corrected later.
  10. Label: Add rehearsal letters and section names as region markers in the track for quick navigation
  11. Get the score and a pencil.Solo the “original” track
  12. Locate tempo changes as listed in the score. These include metronome markings, accelerandi, deccelerandi, fermatas, Grand Pauses, section descriptions (get out your musical terms books).  Find each tempo as performed in the original template track.  Analyze each: are they immediate changes? Do the changes in tempo occur gradually – say in 2 beats? a complete bar? maybe over many bars? Write these tempo details onto the score at their proper location. 
  13. Solo the “straightened” track
  14. Go to the first tempo change written in the score (step 12). Change the tempo track according to your analysis in step 12. Then listen to the result. Compare that result to the original track. These two tracks should be closely matched through the tempo change. If they are not, change the tempo to better imitate the change. Only change the tempo automation to reflect sectional changes in the original, which should be represented in the notes and happen in a defined, local area of the track. The majority of the tempo value through the piece should be flat. 
  15. Tweak to perfection. Use curves. Keep checking results against the original track so that the ‘straightened track’ locations are close to the original timings. For “Hall of the Mountain King”, I was instructed to not increase tempos towards the end as in the template. 
  16. Approval: All straightened tracks must be sent to the conductor for any changes and suggestions. 
  17. Upon his approval, send the  ‘straightened’ track to the visual teams to use as their timing base 
  18. VO: A voiceover containing a count-off to and declaration of each rehearsal letter is to be recorded to a new track in the session. For example “8,7,6,5……letter B”. Tempo changes are also announced, for example,  “up 5 pbm in 4,3,2,1”).
  19. Mix the Conductor’s Version of the Straightened Track: The click and count off were mixed at high levels. The straightened music, if the conductor wishes to have it,  is to be mixed in at a low level. 

What was learned:

  • This approach put a great deal of pressure on the conductor. Not only did he have to match a click in one ear, he had to get the 50 or so musicians to follow him exactly. If they lagged, he had to pull them ahead – if they rushed, he had to drag them back – all the while,  listening to the click, keeping his place in the score, and making entrance cues – and making music. Quite a feat. Maestro Wilkins urged the musicians to play like machines – and stay with him as accurately as possible.
  • Though the music lacked some humanness in minor ‘ebb and flow’ tempi changes, the depth of the music spoke beautifully to the audience. I doubt that any noticed any differences.
  • Maestro Wilkins, after the concert, agreed that this solution worked remarkably well – and thought it was the best solution for the challenge.


What I’ll do next time:

  • Insure that a well fitting ear bud is available before rehearsal
  • Make signal level changes available by the conductor

 

  • Reply

    Kevin

    23 08 2013

    Wow, this is cool. I didn’t know you were doing this. Do you think there is a way to track tempos that would run on an iphone?

  • Reply

    Keith Lay

    25 08 2013

    Yes and No. Mostly No for any computer.
    There are already quite a few DAWs that do a good job of tracking tempo: Logic, Ableton, ProTools as well as Max, but they are built for pop music that includes some kind of drum/rhythm section. When Logic tried to find beats in these 3 classical pieces, it found about 75% of the beats because they were regularly spaced transients, but was lost when the music was quiet. That makes it unusable by itself. I can’t imagine a machine ‘feeling’ the beat of a flute solo, or long notes from a cello solo…… How about a score from Ives, Boulez or Feldman??? Probably never.

Your email address will not be published. Required fields are marked *