World’s First Broadcasted XR Jazz Performance by Shocap and BBC Click
Televised globally on the 25th of December 2020, Jill Barber and her phantom Jazz band took to the stage for the world’s first, broadcasted XR jazz performance. The show was produced by Shocap Entertainment, a mixed reality performance and virtual production company, and saw the famed Jazz singer perform a Christmas setlist in an entirely virtual club with a digital band. The venue was the Palomar Supper club, resurrected in stunning CG to house the band and facilitate an Atlantic-crossing interview between BBC presenter, Paul Carter and Jill Barber, while they remained thousands of miles apart.
Bringing the past into the future
In the first half of the 20th century, legendary musicians Nat King Cole, Billie Holiday, Ella Fitzgerald, and Louis Armstrong performed on-stage at Vancouver’s renowned Palomar Supper club, located at the corner of Burrard and Georgia streets. Flash forward to 2021 and the club has long since gone, with only sparse memories and fleeting images of its eventful history remaining.
It was this concept that inspired some of the creative decisions for the production, with phantom Jazz players accompanying Jill Barber on-stage and ghosts of dancers past parading the showroom floor. With music venues around the world temporarily closed, this mixed-reality production attempted to bring viewers as close to the real thing as possible, matching the atmosphere of a live performance. New technology can shine a unique light on times of old.
Jill Barber’s live XR Jazz
This wasn’t the first time Jill Barber had graced the CG stage of the Palomar. On November 20th, 2020, Jill performed a live rendition of several original songs to an entirely online audience. The event was ticketed by Live Nation Canada and was a milestone success, providing an intimate jazz-club experience in a radically different format. With global social distancing restrictions in place, this proved the effectiveness of modern virtual production techniques as a viable alternative to traditional, in-person entertainment. Viewers could attend the show from their own homes and experience an intimate, live music experience in real-time.
With both live and pre-recorded productions entirely possible, Shocap Entertainment is taking the first leaps into the future of virtual entertainment.
Virtual Production
Launched in 2020, Shocap Entertainment is the partnership of Animatrik Film Design and Lifelike & Believable, combining the technology and expertise of the two studios to create cutting edge, XR entertainment. The studio aims to completely change the way we consume multimedia performances, spearheading advancements in virtual production and performance capture.
Production companies from around the world are increasingly utilizing virtual production techniques for a range of projects, tapping into the incredible power of real-time engines and performance capture. For the BBC Click performance, Shocap Entertainment was able to bridge the 7600 km distance between Vancouver and London and virtually seat both Jill Barber and the BBC presenter, Paul Carter, at the same table. Using green screens, Shocap visually transported Paul into the Palomar, producing a strikingly realistic, bar-side interview. This was the first time the presenter had ever conducted a virtual interview in this manner. Omar Mehtab, the BBC’s technical lead for the performance, also played an integral role.
The environment and character animations were generated in impressive CG using the Unreal Engine. The CG-band itself was driven by the movements of real musicians, playing alongside Jill Barber in Animatrik’s world-class Burnaby studio. Dressed in state-of-the-art motion capture suits and surrounded by over 50 motion-capture cameras in front of greenscreens, every minute movement of the musicians was tracked live, and the motion data was then used to drive the movements of phantom CG avatars that appeared on-stage and the dance floor.
Historically, motion capture techniques such as this were only found in big-budget movies and AAA game titles. But with the power of the Unreal Engine, Shocap has been able to implement them successfully for a range of live productions.
Behind the range of cutting-edge software and hardware solutions used by Shocap Entertainment is its very own proprietary Ringmaster Virtual Performance System. The platform connects Unreal Engine, Optitrack, Ncam Technologies, NVIDIA, and more, allowing complete control over cameras, effects and lighting cues during the show. It’s through this system that Shocap Entertainment can generate such impressive virtual productions, all while maintaining a clear anchor in reality. It’s the feeling of a real performance that is integral to an authentic XR experience.
The final performance was broadcast globally and online via BBC iPlayer, you can check the full performance here.
Credits:
Video Recording: Excerpts from BBC Click Clickmas 2020 Special December 25 2020 at The Palomar Supper club. Courtesy of BBC (c) 2020. All Rights Reserved.
Let it Snow: Words – Sammy Cahn/ Music – Jule Styne
