The New Zealand Symphony Orchestra has released a fully immersive Virtual Reality experience of the 90-member orchestra, which was produced by New Zealand content creation company Wrestler.
The VR experience is one of the first in the world that captures the orchestra in 360-degree video and sound from five different camera positions.
NZSO Marketing Manager Thomas Drent says it will revolutionise how New Zealanders can experience their national orchestra and expects that the VR experience will entice more people to see the Orchestra live.
During the performance, the viewer can move freely around the Orchestra; from the conductor’s podium beside NZSO associate conductor Hamish McKeich to the different sections as the musicians play.
“The NZSO is always exploring new ways to expand its audience and bring the Orchestra to all New Zealanders,” says Drent. “This new VR experience allows people to immerse themselves in the Orchestra in a way that would normally only be possible if they were a player or conductor. It looks and sounds amazing and is incredibly realistic. You will believe you are actually standing amongst the musicians as they play.”
According to Drent, the potential uses of VR for the NZSO are limitless, and developments in VR could mean that one day a viewer could conduct a virtual orchestra or play a virtual instrument in a VR performance.
A Holophone microphone was used to record the audio in 360 degrees so that the sound of the performance changes as the viewer moves around the Orchestra
Wrestler used an Omni camera array to film the Orchestra in 360 degrees. “It’s made up of six cameras that all shoot simultaneously and through specialised software we can stitch those six shots together and create a piece of footage mapped to the sphere,” say Wrestler VR Developer Jeff Jones. This means that the VR video precisely follows the viewers head movements, so even if they turn away from the Orchestra, or look up or down, they will always see the interior of the Michael Fowler Centre.
“It’s a fantastic opportunity to actually get on stage and go ‘I can sit next to the timpani or next to the trumpet or next to the harp’. It’s one more way to share our music-making experience. I think it’s fabulous,” says Section Principle Percussionist Laurence Reese.
To support the 360-degree visuals, a Holophone microphone was used to record the audio in 360 degrees so that the sound of the performance changes as the viewer moves around the Orchestra, like it would in a live setting. A Holophone H2-PRO7.1 was embedded with eight microphones to achieve the desired effect, and the NZSO is one of the first Orchestra’s in the world to include surround sound in a VR video.
“The sound recording for the project was a really interesting challenge and also a little bit daunting,” says Wrestler sound engineer Chris Ward. “We had to come up with something that had a very small profile that was discreet, that was easy to move between set ups.”
Wrestler Head of VR/AR Kat Lintott says the company was stoked that the NZSO wanted to experiment with VR. “To be able to walk around the stage while many musicians play around you is just breathtaking”
McKeich and the NZSO players viewed the VR video and were blown away by what Wrestler captured.
“The whole project was a great success and it was fantastic for me just to be able to see the orchestra in a new way and in a contemporary way as well” McKeich says.
The NZSO in VR experience can be downloaded from Google’s Play Store for Android phones and Apple’s App Store for iPhone and can be watched with smartphone-friendly VR headsets, including inexpensive cardboard models.
For those without a smartphone or VR headset, a version of the video will be available on the NZSO’s Facebook and YouTube, which will allow the viewer to use their mouse to pan 360 degrees while the NZSO performs.
