Project Spotlight: Interactive Orchestra with Serena Chao

Written by Samantha Peng
Jan. 14, 2019

A student team is working with the Director of Orchestras at the UBC School of Music Dr. Jonathan Girard on a new VR project called Interactive Orchestra. They are working to build a virtual podium for music students to practice their conducting skills.

Project coordinator Serena Chao says usually, students take turns being the conductor with their peers, but it’s time-intensive to rotate and allow for equal practice time.

“It takes a lot of resources to get hands-on experience,” said Chao. “What we want is for students to jump into the experience whenever they want in Interactive Orchestra.”

First phase of development

The project started at the beginning of the winter term and is in its first phase of development: to track user-gesturing.

To program movements, the team has been using “Oh, Canada”. Chao explains that most of the team doesn’t have music experience and using the Canadian anthem is easy to work with.

However, Chao says the challenge is that everybody conducts differently and giving each person an accuracy score based on how well their gesture shape is will be difficult.

“Therefore, the scoring for how accurate the gesture is isn’t as important but we do want to detect where they are in the score and be able to track it,” she said.

She says the team is also working to build the project’s environment. There will be stage and the menu will be the music stand in front of you. When you look around, it’ll be like you’re standing on the podium of a real orchestra.

What’s phase two?

Next, Chao says the team hopes to program the project so that Interactive Orchestra adjusts its beats per minute, speed, and steadiness to that of the user. She explains that in real life, an orchestra adapts to the pace of a conductor, not the other way around.

After, they plan to cut the music in the project according to the different sections of instruments within the virtual orchestra.

Chao is familiar with the workings of a real orchestra as an experienced flautist herself. She’s excited to apply her knowledge and work with Dr. Girard.

“This project is so cool. In our first meeting with Dr. Girard, he said this has never been done before. If we do this, it’s going to be ridiculous.”