Multimedia system provides new view of musical performance

As posted from ACM today,

University of Leeds researchers have developed new multimedia technology that will enable musicians to use three-dimensional (3D) computer analysis to improve their technique. Professor Kai Ng has created i-Maestro 3D Augmented Mirror (AMIR), a system that records a musician’s posture and movement while they play, using motion capture and maps the results against ideal performance settings. “Many musicians already use video recordings of their performance to analyze technique, but this only provides a 2D image,” Ng says. “The 3D image and analysis provided by AMIR will be of immense value to musicians and teachers alike.” The prototype was designed for musicians playing stringed instruments, but AMIR could be adapted for other instruments. AMIR works by following markers attached to key points on the instrument and the musician’s body and recording the movement on 12 cameras at 200 frames per second. Bow speed, angle, and position are measured for real-time analysis and feedback. The system also uses a Wii Balance Board to monitor data on the musician’s balance. The musician or teacher can see and hear a video of the performance along with an on-screen analysis of posture and bow technique, and can even go through the performance frame by frame if necessary.

Basically, the university has come up with a way for musicians to use 3-D technology to improve their playing technique. It compares a musician’s body movements to “ideal performance settings”. This is interesting because for professional musicians, who’s to say what the ideal performance setting is? This tool is using technology to apply an objective judgment of a musicians body position in order to affect the (subjective) sound quality of their playing.

Admittedly, I think that most people who would benefit from this tool are students, not necessarily concert musicians, who probably are holding their body in an awkward body position, and thus maybe contributing to a poor or impaired sound quality.

For dance/movement arts, I doubt that this tool could be used for “ideal performance settings” since body position in a performance is mostly up to the choreographer to decide. However, I do think this would be useful for dancers or others wanting to improve their sense of balance and body control – such as the elderly, people with physical or motor disabilities, or people recovering from injuries. Wow…I hope I can invent something like this one day!

The full article is available as a press release at the University of Leeds website. The website has other links, but here’s a link to some screen captures of the tool.

Press Release – http://www.leeds.ac.uk/media/press_releases/current09/i_maestro.htm

Screen Capture – http://www.i-maestro.org/contenuti/contenuto.php?contenuto_id=52&tool=gp

Pearls Before Breakfast

Recently a colleague sent me a link to an article about a social experiment run by The Washington Post. I thought it would be a nice thing to post it today, on the anniversary. In the experiment, on January 12, 2007, the paper stationed an incognito Joshua Bell, arguably the best violinist alive today, in a Metro stop in Washington D.C., playing some of the most difficult violin music ever composed on his own $3.5 million dollar violin. It was all set up for what could have been a great display of humanity appreciating the beauty of music.

“In the next 43 minutes, as the violinist performed six classical pieces, 1,097 people passed by…Each passerby had a quick choice to make, one familiar to commuters in any urban area where the occasional street performer is part of the cityscape…Do you have time for beauty?”

As it turned out, Bell made $32.17. There were no crowds and no applause. (Lucky for him that he is an exceptional musician. It turns out that the shoeshine woman calls the cops on all the other musicians.) The catch of the experiment was that they placed in the Bell in the station at 7:51 a.m. during the morning rush hour, when people would be the most unlikely to stop, precisely to see how many people would actually slow down to appreciate the beauty around them. Most people barely slowed down while they passed. People on cell phones talked louder. It was only the children who paid attention. Unfortunately it was their parents who dragged them away. Only one person recognized Bell. She had seen him in a recent concert where tickets cost over $100 each.

Practically speaking there are a couple of unsurprising reasons for the outcome of this experiment, which I’ll try to take on using a user experience point of view. First, the context was completely wrong. A concert violinist playing a solo performance at rush hour in a busy Metro station in D.C. is in the wrong place at the wrong time. Second, not everyone likes classical music, or would recognize the songs he played. If they had placed him right outside of a music conservatory there would have been a riot. I also can’t help but wonder if people who wanted to, but didn’t stop did so because they just wanted to fit in with everyone else ignoring street musicians. Such is urban life, but that seems to me a cop out.

Philosophically, the whole thing arises within me an existential argument. I can’t help but question the value of my work when people seemingly don’t even have the time to stop and appreciate beautiful music on their way to work. In fact, some homeless man had died in the corner in that same station a few year before. No one had stopped then either. On my Facebook account, one friend admitted that he wasn’t sure if he’d stop just due to being preoccupied with work. Are our American priorities misaligned so much that we can’t even stop to pay attention to such beautiful art, or is it just that he was so much out of place?

Well, I believe that my friend’s priorities are aligned properly, but I do worry about our ability to stop and enjoy all that life offers, and not get too preoccupied with work and money. It is important for User Experience designers to keep these aspects of urban space and society to keep in mind when we are producing our work, so that our work remains accessible. In any case, getting people to stop and appreciate the art around them is what I hope to accomplish with my future work and to explore more in this blog.

The article is here. Pearls Before Breakfast – http://www.washingtonpost.com/wp-dyn/content/article/2007/04/04/AR2007040401721.html Check it out. There’s video.