By Vincent Geels, The Philadelphia Orchestra

I recently sat down with Youngmoo Kim, associate professor of electrical and computer engineering at Drexel University, to talk about the genesis of “iNotes,” the orchestral performance companion mobile app. A graduate of Swarthmore College with dual degrees in music and engineering, Youngmoo has long been fascinated by both of these worlds. He’s definitely no stranger to classical music—he used to sing with the Tanglewood Festival Chorus—and he’s also no stranger to thinking about the potential intersections of music and technology, for which he credits his alma mater’s forward-thinking music professors.

As it turns out, interactive classical companions stretch back to the era of CD-ROMs. Remember those? Youngmoo told me about a project launched by Warner New Media in the early 1990s that delivered sound recordings of famous classical works paired with “HyperCard-based Audio Notes.” Developed by Apple, HyperCard, a precursor to the World Wide Web in the late 1980s and early ’90s, delivered text-based information through virtual “cards” in a pleasing visual format on your computer screen. When programmed to pop up at specific points throughout each piece in Warner’s Audio Notes series, these cards gave classical music novices the opportunity to learn more about the featured work on each CD-ROM.

Fast forward to 2007, and things have changed just a smidge. Apple is preparing to release its MacBook Air (goodbye CDs), the iPhone hails the dawning of a new breed of smartphone, and the World Wide Web is being touted as the best thing since HyperCard (thanks Al Gore). Perhaps most important to our story is the fact that these technological milestones intersect with two major points: The Philadelphia Orchestra launches its Global Concert Series, issuing HD broadcasts of its live performances to large-screen venues; and Youngmoo brings a group of his students to see one of them.

“That’s when we really started talking about it,” notes Youngmoo. The release of the iPhone introduced a whole new conceptual playing field for a truly interactive companion app with the processing power to match. “Signal processing, tone analysis, and reference matching—it’s a unique combination of skills,” he explains. To build a computer program that actually “listens” to a musical score and follows along is no small feat, and it requires major legwork to produce an app that seamlessly integrates these technologies, presenting multiple streams of information to accommodate a user’s level of familiarity with classical music. In short, technologies have only recently developed to the point where such an undertaking is possible; still, the fact that the technology is there doesn’t necessarily mean an idea like this will get picked up and executed as a commercial venture.

That’s the beauty of it though. “This is an incredibly unique project, and our [Drexel and the Orchestra] interests aligned—audience-building combined with the utilization of advancements at the forefront of technology,” says Youngmoo. In 2009 Drexel’s MET-lab began building the system from scratch, tackling the tracking technology, networking infrastructure, and scalability necessary to create a workable app; within this framework, Philadelphia Orchestra staff began producing the necessary content for a mobile and immersive experience.

By May 2010, Drexel and the Orchestra were ready to demonstrate proof of concept and chose to do so during a Philadelphia Orchestra SpectiCast broadcast. When the Orchestra performed Richard Strauss’s Don Quixote in Verizon Hall, several staff members and MET-lab representatives watched from Drexel’s campus and followed the music alongside in-depth program notes on iPod Touches that were handed out to audience members as a first demonstration.

The iNotes app has incredible potential as an educational tool. During an orchestral performance there’s a lot to take in: the physical experience of being in a hall and hearing an orchestra play, the history surrounding the repertoire, and the musicological dimension of each work, for starters. “This system provides a way of enlightening audiences. It has the potential to introduce new generations to classical music—to find something to latch onto, to have reasons to come back,” says Youngmoo. With the companion mobile app there’s a palpable level of excitement in the air as he talks about the future: “I think we’re only scratching the surface of the applications of technology to solving problems.”