The Musical Theater Project Demonstrates the Value of Building Evaluation into Programs from Day One

One girl and two boys growl like tigers while wearing smock-style costumes.

Students participate in a Kids Love Musicals! residency. Photo credit: Heather Meeker

When leaders at the Musical Theater Project in Northeast Ohio decided they wanted to expand their Kids Love Musicals! residency program to serve students with disabilities, they were deliberate in their planning. They sought out resources and expertise from peer arts organizations already working with students with disabilities, and they attended professional development sessions on arts and special education topics. As they laid out their expansion plan, they identified program assessment as a priority and sought to include comprehensive evaluation strategies as a part of the new residencies.

With this in mind, Heather Meeker, Executive Director of the Musical Theater Project (TMTP), connected with leaders at the Schubert Center for Child Studies at Case Western Reserve University (CWRU), located nearby in Cleveland, Ohio. “CWRU is interested in being deeply involved in their community, so developing a mutually beneficial research project was of great interest to them,” says Meeker.

The Schubert Center introduced Meeker to psychology professor Sandra Russ and doctoral student Olena Zyga, who agreed to work with TMTP to assess the new residencies. TMTP agreed to support the academics’ work by raising money to pay for student researchers and faculty time, and Meeker says funders have been especially interested in supporting this collaborative assessment.

The Kids Love Musicals! residencies for children with disabilities aim to teach social skills and emotional understanding through the stories and characters from classic American musicals such as The Wizard of Oz, The Jungle Book, and You’re a Good Man, Charlie Brown. The multi-year evaluation project with the Schubert Center seeks to better understand if engaging in the residency program impacted participants’ socioemotional skills, including the ability to make eye contact, engage with others, take turns appropriately, and demonstrate emotional understanding. A secondary goal is to understand whether gains seen during the residency program extend to other environments.

Russ and Zyga created a custom measurement scale for the program, using their expertise in the fields of psychology and play. TMTP initiated their new residencies for students with disabilities, collecting multiple forms of data throughout. Residency sessions were videotaped across multiple school sites and to include a range of student ages and ability levels; the videos were then coded and scored according to the measurement scale. Teachers were also asked to report on the same variables that were being coded in each session for every student, both before the residency program began and after it had finished.

Analysis of the first round of data, which specifically focused on The Wizard of Oz residency, suggests that students who participated in the Kids Love Musicals! program did make gains in eye contact, turn taking, engagement, and symbolic flexibility. These results were recently published in the Journal of Intellectual Disabilities. Meeker is thrilled that their collaboration with the Schubert Center led to the research being shared broadly, both through journal publication and in various conference presentations by her and Zyga.

Four children stand in front of two adults, all wearing curly gold ribbon on their heads and making roaring faces.

Teaching artists work with students in the Kids Love Musicals! residencies. Photo credit: Heather Meeker

The research collaboration between TMTP and the Schubert Center continues post-report publication, including a new round of data collection focused on identifying if similar gains are seen across curriculums presented to students. Specifically, they are asking if children made the same gains while learning The Jungle Book as made while learning The Wizard of Oz. Analysis of this data is currently underway, with initial results suggesting that curriculum differences do not significantly impact the student outcomes. A final phase of data collection, completed at the end of the 2016-2017 academic year, focused on comparing the active residency period with a pre-residency control period.

Given the success of their collaboration with the Schubert Center, Meeker encourages organizations interested in conducting robust program evaluations to consider partnering with a college or university in their own community. “If a project can be designed with the idea that both the organization and university students can benefit from it, a collaboration can really be a win-win situation,” she says.

Of course, Meeker also warns of the hard work and complications that come with conducting a large-scale assessment. She explains, “We had to make peace with the fact that we would not get 100% compliance from teachers in our data collection efforts, and that not all of the data we worked so hard to collect would ultimately be used in the study. We also did not anticipate the delays that sometimes come with working with a university, like waiting for internal review board approvals for everything from project proposals to parent permission forms.”

But the reward for that hard work is great, Meeker says, as their research has clarified so much for TMTP about the program internally. She concludes, “If you are constantly looking to improve your work, then thorough evaluation is crucial. This project has empowered us to do even more with our programming.”

Advertisements

10 Great Reasons to Attend the LEAD® Conference

Thinking about attending the annual Leadership Exchange in Arts and Disability (LEAD®) Conference? Here are the top 10 reasons why it’s a smart choice for professional development for arts administrators of every level.  

  1. Access to ideas. Tap into the collective “brain trust” of arts administrators from museums, theaters, parks, zoos, libraries, and other cultural venues.

    lead8

  2. Become indispensable to your organization. What you’ll take away from LEAD® will show current and future employers that you’re knowledgeable and dedicated to inclusion and accessibility in the arts. Plainly put, it’s a great resume builder.

    lead12

  3. Networking. Connect with arts managers from around the world who have similar passions and knowledge of accessibility.

    DSC00676_SBurzio cropped

  4. One-of-a-kind professional development. No other conference focuses on accessibility in cultural venues like LEAD® does. The conference provides an intimate, rich atmosphere, where arts professionals of all levels can learn and share what they know.

    lead4

  5. Experiential opportunities. LEAD® provides the opportunity to experience accessibility services and programs. Through our pre-conference Capacity Building Workshops and optional performances with accessibility services, you can see access in action!

    lead7

  6. Access to experts. The Kennedy Center engages leading thinkers in the field to present at the conference. And after the event, attendees are invited to join an exclusive listserv to continue the conversation and ask questions.

    lead5

  7. Straight from the source. Get ADA and accessibility law information directly from the Department of Justice.

    lead9

  8. Practical information. You’ll bring ideas and practices to your organization that can be implemented right away.

    lead11

  9. Something for everyone. LEAD® has sessions and workshops for beginners and the more experienced. It doesn’t matter what your background or knowledge level is – we have sessions for you.

    lead10

  10. Registration candy. Say hello to the staff and help yourself to our always-stocked registration candy basket. If this doesn’t sell you on attending, we don’t know what will…

    lead1