At ArtsConnection in New York City, evaluation is more than just a buzzword used in funding applications; it is an essential part of the arts education organization’s programs. Deputy Director for Education Carol Morgan says that ArtsConnection evaluates its programs in two ways: through working with external, independent evaluators and through practitioner research facilitated by ArtsConnection staff. This two-pronged approach has provided them with a wealth of valuable information over the years, and their work continues with an ongoing study of the Spectrum Musical Theater school residency program for students on the autism spectrum.
Morgan says program development at ArtsConnection begins with a needs assessment process, when her staff meets with teachers to discuss their artistic and educational goals for their students. “We start to build a shared language and shared concern,” explains Morgan, “…and that in itself is an inquiry process. Planning and reflection are basic level inquiry.”
ArtsConnection’s Spectrum Musical Theater program grew out such an inquiry process with Public School P94M, also called the S.P.E.C.T.R.U.M. School. This school in New York City’s District 75 has 8 sites in lower and midtown Manhattan with 300 children in kindergarten–high school. Students at P94M have autism or other disabilities, including emotional and behavioral disturbances and intellectual disabilities. ArtsConnection and the P94M principal discussed how they could best collaborate to help students achieve their goals.
The Spectrum Musical Theater teaching artist residency program is a three-year pilot project at P94M. Program Manager Emily Lukens says the first year was about experimenting to create a program that focused on social and emotional literacy through musical theater. The second year was about fine-tuning, and now in the third year, they are focused on thoroughly evaluating the program and collecting relevant data.
Lukens says the program evaluation has two main components: an external evaluator conducting research specifically on how movement lessons impact students, and internal assessments facilitated by ArtsConnection staff and completed by the teachers and teaching artists. The internal assessment includes frequent “reflection sessions” between teachers, teaching artists, and program staff. According to Lukens, “These reflection sessions inform the next session and allow for discussion of both individual students and the group as a whole.”
As another part of the internal Spectrum Musical Theater evaluation, ArtsConnection staff created a custom student assessment form by merging one of their existing arts education forms with an assessment tool P94M uses to measure student growth. The new form assesses observable behaviors in an arts classroom as well as any growth in the student’s social and emotional skills.
The new Spectrum Musical Theater assessment tool is being completed for five students from each residency class, four times each throughout the residency. Lukens says they selected students on a variety of levels for the evaluation, including “…some higher functioning students, some lower functioning, children who are verbal and non-verbal, and students who have expressed interest in the arts and some who haven’t at all. Having this large range helps train our teaching artists and school teachers on how to really look at what students are doing, what their reactions are to certain music, movement, or directions, and how this work may be impacting their choices and lives.”
In addition to the student evaluations, teachers and teaching artists also provide assessments of the program itself. The educators enter all their observations into a spreadsheet to provide a big picture analysis at the program’s conclusion.
Lukens says that ArtsConnection hopes to replicate Spectrum Musical Theater in another multi-site school, which is why incorporating data and assessment into the program is so critical. But Morgan adds that program evaluation data is valuable for their organization in so many ways. For instance, their Board of Directors has found previous evaluation reports helped them better articulate and contextualize ArtsConnection’s work.
Morgan urges other arts education leaders to not just conduct evaluation studies, but to also be consumers of each other’s research. “It is so important for the field to think not only about what we do but how we do it and why we do it, and put it in a broader context,” she continues, “What’s underneath those promising practices? … [R]eading and discussing other work, research, and thought leaders can greatly improve our own work.”