The Five Ws + an H of Program Evaluation

By Erin J. Hoppe

Whether you work at an organization with dozens of employees or just one, evaluation is essential to accountability, transparency, and continuous improvement. I come from the later of these dichotomies, but will always prioritize evaluation as a way to measure our success and work smarter. With limited funds but unlimited demands, it is essential to take a critical look at our work. Here are a few tips on how to make evaluation work, no matter the organizational circumstances.

  1. Why – Start here. Aside from the few reasons previous noted, you need to identify the specific reasons why evaluation is important for your program. Are you trying to understand the impact, increase efficiency and effectiveness, or demonstrate value to stakeholders? Having this answer will help you make a plan and address the rest of the evaluation process.
  2. What – This matters a lot because it shapes your research question and strategy. Clarify what you want to learn from this process and what you will do with the information. This is more specific than the “why.” What you want to know will determine what data you collect, from who, and how—do you need a pre-post-test or interviews?
  3. Who – No one is an island and no evaluation has ever been conducted by a single person. Someone is providing the data you are collecting. Someone is analyzing the information. Someone is expecting a report on the results. Build a team to help you get through the process and always over-thank participants for the extra work you are asking of them.
  4. Where – Is this evaluation taking place in your building or schools across the state? The lines of communication between administrators and participants should be wide open and responsive. Think about providing the evaluation in multiple formats and make sure there is a clear path from data collection to analysis to reporting.
  5. When – Evaluations can be a short survey after an event or span several years. Either way, I make the same recommendation for evaluation as I do for accessibility: it should be a line item during planning meetings and in the budget. This doesn’t have to mean spending more than you can afford, but it does demonstrate value.
  6. How – Large institutions might have a team with “evaluation” in their job description and funds to make it happen. Others need to find funders and outside experts. Either way, with a clear “why” and “what” the work will happen.

The best advice I can offer in program evaluation is to be thoughtful, flexible, and tenacious. Whatever the scope of your project, the results should inform your practices (even if they aren’t what you expected), and just might move the field forward so we all learn something new. I look forward to reading your findings.

 

Erin Hoppe's headshot

VSA Ohio Executive Director Erin Hoppe

Erin J. Hoppe is approaching her ten-year anniversary as executive director of VSA Ohio (www.vsao.org). Her background in evaluation includes work at VSAO, The Ohio State University, American Institutes for Research, and the Smithsonian Institution. She is a board member for Columbus Arts Marketing Association, Ohio Citizens for the Arts, and ADA Ohio. If you can’t find her in the office, she is probably working on a home improvement project or bird watching.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s