To establish the Knight Case Studies Initiative (KCSI) at the Columbia University Graduate School of Journalism to develop and distribute journalism case studies to enhance journalism education and create future journalism leaders.
ASSESSMENT PURPOSE & APPROACH
To guide the focus of the evaluation, a set of five questions was identified based on the expected outcomes of the Initiative. The five overarching questions that guided the evaluation are:
• How are the Knight cases being used?
• To what extent do users perceive the Knight cases to be of high quality? Which of the case studies do users perceive to be most and least useful?
• How do journalism students respond to the use of the Knight cases?
• What outcomes are evident for participating students?
• To what extent do journalism faculty find the Case Method to be an effective instructional approach?
The assessment examined case adoption and impact by interviewing 25 teachers at different universities and surveyed 90 teachers using or familiar with KCSI cases.
Evaluator: Metis Associates
• Case Adoption – KCSI marketing data showed that its cases have been taught in 130 courses attended by over 2,000 students. Furthermore, 293 teachers from 194 colleges and universities registered on KCSI’s website to use cases. Columbia originally designed the cases for graduate coursework; however, 55% of surveyed users strictly taught undergraduate courses, compared to 29% who applied cases purely in a graduate setting. Ensuring grantees budget adequate time and resources for these kinds of activities is important.
• Case Quality & Satisfaction – Teachers responded positively to the quality of the cases, with 65% rating case quality 9/10 or 10/10. Furthermore, 92% of teachers using cases planned to do so again. Teachers found the case method and the specific cases provocative and engaging.
• Content & Relevance – Teachers appreciated the quality and thoroughness of the materials but noted ways to strengthen the content. Some teachers, especially those who used free cases, cited low content relevance to their courses as a barrier to adoption. The length of cases deterred adoption for others, and some suggested KCSI develop a few shorter cases to facilitate integration into teaching plans.
• Student Impact – Teachers reported that using the case method had some or much impact on students’ critical thinking skills (82%), judgment skills (80%) and communication skills (75%). Many felt the cases provided students with a glimpse into the working practices of media organizations and issues faced by professionals.
• Web-based Case Management System – Case access and distribution cases emerged as a significant area of concern. Several professors interviewed were unaware of KCSI’s model, which requires each individual student to register and pay for cases on the KCSI website. It appears likely that several professors have distributed hard copies of cases to students, raising major intellectual property and revenue model concerns. Though KCSI recorded $2,000 in case sales in September 2010 (an increase from an average of $300/month in Spring 2010), revenues have fallen short of initial estimates.
To extend the original grant as an evaluation is done on distribution and sustainability of the Knight Case Studies Initiative at Columbia University
Knight Foundation aims to help sustain democracy by leading journalism to its best possible future in the 21st century. @michaelmaness on strategy
As part of our grant making process, program teams work with grantees to establish indicators that will be tracked to provide feedback on project implementation and outcomes. In certain cases, we also partner with grantees to conduct in-depth third-party evaluations to understand the effectiveness and impact of specific projects.