Research

Measuring Horizons Success

During summer 2010 Horizons Greater Washington teachers and students made great strides towards closing the achievement gap. With students returning to the program each summer, summer learning loss is being mitigated and on average, academic skills are increasing and students’ attitudes towards learning are positively increasing.

Preventing Summer Academic Decline

 

Reading

• Horizons students gained 3 months, on average, in reading level.
• Horizons Middle School students gained 5 months, on average, in reading level.
• Summer 2010 students read, on average, 8 books during the six-week program.
• Reading as few as four to five books over the summer can prevent a decline in reading achievement scores from the spring to the fall. Source: Kim, Jimmy. “Summer Reading and the Ethnic Achievement Gap.” Journal of Education for Students Placed at Risk. 9.2 (2004): 169–188

Math

• Horizons students gained 5 months, on average, in math level.
• Horizons Middle School students gained 9 months, on average, in math level.
• Piloting a new STEM initiative, fifth and sixth graders participated in an engineering workshop using the NXT kits for robotics to build and program their own robots.

Improving Readiness for Learning

Attendance

• Over the six-week summer program, student attendance rate exceeded 90%.
• 90% of students returned from summer 2009.
• 92% of students answered with a definitive YES!, when asked if they would come back to Horizons next summer.
• 100% of parents surveyed said they would enroll their child in Horizons next year.

Reinforcing Achievement

• Horizons 3:1 student to teacher ratio fosters provides positive interactions between students and caring faculty members.
• 96% of students answered with a 4 or higher when asked how much they enjoyed their summer at Horizons. Scale based on 1 to 5, 5 equaling “Loved it!”
• 94% of parents answered with a 5 when asked if they were satisfied with their child’s summer experience at Horizons. 6% answered with a 4. Scaled based on 1 to 5, 5 equaling “Completely.”
• 100% of Horizons students learned to swim, carrying over their confidence to the classroom, and convincing students that they can achieve so much more.
• Weekly field trips and partnerships, such as collaboration with Glen Echo Park, offer eye-opening cultural experiences.

Evaluations of Horizons National at Yale University

“In studies my colleagues and I have conducted over the last 25 years, Horizons has consistently demonstrated a positive effect on children’s attitudes toward learning.”

- Dr. Edward Zigler, Zigler Center for Child Development
and Social Policy, Yale University, Founder of Head Start

Horizons National supports its Affiliate programs through consistent monitoring and evaluation, working with Yale University since 1985 to develop and implement effective evaluative protocols throughout the organization.

Recent Evaluations

In 2006-2007, Horizons conducted a reading achievement testing pilot to determine what tools were most likely to be able to measure changes in students’ reading skills over the six-week summer program. Initial results were very positive, showing an improvement in reading skills for those performing below grade level, and a prevention of the summer slide for students who would otherwise have lost skills.

In 2007 a comprehensive evaluation was conducted by Dr. Beth Daponte, Senior Research Analyst at Yale University and an evaluation consultant. The approach taken was to first rigorously describe the program using Program Logic Models and a diagram of the Program’s Theory of Change, followed by a literature review on issues pertinent to the program. STAR reading and math achievement tests results were analyzed, and school year attendance rates were studied.

Key results included:

• Students performing below grade level during the school year gained an average of 4 months of reading skills during the six-week summer program. Many gained as much as a full year.
• Horizons provides an opportunity for children to maintain and often advance their academic and non-academic skill set in ways that will ultimately increase a child’s likelihood of lifetime success.
• Horizons Affiliate Programs maintain fidelity to the original model while allowing for local flexibility in implementation.

Previous Evaluations of Horizons at Yale

In the early 1980’s, long before formal program evaluation became the norm for non-profit service providers, Horizons began to work with Dr. Edward Zigler (currently Director Emeritus of the Zigler Center in Child Development and Social Policy) at Yale University to investigate the impact of Horizons’ program model on the low-income students it served. Dr. Zigler, one of the founders of Head Start, has a long and very distinguished career centering on the education of children, particularly those from low-income families, and is an expert on evaluation of programs such as Horizons. Dr. Zigler conducted two comprehensive evaluations on the New Canaan program in the early eighties and the early nineties. Both studies demonstrated that Horizons was fulfilling its goals, and revealed areas where the program might be improved.

Conclusions from those studies included:

• Motivation to learn improves over time for students who attend Horizons
• Horizons prevents summer math decline; many students post significant summer gains and preserve the advantage during the school year
• Horizons positively affects its students’ attitudes and learning
• Horizons has a positive impact on attendance rates during the school year
• Horizons students exhibit an increased ability to focus on academic tasks

In 2000, five years into its site expansion, Horizons instituted a rigorous program of evaluation to ensure fidelity to the original model by each of its affiliates. This protocol was continued for 5 years, and helped establish common guidelines, best practices, and a shared systems among Horizons Affiliates.

Future Evaluation

Horizons National will continue to work with Yale to assist affiliate programs in tracking progress towards meeting program goals, monitoring activities and services, and demonstrating their impact, including longitudinal achievement assessment in reading and math.