Cantor-Fitzgerald Center for Research on Diversity in Education:
Final Report Summary of Evaluation Findings
I. Executive Summary
The following summary offers a synopsis of the findings of a formative evaluation project to assess the effectiveness of the ADL's A WORLD OF DIFFERENCE® Institute comprehensive pilot program in three public school districts in New York, Missouri, and New Jersey. The study was designed to track the activities in each site for eighteen months (August 1998-December 1999). This external evaluation report was conducted under the auspices of the Cantor-Fitzgerald Center for Research on Diversity in Education at the University of Pennsylvania Graduate School of Education. The Center, established in 1996 through a generous grant from the Cantor-Fitzgerald Fund, provides ongoing evaluation of the Institute's programs and scrutinizes its training and curriculum materials to ensure their efficacy. Joan and Allen Bildner and the Bildner Family Foundation provided additional support for the comprehensive pilot program.
The ADL's A WORLD OF DIFFERENCE® Institute Comprehensive Program was designed to include the following components:
- Establish and train a Diversity Team to lead initiatives to create systemic change relating to diversity issues within each district;
- implement A CLASSROOM OF DIFFERENCE™ training programs for teachers and administrators;
- introduce and train staff with the ADL's Anti-Bias Study Guide - Secondary Level;
- train parents and families of students in the schools;
- launch a Peer Training program for students; and
- provide extensive on-going consultation from ADL staff.
While the implementation of the Comprehensive Program varied somewhat dramatically from site to site (reasons are detailed below), each clearly benefited from the experience. The overall findings of the study were extremely favorable with respect to the quality and impact of the ADL Anti-Bias Study Guide, the Peer Training program, the ADL trainers and training programs offered to the sites. The greatest areas of need were reflected in the challenges presented by the timeline for implementation of the pilot program, the change in staffing both in ADL and in each site, the varying levels of commitment on the part of school staff and administration, and the ability of ADL staff to effectively respond to the particular needs and goals of each site.
II. Evaluation Findings
Review and Use of the Anti-Bias Study Guide
The Anti-Bias Study Guide was seen as a key component in the successful implementation of the pilot program. Initially, all sites experienced reluctancy on the part of staff to incorporate this resource into their curriculum. The key to overcoming this resistance appears to have been the successful alignment of the materials to existing curriculum within the school. Once teachers were able to see how it could be used within their existing course, they became increasingly willing and excited about the material and its use with students. The actual implementation of lessons, however, did vary from site to site.
In conjunction with the pilot program, an independent external panel was formulated to offer critique of the ADL's A WORLD OF DIFFERENCE® Institute Anti-Bias Study Guide (Secondary Level). This panel was comprised of secondary level teachers and college-level professors. Panelists were selected based on their expertise in curriculum, multicultural education and teaching.
The findings of the panel were very favorable. Overall, they rated the Anti-Bias Study Guide a 4.2 on a scale of 1(lowest) to 5 (highest). General conclusions were:
- The Anti-Bias Study Guide offers a comprehensive approach to anti-bias education.
- The Guide offers a "user friendly" format for teacher's use in the classroom.
- The Guide is easily infused into a standards-based curriculum.
- The lessons of the Guide are appropriate for all students and can be easily used within a variety of courses.
Comments from the panelists:
"I found this Guide extremely easy to follow and understand. Not only were the procedures clearly written and logically organized, but the resource lists, handouts, and discussion questions greatly support appropriate instruction, conceptual depth and student engagement."
"The Study Guide links so directly to the kinds of thinking and work current standards emphasize - namely analysis and critical thinking, performance and experiential learning, and integration of multiple disciplines."
"Students (using the Guide) learn about bias conceptually, they think about it personally, they learn how to recognize it in their environments and lives, they study its history, they examine examples of social action, and they engage in planning for initiating social action themselves."
Students who were engaged in lessons from the Anti-Bias Study Guide reported significant meaning and impact from the material:
- 81% of students reported the lessons help increase their understanding of others viewpoints that are different from their own.
- 76% reported the lessons help them communicate with students different from themselves.
- 66% reported the lessons help them develop a broader view of issues and problems in society.
- 62% reported the lessons help them reflect or think about their own biases and prejudices.
- 78% reported the lessons help to increase their respect for students in their school who are different from themselves.
Impact of Peer Training Program
Of all the elements of the comprehensive program, the Peer Training component has the most direct and intense impact on students. The response to this program was overwhelmingly positive. It is seen be teachers and administrators as the program which can be most easily maintained - and flourish - after the conclusion of the pilot study. Surveys completed in two of the three sites by Peer Trainers suggest that students see major benefits of the program. Specifically, students reported:
- Improved self-confidence in leading a presentation
- Increased understanding of the impact of discrimination and prejudice
- Increased ability to communicate with others who are different from themselves
- Increased confidence in speaking out on issues of bias to fellow students
When asked about the benefits of being a Peer Trainer, some of the comments heard were:
"I have realized my own prejudices and am working them out with my peers"
"I have built a stronger patience level for myself. Being a member has helped me be aware of the racial conflict around me and how to deal with situations."
"(As a Peer Trainer) people really listened to what we were discussing…A lot of new areas were introduced and I know some of (my peers) were hesitant but did open up to us. That's a great thing in itself because they acknowledge that there are issues out there."
This program showed me...I don't have to be like other people. I could be myself and could love who I am, for what I am. It also makes me put an emphasis on what I believe in … and gave me more respect for my peers."
Overall Impact of the Pilot Program
The participating sites found that training and assistance provided by the A WORLD OF DIFFERENCE® Institute staff was above average to excellent. Many reported that the CLASSROOM OF DIFFERENCE™ training for the staffs of the schools offered well-needed information and skills to sensitize them to diversity issues. Administrators reported that their staff members have an enhanced capacity to understand diversity and bias and to conduct training.
None of the three sites were able to implement all components of the pilot program. However, they were able to institutionalize specific aspects of the program, which will continue on after the conclusion of the pilot program. The Diversity Team will continue in two of the three sites. All three sites plan to continue with the Peer Training Program, with one site now offering the Peer Training program as an intervention strategy to reduce disciplinary problems. One site has integrated some of the lessons into the social studies curriculum and another is relying on teachers to use the Guide voluntarily in their classrooms.