The program that I have decided to evaluate is the Science of Reading program, Reading Horizons. Reading Horizons provides training and curriculum for teachers to implement the science of reading within their classrooms as well as in a virtual setting. Reading Horizons also offers intervention programs for grades 4 and up to help students who struggle with decoding. The program offers teacher training, interactive software and direct instruction manuals. I work in a K-12 school with approximately 400 students. Our elementary teachers have decided to implement this program. The K-3 teachers will be implementing the full curriculum while the grade 4-6 teachers are interested in using the intervention programming to assist with the struggling readers in their classes. Our school is located in a rural area of Saskatchewan and the demographic of our students varies from affluent families to lower class families, and all in between. Our school's goal for using this program is for students in grades 1-6 to improve their phonemic awareness and reading skills. Our staff is currently in the planning process about how this will be gauged.
Resources for this program include: - teacher training software -teacher manuals (daily lessons and curriculum included) - classroom posters -letter cards (alphabet and sound) - Reading Horizon support website -common word flashcards -flip charts -discovery games - chapter assessments and skill checks https://www.readinghorizons.com/
Evaluation Questions
The purpose of this program evaluation is to determine if the Reading Horizons® program raises our grades 1-6 students' reading scores throughout the year and if so, in what ways? My evaluation questions are: 1. Does this program help all students to advance by at least one F and P reading level in a school year? 2. Are the resources adequate and accessible for all students so that they can practice their decoding and phonemic awareness skills daily? 3. Is the program helping students to reach their own learning goals(grade level or otherwise specified in an IEP) within the school year? 4. Is the program curriculum assisting teachers in teaching their students' reading and decoding skills?
Program Theory
Theory of Action: Kindergarten to grade 5 teachers will receive training on how to use and teach the Reading Horizons® Discovery curriculum. They will then implement Reading Horizons® Discovery curriculum into their classroom teachings daily. Teachers will support students at various reading levels by helping them understand language structure, phonics and teach them new decoding and word marking skills. Students will have the opportunity to learn and regularly practice new phonemic awareness skills. Theory of Change: By the end of the school year, all kindergarten to grade 5, students will advance by at least one Fountas and Pinnell™ reading level. (Fountas and Pinnell is the current preferred testing program of this particular school division). Evidence: The Reading Horizons® program helps every student build a solid foundation in reading and prevents general education students from falling behind (Elementary Reading & Spelling Curriculum (readinghorizons.com). Studies such as, Project SOS and REST, have shown that all students will show significant improvement with their reading skills when using the Reading Horizons® program (The Effectiveness of Reading Horizons as an Intensive Remedial Program).
Potential Logic Model
Evaluation Approach
Identify and Describe The evaluation approach that would best suit the implementation of the Reading Horizons® program is an outcome/effectiveness evaluation. This is the first year that Reading Horizons® is being implemented in this school. Implementation evaluation is not desired in this case because the implementation of Reading Horizons® curriculum is very prescribed. Outcome evaluation is appropriate in this case because it measures program effects in the target population by assessing the progress in the outcomes that the program is to address (CDC). This type of evaluation is useful after the program has made contact with at least one person in the target population and will show whether the program is being effective in meeting it’s objectives (CDC). This evaluation is to contribute to the quality of the program and to maximize its usefulness as it is an Evaluation for Learning and Use (Green, 2021). Rationale The purpose of this evaluation is to determine whether or not students are improving by one F & P reading level within the span of one school year when engaging in daily Reading Horizons® lessons and activities. The evaluation questions listed above will guide this evaluation.
Both qualitative and quantitative data will be collected during this program evaluation.
Qualitative Data
In regards to qualitative data, data will be collected through direct sources being the teachers and students. This data will be obtained through interviews, observations and staff-surveys. This data will be collected from all staff and from a sample of 8 students from each participating class. This data will be analyzed by grouping shared experiences, opinions and statements in order to answer the evaluation questions. Anecdotal notes will be taken from observations, and voice recordings will be taken of interviews to assist with making accurate anecdotal notes. Anecdotal notes will be put into excel to analyze by looking for repeated key phrases and shared experiences and opinions.
Quantitative Data
In regards to quantitative data, students' reading scores will be analyzed at various times throughout the school year. Scores will be analyzed from previous years, students' scores from the beginning of the year before starting the Reading Horizons Program, in the middle of the year while being fully immersed in the program and at the end of the school year. It will be determined if their reading scores have advanced and to what capacity. These scores will be recorded and tracked through Microsoft excel. This will keep data organized and easy to view. Qualitative will be graphed in order for evaluators to critically analyze the findings.
Data Collection Matrix
Enhancing Evaluation Use
This evaluation of the Reading Horizons® program will have an impact, as it will be determine if the program is benefitting the learning needs to students in grades K-6, specific to the area of reading and phonemic awareness. By using excel, the data from this evaluation will be clearly visible in graphs and will be easily accessible to school staff.
The stakeholders of this program are 6 teachers and two principals and the program is working with approximately 132 students ranging from grades K-6. The results of the evaluation will be presented through a Google Slideshow which will clearly display both the qualitative and quantitative data along with the results of the evaluation.
The data and findings from this evaluation should show next steps for the program in what needs to be changed or improved upon. The data should also show what is working well and what strengths the program has.
Collaborative meetings with all teachers and admin involved in the program will occur to establish next steps. In these meetings, stakeholders will make meaning of the results and establish a plan on how changes and improvements will be made.
Commitments to Standards of Practice
By using the evaluation for learning and use approach to evaluation, this evaluation aims to make recommendations for improvements to the Reading Horizons® program if necessary and to identify the specific needs of teachers and students in terms of teaching phonemic awareness and learning phonemic awareness skills. Utility Standards: The utility standards are intended to ensure that an evaluation will serve the information needs of intended users. This evaluation for learning and use will serve the information needs of the intended users, the teachers. The teachers completing this evaluation are competent and capable of performing this evaluation. Feasibility Standards: The feasibility standards are intended to ensure that an evaluation will be realistic, prudent, diplomatic, and frugal. The means of collecting data for this evaluation are mainly interviews and observations. These will be done in a timely manner and will keep classroom disruptions to a minimum. It is expected that individual students will have different experiences with the program. The evaluation will be effective and will produce information and data of value. Propriety Standards: The propriety standards are intended to ensure that an evaluation will be conducted legally, ethically, and with due regard for the welfare of those involved in the evaluation, as well as those affected by its results. This evaluation will be conducted in a way that will protect the rights and privacy of the children involved. The evaluation will be complete and fair and record the strengths and weaknesses of the Reading Horizons® program. The evaluation findings will be available to all stakeholders. Accuracy Standards: The accuracy standards are intended to ensure that an evaluation will reveal and convey technically adequate information about the features that determine worth or merit of the program being evaluated. The Reading Horizons® program has been described accurately. The purpose and procedures of this evaluation have been explained in detail so that the data and findings can be fairly and accurately analyzed. The procedures of this evaluation have been purposely chosen so that their findings are accurate and reliable.