Parker, celeste the effectiveness of two reading intervention programs nfaerj v 26 n3 2013

Parker, celeste the effectiveness of two reading intervention programs nfaerj v 26 n3 2013

• 1.
  NATIONAL FORUM OFAPPLIED EDUCATIONAL RESEARCH JOURNAL VOLUME 26, NUMBER 3, 2013 The Effectiveness of Two Reading Intervention Programs in a South Texas Urban School District Celeste A. Parker, EdD Special Education Supervisor San Antonio ISD San Antonio, TX Glenda Holland, EdD Professor and Department Chair Coordinator, Educational Leadership Doctoral Program Texas A&M University-Kingsville Kingsville, TX Don Jones, EdD Assistant Professor of Educational Leadership Department of Educational Leadership & Counseling Texas A&M University-Kingsville Kingsville, TX Abstract With illiteracy in the United States at a staggering rate, school districts are spending more and more instructional dollars searching for intervention programs to address students’ reading deficits. While, in the past, school districts have received funds through various state funding sources, the financial forecast for school districts is no longer conducive to implementation of multiple reading intervention programs. The present study of two intervention programs, READ 180 and Voyager Journeys III, in a south Texas urban school calculated the effectiveness of each reading intervention program. Students’ pretest and posttest performance on the Scholastic Reviewing Inventory (SRI) and the effect of the programs on Texas Assessment of Knowledge and Skills (TAKS) test results were analyzed. The results indicated that students enrolled in Voyager Journeys III had statistically significant higher results on Scholastic Reading Inventory post tests when using the pretests as the covariate. Students enrolled READ 180 had statistically significant higher test results on the 2011 ninth grade Reading TAKS scores when compared to students enrolled in the Voyager Journeys III for the same year. Keywords: reading, school improvement, accountability
• 2.
  NATIONAL FORUM OFAPPLIED EDUCATIONAL RESEARCH JOURNAL 2____________________________________________________________________________________________ The federal No Child Left Behind law (U.S. Department of Education, 2002) required school districts to implement interventions designed to ensure that struggling students could master the essential learning elements set forth by states and districts. One way to assist students in mastering those essential learning elements may be to move toward more student-centered philosophies (Woodward & Talbert-Johnson, 2009). Many slow readers may have learned basic sight reading but do not have the building blocks to learn new words nor are they exposed to these new words enough time needed to increase the terminology bank (Hirsch, 2003). In recognition of the need to improve reading instruction and consequently to improve lifelong opportunities for children, “The Individuals with Disabilities Educational Improvement Act (Mastropieri, 2010) permits school districts to use the “Response to Intervention (RtI) methodology that decides if the child reacts to scientific, research-based intervention as a part of the evaluation procedures” (Mastropieri, 2010, p. 25). In response to this need, school districts have been searching for research-based instructional programs that will help their students read more capably. The U.S. Department of Education (2002) indicates that 38% of students in the fourth grade nationally are not able to read at a fundamental level. These students are not able to read and comprehend a short paragraph. Reading difficulties are prevalent in children who are living in poverty. Many of these fourth grades students who cannot read attend low-income urban school districts. (Dufour, Eaker, & Karhanek, 2004). For students who have not learned to read at all, overcoming their early failures is even harder. Applegate, Applegate, and Turner (2010) described how the cause of this problem is frequently assessed: First is the assumption that the problem lies inside the students who are failing. Students who do not learn in the identical way as other students can be described as having specific or generalized learning disabilities. Second, the problem could be within the curriculum itself. This condition is often considered by school personnel as unlikely. After all, hasn’t the program been tested and proven under thorough research conditions in classrooms all through the country? Third, the problem could be between the program and the learner. This possibility is well recognized and respected among literacy educators, but it is also a fairly unusual assumption because of halo effect that surrounds the research base of the core program. Surely, if implemented correctly, there will be success for all but the most impaired students. (Applegate, Applegate, & Turner, 2010, p. 11) Research in the realm of Strategy Instruction reveals the application of this methodology to provide interventions similar to those that capable readers utilize (Biancarosa & Snow, 2006). Computer-Based Instruction Many of today’s high-school seniors have experienced a powerful change in instructional methods during their school careers. They began elementary school using paper, pencils, and books, but now they are doing much of their work with computers, CD-ROMs, World Wide Web browsers, web-based editors, and e-mail. These students have learned forms of literacy that were
• 3.
  CELESTE A. PARKER,GLENDA HOLLAND, AND DON JONES ____________________________________________________________________________________________3 not even conceived when they entered kindergarten. Given the rapid rate of change in the development of technology-based literacy instruction, it is likely that today’s elementary students will experience even more intense changes. In 1994, only 3% of U.S. K-12 classrooms had a computer connected to the internet; by 1998, this number had risen to 51%, and nearly every classroom had Internet access by 2000 (U.S. Department of Education, 2002). The application of computers to reading instruction has a relatively short history. The earliest published study (Atkinson & Hansen, 1966) arranged for students to access reading lessons on a mainframe computer located at Stanford University. The amount of time that students are expected to spend actively involved with a computer-based application to achieve reading outcomes also differs from one application to another. For example, some applications expect students to be dedicated to the computer for 15 to 20 minutes every day. Others call for 20 to 30 minutes every other day, and some let the individual classroom teacher determine the amount of time to be dedicated to the program (Schumm, 2006). “There are approximately 8.7 million fourth through twelfth graders in America whose chances for academic success are dismal because they are unable to read and comprehend classroom materials” (Kamil, 2002). When a student becomes frustrated with reading, reading will be avoided and the student’s reading ability will not improve (Cunningham & Stanovich, 2003). As struggling readers become more actively involved, their comprehension improves. READ 180 is intended to offer a reading intervention setting that maintains a student’s language and conceptual development through authentic task development (DeVivo & Aguhob, 2004). The Cognition and Technology Group at Vanderbilt University (1990) described READ 180 as one of the best new ideas for literacy development. Another computer-based learning system is Voyager Journeys III, a recent addition to Voyagers secondary intervention program series, Passport Reading Journeys. This additional level is for high-school students who are having problems with reading by integrating encouraging and high interest subject matter, adding a new online terminology module, and giving opportunities for career exploration and content area links, which will assist students to be successful in their core courses and on state assessments of comprehension, vocabulary, fluency, and writing (Voyager Learning, n.d.) Literacy skills are at the very center of educational development. Students who are not able to read become a great social concern and a drain on school districts’ remedial resources (“Learning Disabilities in Children,” 2010). The problem was to find the program best suited for remediating the reading deficits of struggling students. Students who struggle in reading can be supported by supplemental programs to assist them with their deficiency. In present-day educational programming, computer-assisted learning programs are frequently presented as cure alls. The two programs explored in this study were both computer-assisted learning programs, both programs offered individual and small group instruction. Both programs promised that students will become proficient in reading if they are exposed to this methodology. READ 180 offered direct reading instruction with age-appropriate content. High interest educational activities made certain that creative lessons and planned instruction take place (Scholastic, 2005b). READ 180 scaffold knowledge and assist students to generate mental images (Cognition and Technology Group at Vanderbilt, 1990). An experimental study of Voyager Passport was conducted in Miami using low achieving ninth- and tenth-graders with Limited English proficiency. Both schools and students were matched using the Florida Comprehensive Assessment Test (FCAT) pretests and other
• 4.
  NATIONAL FORUM OFAPPLIED EDUCATIONAL RESEARCH JOURNAL 4____________________________________________________________________________________________ standardized tests. Significant positive effects were found for ninth-graders, but no significant effects were found for tenth-graders. (Schneyderman, 2006). This study examined whether there is a statistically significant difference in the literacy skills of ninth-grade students enrolled in READ 180 reading program and Voyager Journeys III reading program. The following research questions guided this study. Is READ 180 or Voyager Journeys III the more effective reading program as measured by the 2010-11 Scholastic Reading Inventory pretests and post tests? Is there a difference between the 2010-11 TAKS reading scores of ninth-grade students enrolled in the READ 180 program and the TAKS reading scores of ninth-grade students enrolled in Voyager Journeys III? Research Methodology The research methodology was used for the purpose of this study is quantitative. “Quantitative research is a means for testing objective theories by examining the relationship among variables. These variables, in turn, can be measured, typically on instruments so that numbered data can be analyzed using statistical procedures” (Creswell, 2009, p. 13). Research Design The one-way Analysis of Covariance (ANCOVA) was used with one categorical independent variable and one continuous variable. The independent variable can consist of any number of groups (levels). The analysis of covariance (ANCOVA) is a general linear model tests whether certain variables have an effect on the outcome variable (Hartman, 2011). An Independent t-test was utilized to analyze the TAKS data. The TAKS data were collected through the Texas Open Records Act (2011). Therefore, any identifiable information concerning student identity was suppressed. Population and Sample The population was composed of ninth-grade students in an urban south Texas school district who were place in reading programs when their TAKS scores fell below 2100. The students enter into tier 2 intervention programs based on low Lexile scores. In this school system the students were enrolled in one of two programs, READ 180 or Voyagers Journeys III. Instrumentation For the first research question, pre test and post-test scores on the Scholastic Reading Inventory (SRI) were utilized. “The SRI is an adaptive reading assessment taken on the computer. The number of questions for each student varies and it can be administered to students in Grades K through 12. The SRI is based on the Lexile Framework for Reading with a design that is measured by norm-referenced and criterion-referenced reading tests” (Scholastic, 2005a, p. 21). For the second research question, results of the ninth-grade TAKS reading tests for 2011 were utilized. TAKS is the Texas state test that assesses a student’s skill levels beginning at the
• 5.
  CELESTE A. PARKER,GLENDA HOLLAND, AND DON JONES ____________________________________________________________________________________________5 third grade level and provides an exit level test at the eleventh grade. These skills are based on the Texas Essential Knowledge Skills (TEKS). The TEKS are the standard levels of education in coursework set forth by the State of Texas (Texas Education Agency, 2010). Reliability and Validity Reliability and validity of testing instruments have already been established for the Scholastic Reading Inventory, using a sample of 512,224 students in one state. On national norming studies, this state has shown like means and standard deviations to the population as a whole, making the sample appropriate for approximating national average. Since each student in the model has taken an assessment linked to the Lexile Framework, these national norms can be explained in Lexiles. The ninth-grade TAKS reading assessment is a criterion-referenced test. Committees of Texas educators identified those TEKS student expectation for each grade and subject area assessed what should be tested on a statewide assessment. These TEKS are objectives that are taught at grade level during the course of the year that are built on previous years’ objectives (Texas Education Agency, 2004). Procedures The study began by inquiring whether or not permission was needed from the superintendent of the school district to conduct the study. It was determined that the data were accessible through the Texas Public Information Act. The study was continued after receiving the scores from the south Texas urban school district. The ninth grade students’ scores were entered into the SPSS. SRI pretest raw scores were compared to posttest raw score results, using the ANCOVA to analyze the data. The ninth grade TAKS scores were accessed from the school district’s data base and were entered into SPSS. The independent t test was used to analyze the difference in TAKS scores of the two groups of students. Data Analysis Scholastic Reading Inventory. To answer the question regarding which whether READ 180 or Voyager Journeys III the more effective reading program, the 2010-2011 SRI pre test and post test scores were collected from the school district’s reading department. The data were entered into SPSS. To conduct this analysis an analysis of covariance (ANCOVA) was used with the Scholastic Reading Inventory pre test score as the covariate. Descriptive statistics are reported in Table 1.
• 6.
  NATIONAL FORUM OFAPPLIED EDUCATIONAL RESEARCH JOURNAL 6____________________________________________________________________________________________ Table 1 Descriptive Statistics for SRI Pretest and Posttest for Read 180 Test Mean Standard Deviation N Pretest 618.60 191.11 172 Post test 705.66 210.78 172 Group 1 participants were 172 ninth grade students enrolled in READ 180 in an urban south Texas high school. Table 1 represents the means and standard deviations of the Scholastic Reading Inventory (SRI) pretest and pretest scores. Table 2 Descriptive Statistics for SRI Pretest and Posttest for Journeys III Test Mean Standard Deviation N Pretest 591.89 196.27 114 Post test 774.65 210.78 114 Group 2 participants were 114 ninth grade students enrolled in Voyagers Journeys III in an urban south Texas high school. Table 2 represents the means and standard deviations of the Scholastic Reading Inventory (SRI) pretest and post test scores. Inferential statistics are reported in Table 3. Table 3 Inferential Statistics for ANCOVA Analysis for SRI Data Type III Sum Source of Squares df Mean Square F Sig Corrected Model 6624619.11 2 3312309.55 181.34 .000 Intercept 1958758.42 1 1958758.42 107.24 .000 PRETEST 6298336.92 1 6298336.92 344.82 .000 Program 547622.02 1 547622.02 29.98 .000 Error 5169135.49 283 18265.50 Total 165525852.00 286 Corrected Total 11793754.60 285 The inferential statistics indicate that overall there was a significant difference in Program effect [F (1,283) = 29.98, p < 0.000]. Based in the descriptive statistics, the students using the Voyager Journeys III program had a lower pretest and a higher posttest than the Read 180 group. The Pretest covariate F-value, F (1,283) = 344.82, p < 0.000, indicates that the pretest was significant in adjusting the posttest scores. Based on the descriptive data, students
• 7.
  CELESTE A. PARKER,GLENDA HOLLAND, AND DON JONES ____________________________________________________________________________________________7 participating in the Voyager Journeys III program were academically more successful over the year, as measured by Scholastic Reading Inventory scores than students participating in the Read 180 program. Texas Assessment of Knowledge and Skills Reading scores. To determine the difference between the TAKS reading scores of ninth-grade students enrolled in the READ 180 program and the TAKS reading scores of ninth-grade students enrolled in Voyager Journeys III, 9th grade TAKS scores for students enrolled in READ 180 and Journeys III were collected from the school district’s Research and Evaluation department. The TAKS data were entered into SPSS. The independent t-test was performed to determine if a statistically significant difference in the TAKS scores of students in the two programs existed. Table 4 provides the descriptive statistics for the analysis. Table 4 Read 180 and Voyager Journeys III Descriptive Statistics for TAKS Data Program Year Mean Std. Deviation N Journeys IIII 2011 2083.98 152.03 269 READ 180 2011 2154.54 184.22 105 Total 2011 2103.85 164.57 374 An independent t test was conducted to evaluate the hypothesis that there is a statistically significant difference between Reading TAKS scores for students who participated in READ 180 (m = 2154.74, sd =184.22) and Voyagers Journeys III (m = 2083.98, sd =152.03). The test was significant t (-3.50), p = .001. Students enrolled in READ 180 performed better on the TAKS reading test than the students enrolled in Voyagers Journeys III. The results of the SRI data analysis indicated that there was a statistically significant gain between pretest and posttest for students enrolled in the Voyager Journeys III when compared to the results of READ 180 SRI scores. This indicated that Voyagers Journeys III students did better than READ 180 students based on Lexile scores. The results of the TAKS data analysis indicated there was a statistically significant difference in student achievement by program enrollment, as measured by the TAKS reading scale scores. The results of the independent t test indicated there was a statistically significant difference in student achievement, with the READ 180 students scoring better. The conflicting results of the READ 180 scores and the SRI pretest and posttest may be that the TAKS is a criterion referenced test and the SRI is a norm referenced test. The tests may assess different constructs. These different constructs may also be mirrored in the READ 180 and the SRI scores. Conclusions There is no conclusive evidence to support either reading intervention program. The variables that could have affected the outcomes of the analysis could be sample sizes in the sample group; circumstantial events in the administration of the pre and posttest, and fidelity and integrity to the presentation of the material to the students. Student attendance and participation
• 8.
  NATIONAL FORUM OFAPPLIED EDUCATIONAL RESEARCH JOURNAL 8____________________________________________________________________________________________ could also have affected the outcome. There are different studies concerning the impact of fidelity and integrity to the program. A study of READ 180 and instructor divergence found that instructor divergence from the program affected the impact of the READ 180 results (Goin, Hasselbring, & McAfee, 2008). Also, the two reading programs are created to work with relatively small groups of students. Implications of the Study The outcome of this research cannot support the use of either reading intervention program; however, the study can be used as a platform to conduct other studies that explore the effectiveness and integrity of reading intervention programs. Future studies should investigate comparisons of other reading intervention programs before a costly commitment is made to a particular reading intervention program for any school district. A control group would be beneficial to enable study of scores between the control group and the experimental group. Future studies could include longitudinal investigations on the State of Texas Assessments of Academic Readiness (STAAR) End of Course Reading portion of the test, and reading intervention effects on socioeconomic status, ethnicity, language proficiency, and special education populations. Also, further examination needs to be given to the alignment of reading intervention programs with standardized tests as intervention program results may not accurately be reflected by the tests, which may not measure what interventions are designed to teach. References Applegate, A., Applegate, M. D., & Turner, J. D. (2010). Learning disabilities or teaching disabilities? Rethinking literacy failure. Reading Teacher, 64(3), 211-213. Atkinson, K., & Hansen, O. (1966). Computer assistance with the educational process. Review of Educational Research, 36(5), 588-603. Biancarosa, C., & Snow, C. E. (2006). Reading next—a vision for action and research in middle and high school literacy: A report to Carnegie Corporation of New York (2nd ed.). Washington DC: Alliance for Excellent Education. San Antonio: Intercultural Development Research Association. Creswell, J. W. (2009). Research design qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage. Cognition and Technology Group at Vanderbilt. (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(6), 2-10. Cunningham, A., & Stanovich, K. (2003, November-December). Reading can make you smart. Principal, 34-39. DeVivo, K., & Aguhob, M. (2004). Compendium of READ 180 research. New York, NY: Scholastic. Dufour, R., Eaker, R., & Karhanek, G. (2004). Whatever it takes: How professional learning communities respond when kids don't learn. Bloomington, IN: Solution Tree.
• 9.
  CELESTE A. PARKER,GLENDA HOLLAND, AND DON JONES ____________________________________________________________________________________________9 Goin, L., Hasselbring, T., & McAfee, I. (2008). Executive summary: DoDEA/Scholastic READ 180 Project: An evaluation of the READ 180 intervention program for struggling readers. Washington DC: U.S. Department of Defense. Hartman, J. (2011). ANCOVA (A GLM Approach) [Power Point slides]. Retrieved from http://bama.ua.edu/~jhartman/689/ancovaglm.ppt Hirsch, E. D., Jr. (2003). Reading comprehension requires knowledge—of words and the world: Scientific insights into the fourth-grade slump and the nation’s stagnant comprehension scores. American Educator, 27(1), 28-29. Kamil, M. L. (2002). Adolescents and literacy: Reading for the 21st century. Retrieved from http://ierc.siue.edu/documents/AdolescentsAndLiteracy.pdf Learning disabilities in children: Learning disability symptoms, types, and testing. (2010). Retrieved from http://www.helpguide.org/about_rotary.htm Mastropieri, M. A. (2010). Literacy and Learning (Vol. 23). Bingley, England: Emerald Group. Scholastic. (2005a). READ 180: Placement, assessment, and reporting guide. New York, NY: Author. Scholastic. (2005b). READ 180: Based on scientific research. New York, NY: Author. Schneyderman, A. (2006). Some results of the Voyager Passport Reading Intervention System in several school districts. Miami, FL: Miami-Dade County Public Schools, Office of Evaluation and Research. Schumm, J. S. (2006). Reading assessment and instruction for all learners. New York, NY: Guilford Press. Texas Education Agency, Student Assessment Division. (2004, August). TAKS Information Booklet Reading Grade Nine. Retrieved from http://www.tea.state.tx.us/WorkArea/DownloadAsset.aspx?id=2147487752 Texas Education Agency. (2010.) Texas Education Agency. Retrieved from http://ritter.tea.state.tx.us/student.assessment/resources Texas Open Records Act. (2011). Retrieved from http://www.txstate.edu/about/open_records_act.html U.S. Department of Education. (2002). NCLB policy. Retrieved from http://www2.ed.gov/policy/elsec/guid/states/index.html Voyager Learning. (n.d.). Passport Reading Journeys Program overview: Beginnings, I, II, & III. Retrieved from http://www.voyagerlearning.com/cs/Satellite/prj/doc/PRJ_ Overview?packedargs=overviewID%3D1277944504729&cmsid=Voyager Wilkes, M., & Bligh, J. (1999). Evaluating educational interventions. British Medical Journal, 318(7193), 1269-1273. Woodward, M., & Talbert-Johnson, C. (2009). Reading intervention models: Challenges of classroom support and separation instruction. Reading Teacher, 63(3), 190-201.