Roundup 4: Effectiveness of Tier 2 Interventions in RtI/MTSS

It has been a while since I last posted a Roundup, I’m sorry for those that look forward to these. My work has been quite busy lately working with schools to implement Multi-Tiered Systems of Support. Due to this work I have been thinking a lot lately, and reading some, about how best to implement MTSS. This seems to be a difficult question to answer, a lot of research is done in controlled conditions and what might work in those conditions might not in the real world. One of the articles I read and decided to share in this roundup speaks to that exactly.

All four articles in this roundup discuss the effectiveness of interventions at tier 2 and, in a couple of the articles, the authors make recommendations for more effective interventions.

I have two more roundups in the works, all having to do with MTSS. This one is all about tier 2 effectiveness, the next will be about MTSS implementation, and the final in the series will be about RtI decision making (I may switch up that order).

As usual, feel free to contact me using any of the methods on the sidebar. A PDF version of this is also on the sidebar.

Enjoy!

Bouck, E. C., & Cosby, M. D. (2017). Tier 2 response to intervention in secondary mathematics education. Preventing School Failure, 61(3), 239–247. https://doi.org/10.1080/1045988X.2016.1266595

In this article the authors describe tier 2 intervention models in secondary schools, particularly in math. I found this article to be very timely for me, as one of the schools I am working with at the moment is a middle school and is wrestling with this exact problem, how does a secondary school best implement RtI? This is challenging in the secondary schools for a number of reasons. I found this quote on page 241 especially informative:

Inherent features of secondary education, including the structure of the day (e.g., changing class periods and the limited time per class) and other issues, can result in RtI implementation—and research—being more challenging (Fuchs, Fuchs, & Compton, 2010; Prewett et al., 2012). Resistance among educators was also noted as a challenge with using RtI models in secondary schools (Sansosti, Noltemeyer, & Goss, 2010). Another issue with RtI implementation at the secondary level is that fewer evidence-based interventions exist, but most mathematics interventions are focused on elementary students (Sansosti et al., 2010).

The authors propose four models for RtI in secondary schools at tier 2:

1. Having an additional math class, either in the form of an extended block or additional period, in which all students can receive further intervention and not miss tier 1 instruction. A challenge with this model is that it might have implications on contact time for other classes and could become a challenge for credits and graduation. Not in the article, but something I thought of as a challenge when reading this method was the challenge of teacher schedules, what do you take away in order to give this extra period?
2. Small group pullout instruction, which sounds great, but also comes with many challenges. One of the overwhelming positives is that students can receive small group instruction targeted to their needs. One of the major challenges is logistics, who provides these interventions and where? Does the school need to hire another teacher, or teachers?
3. Alternative Math Class, which involves students taking an alternative to their tier 1 math class. In this model students are grouped according to needs and are given instruction according to their specific gaps. A major downside is that this leads to tracking and students may never make it back to tier 1 instruction. This goes against the principles of RtI and MTSS in that students should be receiving intervention as long as they need, but not forever. It also removes the problem solving nature of MTSS.
4. Technology tools, like apps and programs on computers, can be used to supplement instruction. This could be beneficial for students to focus on individual needs, but requires access to, and a certain level of fluency with, technology.

Given all of the pros and cons the authors conclude that the additional math block time has the most potential for changing how students learn math. In these blocks teachers can group students according to need and give them the additional instruction they need. The authors go on to state that, regardless of the delivery model chosen, it is important to “consult the research to select and validate interventions chosen for implementation.” Given the limited research, the authors give four examples of evidence-based practices for tier 2 math in secondary schools:

1. Explicit instruction, which they describe as “An instructional sequence involving the teacher first modeling how to solve the mathematics and then cuing the students to solve the problem as needed, and finally the student solves the problems independently”. See Archer or Hollingsworth for more on Explicit Instruction. I’ll add my own little bit here on explicit instruction, I argue, based on multiple books and studies, that explicit instruction is the foundation for tier 1 instruction for students who struggle, come from poverty, ELLs, and SWDs. I was a little surprised to see it listed here as tier 2.
2. Concrete-Representational-Abstract (CRA) instructional sequence. “The Concrete-Representational-Abstract instructional sequence is an explicit instruction model in which students are first provided instruction with concrete manipulatives to use to solve math problems. When students are successful, they move into solving the math problems with representations or drawings. Finally, students solve the problems abstractly (without concrete manipulatives or drawings)”.
3. Schema-based instruction. “Teachers help students to understand word problems by categorizing problems via structure, such as change or compare problems. Students then create diagrams to depict the schemas to solve. e.g., FOPS (find the problem, organize the information via a diagram, plan to solve, and then solve)”.
4. Number talks. “the teacher poses a math problem. First, students think how to mentally solve the problem and then share the multiple solutions they devised Builds conceptual understanding and fluency (i.e., number sense)”

Shameless plug: I actually made a short video quickly describing CRA and SBI (2 and 3) for my colleagues. It is on YouTube here. Not one of my best, but you can get a sense of it from the video.

Overall this article helped me by validating the struggles schools have in implementing RtI in secondary schools, by validating some practices that I had learned about through my work on a state wide work group, and by stressing the importance of finding evidence-based practices. I can see giving this to those that want to conceptualize RtI in their schools, its a good introduction.

Bouton, B., McConnell, J. R., Barquero, L. A., Gilbert, J. K., & Compton, D. L. (2018). Upside-Down Response to Intervention: A Quasi-Experimental Study. Learning Disabilities Research & Practice, 33(4), 229–236. https://doi.org/10.1111/ldrp.12171

To some this will sound ridiculous, but when I received the latest issue of Learning Disabilities Research and Practice and saw this article, I got excited. As the title states, the authors used a quasi-experimental design to find out if inverting the tiers of intervention was more effective than traditional methods. For those that are confused, which may be nobody, the traditional method of RtI or MTSS is that everyone receives tier 1, core instruction, all of the time. When a student or students are struggling they are identified using either a universal screener or in class assessments to receive interventions in addition to their tier 1 instruction. Those that are still struggling move to tier 3. In this model the researchers explored if, using “dynamic RtI” or upside down RtI (udRtI), a student could skip tier 2 and go directly to tier 3, and if so, would that student perform better after intervention?

As the authors acknowledge, this is not the first time this has been tested. A group of researchers in 2014, Al Otaiba, Kim, Wanzek, Petscher, and Wagner, tested “dynamic RtI” vs traditional RtI and found no significant differences between the two for fall reading scores, but found significantly higher scores for the dynamic group in spring. This study differentiated itself in that tier 3 was delivered as one-on-one tutoring whereas the study in 2014 was groups of 1-3 students. The authors argue that one of the potential benefits of this model is that it “allows students to work one-on-one with a tutor more quickly than with the current RTI framework. The tutors are able to work with the student independently to foster these growth mindset ideas, instead of the student becoming more frustrated in a group setting where they still feel that they are behind their peers academically” (p.230).

Design

This was a small study (each group was 24 first graders) that was conducted as an exploratory study during data collection of a much larger study on RtI. The authors used word-level reading skills as their measure of student growth. This was considered a quasi-experimental study because the classrooms were randomly selected, the intervention group from two urban public schools in the southeastern US. The data were collected in the 2009-10 school year. The control group was selected from nine urban public schools in the southeastern US with data collection happening during the 2006-09 school years. All students were teacher identified as low performing and met other criteria. All students received similar interventions and the process of deciding when to move between tiers was the same, except for the fact that the experimental group in this study went to tier 3, intensive individual intervention first, then went to tier 2 when ready.

Results

For the measures of word reading, sight word efficiency and word identification, the results were statistically significant with fairly large effects (d=.87 and d=.74 respectively). For measures of decoding, phonemic decoding efficiency and word attack, the results were not statistically significant. The authors hypothesize that the results for decoding would be statistically significant with a larger sample size. This study shows that, for word identification, udRtI was effective and it is promising for decoding skills.

My takeaway

This was an interesting study for research sake, but does not tell me that we need to change the way we operate yet. If I were working on a phd or for a university I might work on a follow-up study with much larger sample sizes. This could change the way RtI is done everywhere, and if it has a positive impact on kids I can only see a positive. I guess we will have to wait and see if this is expanded.

Clarke, B., Doabler, C. T., Smolkowski, K., Baker, S. K., Fien, H., & Strand Cary, M. (2016). Examining the Efficacy of a Tier 2 Kindergarten Mathematics Intervention. Journal of Learning Disabilities, 49(2), 152–165. https://doi.org/10.1177/0022219414538514

This study was completed by researchers who have created two math programs for Kindergarten. First they developed the Early Learning in Mathematics (ELM) core curriculum, which consists of 120 lessons focused on four key strands:

1. Number and Operations
2. Geometry
3. Measurement
4. Vocabulary

This core curriculum has proven effective in previous studies.

The curriculum for this study, the second program they have created and studied, is called ROOTS. ROOTS was developed as a tier 2 intervention to be used in conjunction with ELM and focuses on Number and Operations. The authors argue that the focus on Number and Operations is key to success in math as this is the foundation for all future understanding. This study focused on two questions:

1. “What is the impact of the ROOTS program on the mathematics achievement of at-risk students?”
2. “Do ROOTS students reduce the achievement gap with their non-at-risk peers by making greater gains than their non-at-risk peers?”

Design

This was a quasi-experimental study with full day Kindergarten classrooms who participated in the ELM study randomly assigned to the ROOTS condition or the control condition. The researchers controlled for teacher ELM experience by grouping teachers with one year of experience with the program together and randomly assigning them to treatment and control, then doing the same for teachers new to the program. This was a nice way to prevent bias and questions about actual effectiveness. This study used a total of 29 classrooms, 14 in treatment condition (ELM+ROOTS) and 15 in the control (ELM only). “Teachers were asked to nominate the five lowest performing students or those who would most benefit from a small-group math intervention.” In total 67 students participated in the intervention and 73 in the control. All students involved in the study received tier 1 instruction using ELM and students in the treatment group received the ROOTS intervention during independent practice time of ELM skills 3 times per week.

Lessons were delivered by 14 trained instruction assistants and were 20 minutes long. Fidelity of implementation was measured throughout to ensure that the program was being followed as written.

The intervention

ROOTS is a small group tier 2 intervention of 50 lessons that is delivered 3 times per week for 20 minutes each session. In total the intervention lasts between 16 and 20 weeks. I’ll let the authors describe the program further:

The goal of ROOTS is to support students’ development of procedural fluency with and conceptual understanding of whole number concepts.

and

Each lesson consists of 4 to 5 brief math activities that center on three key areas of whole number understanding: (a) Counting and Cardinality, (b) Number Operations, and (c) Base 10/Place Value. Curricular objectives advance students from an initial understanding of whole number through more sophisticated aspects of whole numbers in kindergarten mathematics.

and finally:

A central feature of the ROOTS program is its explicit and systematic approach to instruction… ROOTS incorporates the principles of instructional delivery that have been empirically validated to improve the mathematics achievement of at-risk learners and stu- dents with learning disabilities (Baker et al., 2002; Gersten, Beckmann, et al., 2009; Nelson-Walker et al., 2012). These delivery principles include modeling and demonstrating what students will learn, providing guided practice opportunities, using visual representations of mathematics, and delivering academic feedback.

Results

The researchers used two tests to measure growth and compare treatement and control: the Test for Early Mathematics Ability (TEMA) and the Early Numeracy Curriculum-Based Measurement (EN-CBM). On both conditions the effect was small to medium, g=.375 and g=.301 respectively. The authors conclude that this intervention may help close the gap, as was intended.

My Takeaway

While this might have been effective in this study, I want to see a lot more evidence that this program works before I recommend it to teachers. I also question the effectiveness of the ELM program, the way they described the results in this article it did not seem to be so beneficial.

Coyne, M. D., Oldham, A., Dougherty, S. M., Leonard, K., Koriakin, T., Gage, N. A., … Gillis, M. (2018). Evaluating the Effects of Supplemental Reading Intervention within an MTSS or RTI Reading Reform Initiative Using a Regression Discontinuity Design. Exceptional Children, 84(4), 350–367.

This article was the most interesting of the four that I read, though I did not think it would be when I read the abstract. The reason I found this so interesting was because this study was done in the context of a state-wide MTSS initiative in Connecticut; which sounds a lot like the initiative I am working on. This study targeted grades 1 to 3 to evaluate the effects of providing tier 2 supplemental intervention to students identified as struggling through a universal screener. The authors give an excellent review of the literature in this article, which is worth a read if you are interested in this topic. I am not going to write anything from that section, the actual study provides enough information to write about! I will include one quote that I found really important:

It may not be enough to just have MTSS practices in place if they do not significantly increase the level of instructional intensity. Carefully controlled studies evaluating the efficacy of reading interventions are often able to ensure this level of consistency and intensity in implementation. Schools implementing Tier-2 interventions as part of district or state MTSS initiatives, however, may not.”

This quote sums up the research to practice gap so well, what might work in research because of the controlled nature of those studies might not work in the real world. This study evaluates tier 2 in the real world.

Method

Schools were selected to participate in a pilot phase of a statewide reading initiative focused on MTSS in reading. All four schools were different in size and demographics. Researchers used DIBELS (phoneme segmentation fluency, nonsense word fluency, and oral reading fluency measures) to screen students and identify those in need of intervention. 318 of the 678 students in grades 1-3 were identified as needing intervention (47%) and 360 students received only classroom instruction. The researches selected 395 students, 205 receiving the intervention and 195 in classroom only instruction, to be part of the analysis sample.

In order to provide these interventions to students the research team hired 17 reading interventionists to support the four schools, three schools received four interventionists and one school received five. A little aside here, when I read this I made a note on the paper “must be nice!” Can you imagine actually having 4 extra interventionists to help?! These interventionists were hired by the state and were assigned as full-time employees to the schools for 3-5 years. They attended all meetings and PD activities to try to become members of the school community.

For the intervention the team selected Proactive Early Interventions in Reading (P-EIR). I had never heard of this intervention but went to the What Works Clearinghouse entry in the bibliography to see effectiveness for myself. This does seem to be an effective intervention.

Interventions were delivered in groups of 3-5 students base on benchmark scores. Some students were assigned to cross-grade groups and school-based teams made decisions to move students to other groups every 10 weeks based on the student’s response to the intervention. Intervention sessions were 30-40 minute session four days per week over the course of the year.

Interventionists and literacy coaches received 30 hours of PD on the P-EIR program from the developers and interventionists were supported by school-based literacy coaches throughout the year. Coaching was an integral piece in this intervention and the authors describe their process in detail on page 357.

Results

This tier 2 framework had a statistically significant impact on phonemic awareness and decoding outcomes (d=.39 and d=.36 respectively). The authors also describe their results using the improvement index the What Works Clearinghouse uses. Using this index “students who received Tier-2 intervention accelerated their performance on phonemic awareness by 18 percentile points and on decoding by 14 percentile points beyond what their performance would have been if they had only received Tier-1 classroom instruction.”

Surprising to me was that “Tier-2 intervention had no overall discernible effects on reading fluency and comprehension.”

The authors conclude that overall, tier-2 interventions in the MTSS framework do have a positive effect on student outcomes and that the “Results of this study suggest that increased instructional intensity may be a central mechanism related to the efficacy of Tier-2 reading interventions implemented within MTSS frameworks”.

My Takeaway

The authors claimed that this was as close to real world conditions as they could get, and I believe that statement. However, I cannot help but think that having 17 extra interventionists dedicated to the schools, with 30 hours of PD, and a coaching structure in place had some effect. This study, while informative, and confirming for me, also makes me think of the importance of building proper systems before even attempting to bring in new interventions. Coaching structures, team structures to be able to move students, data systems, and proper assessments are all key components that cannot be skipped.