ASD: Autism Spectrum Disorder; IT: Immediate Treatment; JASP/ER: Joint Attention Symbolic Play and Emotion Regulation

The earlier that evidence based treatments are implemented with children who have autism the better their long-term developmental outcomes [1]. Besides age, however, the content of these early interventions appears to be very important. Children with autism show specific deficiencies in some skills and not others. In the area of social communication , responding to and initiating joint attention are core features of impairment [2]. Initiating Joint Attention (IJA) refers to the way young children use nonverbal gestures such as pointing to share meaning about an idea or object with a social partner, whereas Responding to Joint Attention (RJA) refers to a child acknowledging another’s gestures to convey interest [3].

Both responding to and initiating joint attention skills have been associated with later language outcomes in children with autism [4]. For most young children, IJA is much more difficult to teach and many intervention studies do not separate IJA from RJA, presumably due to low rates of IJA [5]. IJA skills in children with autism can improve with intervention [6] and these IJA gains can improve long-term language outcomes [7,8].

Several studies have now demonstrated improvement of joint attention skills when expert therapists teach children [6] or when parents are coached by therapists [9]. Less common are effectiveness studies in which teachers are taught to improve joint attention skills in children with autism. Somewhat surprising has been the lack of focus on effectiveness studies, given that the majority of early autism interventions are implemented in applied settings [10].

The preschool context is extremely important for implementing evidenced based autism treatments because children spend significant portions of every day in school and large numbers of children attend public preschools, thereby affording under-resourced and underrepresented populations of children access to a potentially large dose of evidence based interventions. Moreover, teachers are ideal agents to implement an intervention at preschool. The communication and social/emotional delays that children with autism often display already qualify them to receive a range of educational services [10] and many public preschool teachers report that they do not use ‘scientificallybased’ or ‘evidence based’ strategies [11,12]. Additionally, teachers often report that they are very interested and motivated to learn evidence based autism practices [12].

There is tremendous potential in having teachers implement a packaged early autism joint attention intervention, especially since we know most preschool based autism practices are not evidenced based. For example, preschool teachers tend to respond to very few communicative gestures of young children with autism [13,14], despite the documented importance of explicitly acknowledging communication bids [15]. When teachers do respond to the communication of children with autism, they typically do not respond in ways that will facilitate better social communication [16]. Similarly, teachers of children with autism typically prompt for verbal communication much less than is recommended for childrenwho are typically developing or developmentally delayed [17].

There are likely a number of challenges preventing teachers from providing more opportunities for social communication in the classroom [18]. In addressing these challenges for teachers, it is important to recognize that the intervention packages themselves may have some strategies that are easier to implement than others and that there may be different effects on child outcomes when strategies are applied in the classroom. Currently, we have little information on evidence based strategies that are easier for teachers of young children with autism to implement. More specifically, to our knowledge, there are no studies regarding the differential ability of teachers to implement the various strategies of a manualized social communication intervention. Indeed, even when teachers are taught to implement a new intervention, many studies do not include measures of implementation fidelity [19]. Both ease of implementation and noticeable effects on child outcome will influence how sustainable an evidenced based practice will be.

This study was concerned with understanding the extent with which teachers were able to implement a packaged joint attention intervention in their public school classroom. The packaged intervention (Joint Attention, Symbolic Play, and Emotion Regulation: JASP/ER) has been previously tested in randomized controlled trials executed by expert therapists [6], parents [9] and teachers [20]. However, the implementation of specific strategies across different contexts (1:1 versus in a group) has not been explored. Thus, this study examines teacher use of strategies for implementing the joint attention intervention in a 1:1 context with the child and in the classroom as a whole both before and after an intervention to teach the specific strategies.

Design

This study concerns the same participants in a randomized waitlist control study aimed at teaching teachers to improve joint attention skills in their young children with autism [3]. In this study, preschoolers with autism were randomly assigned to a group that received the experimental treatment as soon as the study began (IT) or a group that received the experimental treatment six weeks after the study began (DT). Randomization occurred at the unit of classroom and was conducted for each preschool. Our first study found that teachers in the experimental treatment used significantly more treatment strategies than teachers in the control group and that preschoolers in the experimental treatment used more joint attention and sustained longer periods of joint engagement than preschoolers in the control group. The study methodology is briefly described below to place the current study in context.

Participants

This study includes 16 dyads of preschoolers with ASD and the teacher or paraprofessional who worked with the preschooler. All children from the original study were included in this study. Twenty one children were consented for the study. However, five were excluded based upon the study’s established inclusion and exclusion criteria. Table 1 details the participant characteristics. There were no significant group differences on the cited entry characteristics.

Intervention

Teachers in the IT treatment were taught how to implement JASP/ ER, a manualized integrated developmental-behavioral intervention. IT teachers participated in two 30 minute intervention sessions led twice a week for six weeks by the first author (Kathy Lawton). These intervention sessions took place during already occurring classroom play activities. The Process Model of Sustained Program Implementation for Teachers [21] as well as our manualized coaching protocol [3] guided the instruction of teachers. Interventionist IT fidelity was high (99%).

Measures

Two measures were used to assess teacher fidelity: a classroom observation measure as well as a taped play interaction. The classroom observation measure assessed fidelity in a group setting, whereas the taped play interaction assessed fidelity when the child was 1:1 with the teacher. Blinded assessors naïve to treatment assignment collected the data for both measures. These coders were graduate students in Educational Psychologywho were specifically trained to code the occurrence of these strategies.

Table 1: Characteristics of children, teachers, and classrooms at entry.

The same coding scheme was utilized throughout both ten-minute assessments. Blinded coders noted whether teachers displayed any of the 11 JASP/ER strategies during each of the one minute intervals. All teacher behaviors were ones implemented by the interventionists in Kasari and colleagues’ studies [6,7,9] following the child’s lead, imitating the child’s play actions, waiting for communication, using contingent language, establishing play routines, violating play routines, expanding play routines, modeling joint attention skills, prompting for joint attention, promoting eye contact, and setting up the environment. These behaviors reflected the skills the teachers were taught during the packaged experimental intervention and are skills that are believed to directly improve joint attention. These skills are operationalized in a prior publication [3].

At the beginning and end of the study, teacher fidelity was assessed during ongoing playtime classroom interactions. The teacher was instructed to interact with the child and other students as she typically would. As with prior studies [2], reliability was assessed using Intra- Class Correlation Coefficients (ICC). Reliability was acceptable between two coders. Teacher performance on each of the 11 strategies are detailed: follow child’s lead (1.00), imitate child (.583), expand on child’s play actions (.847), waiting for communication (occurrence too low to score), contingent language (.834), violate routines (.500), model joint attention (.951), prompt for joint attention (occurrence too low to score), prompt for eye contact (occurrence too low to score), routine building (.928), setting up the environment (.928).

The teacher was also videotaped playing with the child whom she was assigned to work with in a 1:1 context when the study began and when the treatment concluded. Prior to the start of this taped play interaction, the teacher was instructed to play with the child as he or she typically would in an empty room at the preschool. A standard set of toys were used: a shape sorter, ball, blocks, two firemen dolls, a fire dog, a barn, four animals, train tracks, trains, doll bed, doll table, pretend food, child sized dishes, child silverware, child glasses, and a chef hat. The same protocol was used by Kasari and colleagues [6,7,9]. Reliability was conducted on 20% of the taped play interactions. Interclass correlations were acceptable: follow child’s lead (.945), expand on child’s play actions (1.00), waiting for communication(.960), contingent language (.982), violate routines (occurrence too low to score), model joint attention (.965), prompt for joint attention (occurrence too low to score), prompt for eye contact (occurrence too low to score), routine building (.640), setting up the environment (.994).

Teacher perception of the experimental intervention was assessed using a short questionnaire at the conclusion of the intervention. In this questionnaire adapted from prior studies by Kasari and colleagues, teachers responded to 23 questions using a five point likert scale. The scale assessed how feasible each IT teacher believed the intervention was within a classroom context, her perception of intervention mastery, and how much she thought the intervention induced any observed targeted child outcomes.

Figure 1: Study procedures.

Procedure

The study was comprised of two time points that were separated by six weeks. Figure 1 depicts the procedures that were followed at each of the study time points. The figure omits detail regarding child assessments, since this paper is specifically concerned with teacher fidelity.

This study investigated the ability of preschool teachers to implement strategies associated with a packaged social/communication intervention (JASP/ER) during regular classroom activities. Fidelity data from the taped play and classroom observation were analyzed using a series of ANCOVAs, descriptive statistics, and qualitative data analyses. In each ANCOVA, group was a fixed factor and the covariate was entry performance of the investigated JASP/ER strategy.

Beginning fidelity

Prior to the start of the intervention, teachers utilized few intervention strategies in their classroom and showed mastery of just a few intervention strategies on the taped play interaction. As can be seen in table 2, teachers utilized all of the 11 intervention strategies during the classroom observation; however, they occurred in less than one observational interval.

At entry, teachers in both groups demonstrated competence implementing three strategies during the taped 1:1 play interaction. The strategies of following the child’s lead, contingent language, and setting up the environment all occurred close to the majority of the interval. Following the child’s lead occurred during an average of 6.00 intervals (SD=3.28) for the IT group and an average of 6.57 intervals (SD=3.41) for the DT group. The IT group used contingent language an average of 5.56 intervals (SD=2.70) and the DT group utilized this strategy a similar amount, M=4.14, SD=3.67. Setting up the environment occurred an average of 5.33 intervals (SD=4.35) for the IT group and an average of 4.14 intervals (SD=4.26) for the DT group. It is worth noting that modeling joint attention skills occurred during an average of 2.33 intervals (SD=3.12) for the IT group and an average of 2.14 intervals (SD=1.35) for the DT group. All other strategies were used close to an average of 0 intervals.

Table 2: Average number of intervals teachers and paraprofessionals used each validated intervention strategy (max of 10).

Treatment effects

During the exit class observation, there were significant group differences in the following intervention strategies: following the child’s lead (F(1, 13)=9.29, p<.001; d=1.79, CI=-.61-3.13), imitation (F(1, 13)=8.76, p<.05, d=1.66, CI=-.65-2.24), expanding play routines (F(1, 13)=5.31, p<.05, d=1.58, CI=-.36-1.58), modeling joint attention (F (1, 13)=8.56, p<.001, d=2.68, CI=1.11-3.26), establishing routines (F(1, 13)=12.15, p<.01, d=2.13, CI=.18-2.41), contingent language (F(1, 13)=11.25, p<.005, d=1.58, CI=-.91-2.28), and setting up the environment (F(1, 13)=12.09, p<.01; d=1.90, CI=-.35-3.58). There were no group differences in the other 3 strategies. When the bonferroni correction was applied to each of these analyses, significant differences remained for 4 of the strategies: following the child’s lead, modeling joint attention, establishing routines, and setting up the environment.

During the exit taped 1:1 play interaction, significant group differences existed for following the child’s lead (F (1, 13)=8.59, p<.05 .12, d=1.43, CI=.63-3.33) and imitation (F (1, 13)=6.12, p<.05.28, d=1.22, CI=-.92-2.97). There were no group differences for any other intervention strategies. When the bonferonni correction was applied, there were no statistically significant group differences in following the child’s lead or imitation.

Perception of intervention

Overall, IT teachers and paraprofessionals rated the intervention highly. The mean score on the 5 point likert perception scale was 4.18 (SD=1.69). As shown in figure 2, the teachers and paraprofessionals perceived the intervention to be effective with the child and reported feeling competent using strategies to promote joint attention. The feasibility of the intervention was rated as not overly difficult or overly easy (M=3.28).

This study investigated the ability of preschool teachers to implement 11 strategies of the packaged JASP/ER intervention. Before the intervention started, teachers rarely used any of the intervention strategies in their classrooms, yet they were able to use three strategies during the 1:1 taped play interaction. There were treatment effects for the majority of intervention strategies on the classroom observation and fewer treatment effects on the taped play interaction. Teachers rated the intervention highly.

Figure 2: Perceived feasibility, competency, and effectiveness of the experimental intervention for the IT group (n=9).

This pilot investigation suggests that the validated experimental intervention can alter the behavior of a new intervention agent (teachers). It is noteworthy that the intervention seems to be able to alter the behavior of teachers, given the abundance of teacher training obstacles and the difficulties at implementing evidenced based practice in a school setting [22]. Furthermore, because preschool teachers spend a large amount of time with children who have autism, having teachers implement a validated early autism intervention is low-cost and potentially higher-dose in comparison to clinician-mediated interventions.

Before the intervention started, all teachers were able to implement more intervention strategies during the taped 1:1 play interaction than the classroom observation measure. The taped play assessment was conducted in a separate, quiet room in the preschool where teachers were able to focus all of their attention on the preschooler with autism. In contrast, the classroom observation was conducted during ongoing classroom activities where teachers were responsible for the instruction of multiple children. If teachers knew any of the JASP/ER strategies prior to the start of the intervention, it only seems logical that they would be able to at least implement them during the optimal assessment context of the 1:1 taped play interaction. Indeed, both groups of teachers were similar in their strategy use in the 1:1 interactions.

Because IT teachers only knew how to implement a few JASP/ER strategies under ideal conditions, the intervention needed to teach the teachers how to implement these strategies in the less predictable and more demanding classroom environment. Thus, the focus of the intervention session was not just on instructing teachers how to implement a brand new skill, but also how to utilize a skill that they knew in an ideal setting to a less predictable environment. It appears that the IT teachers learned how to generalize the intervention strategies across play settings due to the fact that the IT teachers ended up using a similar number of strategies in both contexts.

Interestingly, at exit, IT teachers implemented more strategies in their classrooms than they did in their taped play interactions. These findings suggest that IT teachers might not have been able to fully generalize all strategies to a novel setting. The teachers seem to have been most skilled at implementing the intervention strategies in the environment that they were taught in, their classrooms.

All IT teachers were taught all 11 intervention strategies, yet most IT teachers and paraprofessionals only used about seven of these strategies. No IT teacher or paraprofessional implemented all 11 intervention strategies during either the class observation or the taped play interaction. This finding suggests that some JASP/ER intervention strategies are more ‘transportable’ from clinicians to teachers than others. In particular, following the child’s lead and imitation appear the most transportable. This lack of transportability could be due to several factors. The strategies themselves may be too difficult for teachers to implement at the same time as other teaching responsibilities. Additionally, the coaching methodology may not have been the best way to teach these difficult intervention strategies. Furthermore, teachers may have intentionally not used some of the strategies; most teachers adapt the interventions that they are taught by university groups for a variety of reasons, such as their buy-in of an intervention or their perceived need of how the intervention must change in order to work in their classroom [23,24]

Treatment effects existed for only two strategies across both contexts: following the child’s lead and imitation. Future investigations should ascertain why these intervention strategies appeared to be easily learned by teachers. It is possible the strategies themselves might be more feasible than the other 9 strategies. However, the teachers may simply have bought into the strategies more and been more committed to implementing them. Additionally, the order with which the strategies were taught might be an explanation. Both strategies were taught within the first three weeks of the intervention and the teachers may have benefited from the extra practice time with these strategies.

The experimental intervention was favorably perceived by the IT teachers and paraprofessionals. IT teachers and paraprofessionals reported believing that the intervention induced gains in child social communication and felt competent at implementing the intervention strategies. The mean score for the feasibility domain was the lowest. Teachers and paraprofessionals rated the feasibility as the middle score of the five-point likert scale, not overly difficult nor overly easy.

It is well-acknowledged that little is currently known regarding the most influential and efficient methods for transporting empirically supported treatments to communitysettings [24] and that there is a large need for researchers to identify more successful ways to adapt empirically supported treatments to the real-world [25]. Thus, future investigations should examine how best to instruct teachers on the various intervention strategies; teachers may benefit from learning the different intervention strategies differently than this intervention and be aided by sessions that are specifically tailored to the preschool setting. It is also possible that fidelity would be higher if a simplified version of the university-based treatment was presented [24,26]. Likewise, it is also important to determine how important each intervention strategy is for producing joint attention gains [27] and to better understand how fidelity influences child treatment effects. Finally, future studies will benefit from larger samples so that moderators of treatment efficacy can be examined, such as specific interventionist and child characteristics.

Public preschool teachers were successfully taught how to implement strategies shown to lead to improvements in the joint attention of young children with autism. Teachers increased the number of strategies that they used in an attempt to promote the joint attention of preschoolers across multiple play contexts and they favorably perceived the intervention. Two strategies were particularly transportable: following the child’s lead and imitation of child actions. Initial difficulty in demonstrating strategies in the classroomsignificantly improved with treatment. This intervention holds promise for preschool teachers being able to successfully implement an intervention aimed at improving the core autism deficit of joint attention.

We acknowledge the support of school district personnel, Kim Johnson, Sonia Dickson-Bracks, and Becca Dannelley. We also appreciate the participation of parents, teachers and paraprofessionals and research staff (Jilly Chang, Marina Farberov, Janet Bang, Eric Ishijima, Nancy Huynh, Sara Levitt, Monica Perez Guerrerez, Tanya Paparella and Amanda Gulsrud). We are thankful for the consultation of Dr. Jeffrey Wood, Dr. Carollee Howes, and Dr. Thomas Wiseman.