Professional Development Program for the Question Exploration Routine (Manual & Flash Drive)

Description

Research on the Professional Development Program for the Question Exploration Routine (a CD Program)

Overview
The Question Exploration Routine is used by teachers to help students arrive at the answer to a critical question. To test the effects of a CD program for instructing teachers in how to use the Question Exploration Routine, a study was conducted with 20 general education teachers who taught grades 4 through 12. They were randomly selected into an experimental or a control group. For the experimental group, ten teachers worked through this CD program (hereafter referred to as the “virtual workshop” [VW] group) to learn how to use the Question Exploration Routine. They used the routine in their general education classes with a total of 133 students, 33 of whom had learning disabilities (LD). Ten teachers (hereafter referred to as the “actual workshop” [AW] group) participated in a live workshop and received instruction from one of the developers of the Question Exploration Routine. These teachers used the routine with a total of 129 students, including 35 students with LD. The purpose of the study was to determine whether the virtual workshop (i.e., the CD program) was as effective as an actual live workshop. A multiple-baseline across-teachers design was used to determine the effects of instructing the teachers. A pretest-posttest control-group design was used to determine the effects of the teachers’ instruction on student learning in their classes.

Results
The teachers’ implementation of the Question Exploration Routine was measured before and after the instruction. The results are shown in Figure 1. The baseline scores earned by the two groups were compared using a t-test for independent groups. Results indicated no significant differences between the groups’ implementation scores at baseline [ t (18) = 0.23, p = 0.818]. The posttest scores of the two groups were also compared. Again, results indicated no significant differences between the groups’ scores [ t (18) = .55, p = .59]. The two groups of teachers implemented the routine equally well after their respective workshops.

Figure 2 displays the mean percentage scores the two groups of teachers earned on a written test of their knowledge of the Question Exploration Routine. Repeated measures analysis of variance indicated that the posttest scores of VW teachers were significantly different from their pretest scores [F(1, 9)) = -520.26, p < 0.001], as were those of the AW teachers [F (1,8) = -484.21, p < 0.001]. An Analysis of Covariance (ANCOVA) revealed no significant differences between the posttest scores of AW and VW participants [F (1, 16) = 2.17, p = 0.16]. Thus, both groups of teachers gained about the same amounts of knowledge during their respective workshops.

Figure 3 displays the mean percentage scores earned by the two groups of teachers when they completed a Question Exploration Guide, the graphic device on which the Question Exploration Routine is based. Repeated measures analysis of variance indicated that the posttest scores of VW teachers were significantly different from their pretest scores [F(1,9)= 164.0, p < 0.001], as were the same scores of AW teachers [F (1,9) = 180.0, p < 0.001]. An Analysis of Covariance (ANCOVA) revealed no significant difference between the posttest scores of AW and VW participants [F (1, 17) = 0.03, p = 0.864]. Thus, both groups of teachers were equally skilled at planning a graphic device for use with the routine after their respective workshops.

An Information Acquisition Test was administered to students over the critical information taught before the teachers were instructed and over the critical information taught in the last observed lesson of the study. The test was comprised of open-ended items. Thus, students had to know the information and write it on the test, not merely recognize a correct answer in a list of multiple-choice items.

The test was administered at the end of each designated lesson. Results for all the students are shown in Figure 4. Results for the students with LD are shown in Figure 5. A Hierarchical Linear Model (HLM) analysis was conducted to test for differences in the posttest scores of the students in the VW classes compared to those in the AW classes. The HLM analysis was used to control for the dependency in the data due to many students having the same teacher (students nested in classes). In these analyses, the student pretest score was used as a covariate. The Kenward-Rogers method of computing degrees of freedom was used. The results indicated that the posttest scores of students whose teachers participated in the Virtual Workshop were significantly higher than the posttest scores of students whose teachers participated in the Actual Workshop, F(1, 14.7)= 5.99, p = .028.

HLM tests were also conducted within each treatment group to determine if the students made gains in their acquisition of the information being taught. Gains were made in each group. For the students in the VW group, the posttest scores were significantly different from the pretest scores, F(1,9) = 499.83, p < .0001; for the AW group, the pretest scores and posttest scores also differed significantly, F (1, 9) = 172.34, p < 0.0001.

HLM analyses were also used to compare the scores of the groups of students with disabilities. Again, posttest scores served as the dependent variable and pretest scores as the covariate. Although visible differences were apparent in the graphs (see Figure 5), the statistical analyses revealed no significant difference between the posttest scores of students with disabilities whose teachers participated in the Virtual Workshop and those who participated in the Actual Workshop, F (1,15.4) = 3.01, p = 0.10. Students with disabilities made significant gains from pretest to posttest in each group: for the AW group, F(1,8) = 28.59, p = .0007; for the VW group, F(1,7) = 69.33, p <.0001.

Conclusions
These results show that the Professional Development CD Program (used as the virtual workshop in this study) is as effective as live instruction with regard to instructing teachers in how to use the Question Exploration Routine. Both CD instruction and live instruction produced mastery levels of performance in the teachers and significant differences between their pre-instruction and post-instruction implementation and knowledge scores. Furthermore, the CD Program used by the teachers appears to be associated with significantly higher levels of learning in students than the live workshop. Although there were no significant differences between the two groups of students with LD, the students with LD who were taught by the VW teachers earned higher scores than the students with LD who were taught by the AW teachers. All students made significant gains from pretest to posttest, regardless of the type of training their teachers had.

Reference
Schumaker, J. B. (2005). Preparing teachers for academic diversity: Continuation report for SBIR Phase II Grant #R44HD36173 for the Question Exploration Routine CD. Washington, D.C.: National Institute of Child Health and Human Development, National Institute of Health.