Professional Development Program for the Concept Comparison Routine (Manual & Flash Drive)

Description

Research on the Professional Development Program for the Concept Comparison Routine (a CD Program)

Overview
The Concept Comparison Routine is used by teachers to help students compare and contrast two or more major concepts. To test the effects of a CD program for instructing teachers in how to use the Concept Comparison Routine, a study was conducted with 21 general education teachers who taught grades 4 through 12. They were randomly selected into an experimental and a control group. Eleven teachers worked through this CD program (hereafter referred to as the “virtual workshop” [VW] group) to learn how to use the Concept Comparison Routine. They used the routine in their general education classes with a total of 138 students, 46 of whom had learning disabilities (LD). Ten teachers (hereafter referred to as the “actual workshop” [AW] group) participated in a live workshop to receive instruction from one of the developers of the routine. These teachers used the routine with a total of 158 students, including 33 students with LD. The purpose of the study was to determine whether the virtual workshop (the CD program) was as effective as an actual live workshop. A multiple-baseline across-teachers design was used to determine the effects of instructing the teachers. A pretest-posttest control-group design was used to determine the effects of the teachers’ instruction on student learning.

Results
The teachers’ implementation of the Concept Comparison Routine in their classes was measured several times before and after the instruction. The results are shown in Figure 1. The baseline scores earned by the two groups were compared using a t-test for independent samples. Preliminary analysis indicated that the variances for the two groups were not equal at the time of the pretest. Consequently, the degrees of freedom for the t-test were adjusted. Results indicated no significant differences between the groups’ baseline implementation scores [ t(13.33) = 0.284, p = 0.787]. The posttest scores of the two groups were also compared using a t-test for independent samples. The posttest variances were not significantly different and so the degrees of freedom needed no adjustment. The results for this test indicated significant differences between the groups’ scores [t (19) = 2.80, p = 0.01]. The VW teacher group earned a significantly higher mean implementation score than the AW teacher group.

Figure 2 displays the mean percentage scores the two groups of teachers earned on a written test of their knowledge of the Concept Comparison Routine. T-tests indicated that the posttest scores of VW teachers were significantly different from their pretest scores [t(10) = -13.81, p < 0.001], as were those of the AW teachers [t(9) = -15.87, p < 0.001]. An Analysis of Covariance (ANCOVA) revealed no significant difference between the posttest scores of AW and VW participants [F(1, 18) = .067, p = 0.798].

Figure 3 displays the mean percentage scores earned by the two groups of teachers when they completed a Comparison Table, the graphic device on which the Concept Comparison Routine is based. T-tests indicated that the posttest scores of VW teachers were significantly different from their pretest scores [t(10) = -7.40, p < 0.001], as were the pretest and posttest scores of AW teachers [t(10) = -7.42, p < 0.001]. An Analysis of Covariance (ANCOVA) revealed no significant difference between the posttest scores of AW and VW participants [F (1, 18) = 0.113, p = 0.741].

A Concept Acquisition Test was administered to students about the compared concepts before the teachers were instructed and another similar test was administered to students about the compared concepts in the last observed lesson of the study. The test was comprised of open-ended items. Thus, students had to know the information and write it on the test, not merely recognize a correct answer in a list of multiple-choice items. The test was administered at the end of each designated lesson.

Results for all the students are shown in Figure 4. Results for the students with LD are shown in Figure 5. A Hierarchical Linear Model (HLM) analysis was conducted to test for differences in the posttest scores of the students in the VW classes compared to those in the AW classes. The HLM analysis was used to control for the dependency in the data due to many students having the same teacher (students nested in classes). In these analyses, the students’ pretest scores were used as a covariate. The Kenward-Rogers method of computing degrees of freedom was used.

The results indicated that the posttest scores of the whole group of students whose teachers participated in the Virtual Workshop were not significantly different from the posttest scores of the whole group of students whose teachers participated in the Actual Workshop, F(1, 18.7) = .05, p = .82. HLM tests were also conducted within each treatment group to determine if the students’ ability to learn concepts improved. Gains were made in each group. For the students in the VW group, the posttest scores were significantly different from the pretest scores, F(1,10) = 569.89, p < .0001; for the AW group, the pretest scores and posttest scores also differed significantly, F(1, 9) = 545.30, p < 0.0001.

HLM analyses were also used to compare the scores of the groups of students with disabilities. Again, posttest scores served as the dependent variable and pretest scores as the covariate. These analyses revealed no significant differences between the posttest scores of students with disabilities whose teachers participated in the Virtual Workshops and those with teachers who participated in the Actual Workshops, F (1,11.3) = 1.45, p = 0.25. Students with disabilities made significant gains from pretest to posttest in each group: for the AW group, F(1,8) = 49.27, p < .0001; for the VW group, F(1,7) = 221.40, p < .0001.

Conclusions
These results show that the Professional Development CD Program is as effective as live instruction with regard to instructing teachers in how to use the Concept Comparison Routine. Both CD instruction and live instruction produced mastery levels of performance in the teachers and significant differences between their pre-instruction and post-instruction implementation and knowledge scores. Furthermore, the CD Program appears to produce the same levels of learning in students when compared to the live workshop. Although there were no significant differences between the two groups of students, the students who were taught by the VW teachers earned higher scores than the students who were taught by the AW teachers. All students made significant gains from pretest to posttest, regardless of the type of training their teachers had.

References
Schumaker, J. B. (2004). Preparing teachers for academic diversity: Continuation report for SBIR Phase II Grant #R44HD36173 on the Concept Comparison CD. Washington, D.C.:
National Institute of Child Health and Human Development, National Institute of Health.