Final Practice Research Critique

Final Practice Research Critique

• 1.
  Running Head: PracticeResearch Article<br />ITEC 5090: Practice Research Article<br />Comparison of Peer Coaching Versus Traditional Coaching<br />Jessica Hollon and Samuel Hanna<br />University of Wyoming<br />Dr. Steven Aagard<br />Summary<br />Connie Bowman and Sandra McCormick did an experimental quantitative research project on the influence of peer coaching as opposed to traditional coaching. They used two groups of undergraduate elementary pre-service teachers. According to Bowman & McCormick, “timely feedback after teaching episodes is frequently inhibited under traditional university supervisions when large numbers of students are enrolled in a field experience program” (2000). In contrast, peer coaching, which is a process where teams of two pre service teachers observe each other and provide feedback, is an alternative that allows for more timely feedback. <br />In this study thirty-two participants were randomly chosen to be in either peer coaching pairs or in a traditional pre-service teacher education program. Thus, there were sixteen pre-service teachers randomly placed in the control group that would experience the traditional supervision method, and sixteen pre-service teachers randomly placed in the experimental group that would experience the peer coaching method. Then, after the training orientation, they went on a real field teaching experience for seven weeks in elementary schools in a large urban district, where each student taught two 15-minute language arts lessons per week, and also attended one ninety- minute weekly seminar. The weekly seminar for both groups was a chance to learn about clarity skills, pedagogical reasoning, and post lesson conferencing. The frequency, quality, and overall demonstration of these teachers’ clarity skills and pedagogical reasoning were what the pre service teachers would be evaluated on at the end of the study.<br />The pre-service teachers were divided into two groups, one group received peer coaching, whereas the other group experienced traditional university supervision. Peer coaching, as defined by Bowman and McCormick, is “a process in which teams of pre-service teachers regularly observe each other to provided suggestions, assistance, and support” (p. 256). By videotaping their lessons and making audio tapes of post lesson conferences either with a peer coach, for the experimental group, or a university supervisor, for the control group; the participants were evaluated. The evaluators of the participants’ videotaped lessons had an average of twenty-three years teaching experience and had worked with pre-service teachers in the past. <br />These evaluators also underwent four three-hour trainings where definitions of the targeted skills were discussed and sample videotapes were rated and compared. The same trainings took place with the evaluators of the post lesson audio tapes. These evaluators had an average of twenty years teaching experience and in their training they practiced evaluating simulated audio recordings on pedagogical reasoning. After each audio and video group training session, the evaluators independently evaluated three additional audio or video tapes to see the level of inter-rater agreement. In both types of training, the levels of inter-rater agreement ranged from 85-92 percent agreement in all categories being rated. <br />Following the trainings, videotaped lessons from week One (pre-assessment) and week Seven (post assessment) were evaluated for each participant by each of the three video tape evaluators. Audio tapes from weeks One, Four, and Seven for each participant were also evaluated by each of the three audio tape evaluators. One randomly selected videotape and audio tape was selected during the ratings and inter-rater reliability was again found to be maintained. <br />The results of this experimental study supports peer coaching as an alternative means to developing clarity skills, pedagogical reasoning and action, and attitudinal measures. The authors feel as though the advantages of peer coaching are apparent in this research, and that collaboration like this can foster expert instruction as well as collaboration.<br />Analysis<br />In order to critically examine the research article’s credibility, or “the extent to which the information is understandable, trustworthy, valid, and logical,” (McMillan & Wergin, 2006, p. 7) individual sections of the article and this experimental research need to be analyzed. First, the research hypothesis in Connie Bowman and Sandra McCormick’s experimental research study is implied, rather than explicitly stated. The hypothesis is that peer coaching will produce results different from traditional coaching. Their hypothesis supports Vygotsky’s social learning theory which lies in the notion that the “construction of meaning occurs first as exchanges between individuals” (Bowman & McCormick, 2000, p. 256). Thus, the social interaction that occurs with peer coaching would help the participants of this group to learn more.<br />Second, the general purpose of this study is clearly stated as comparing two types of pre service teacher supervision methods. This study’s significance is shown in the Bowman and McCormick (2000) statement: <br />Pre service teachers must learn to meld pedagogical theory to classroom functions, a particular issue is the need for close supervision and guidance when they participate in field-based training. Timely feedback after teaching episodes is frequently inhibited under traditional university supervision when large numbers of student are enrolled in a field experience program (p. 256).<br />Third, as for the method of sampling, it is plainly stated at the beginning of the research article that the study population was made up of “undergraduate elementary pre-service student teachers.” There were thirty-two total students, including three males and twenty-nine females. There were sixteen students randomly placed in the “experimental group,” who received peer coaching, and the other sixteen students were placed in the “control group,” who experienced traditional university supervision. There were two males in the experimental group and one male in the control group. The members of the experimental group were placed in pairs, whereas the members of the control group were placed individually. Moreover, Bowman and McCormick gave a very good and detailed description of the examined groups; their age, their education, their gender, and their previous background with field experience.<br />The thirty-six pre service teacher participants used in this study are described adequately. Their demographics in terms of age, class standing at the university, and enrolled classes are very similar. The method of selecting what group the participants were in was random. This means that “chance alone determines whether subjects are placed in the experimental or control group, thus eliminating selection bias” (Ary, Jacobs, Bazavieh, & Sorenson, 2006, p. 304). However, subjects could still be motivated to give biased responses on the evaluation of their experience. One major reason this may happen, is because the participants are evaluating a program that is also grading them and part of their overall college education. They are reporting these results to their own evaluators, or professors and those in positions of authority. By giving favorable responses, some participants many think this will help them in their pursuit of a degree, or put them in a better light with their professors.<br />Furthermore, the methods for data collection were very well defined. The skills examined were observed using videotapes of the pre-service teachers as they taught lessons. To assess the post-conference content, audiotapes of the post-conferences were evaluated using Shulman’s Model (1987). Procedures for collecting data were organized in detail. First, the three-hour orientation with an overview of the seven clarity skills and introduction to the type of language arts lesson required, procedural discussion, modeling, participant activities, and simulated post-conference scaffolding. Second, they had the pre-assessment data, which included a videotaped lesson and audiotape post-conference. Third, they had the intervention itself, which lasted from week two to six, along with the post-assessment data. <br />Moreover, both pre and post-tests were administered measuring observable skills from the audiotapes and videotapes using a Likert-type scale as the rating instrument. The Likert-type scales measured frequency of occurrence, quality of use, and overall demonstration of the clarity skills, attitude, pedagogical reasoning and actions. The raters, all teachers with a mean of twenty-three years of teaching experience, were trained to measure these observable skills. All of the sixty-four videotaped sessions were scored independently “Using Cronbach’s alpha, internal consistency yielded .89 for frequency of occurrence and .84 for quality of use, and 92% for overall demonstration” (p. 259). Thus, the element of reliability is fulfilled in this study. Reliability in research is defined as “the extent to which the measure would yield consistent results each time it is used” (Ary et al., 2006, p. 230). The level of inter-rater agreement when evaluating the video and audio tapes speaks to the high reliability of these evaluation methods and the raters’ ability to consistently evaluate the participants. <br />As for the validity element in this study, we believe it was fulfilled through the random assignments of the tasks and the students. Random assignment placed the thirty-two participants in to the control group or the experiment group. Each group had sixteen participants, and the groups were then randomly assigned to schools in the area. The participant/peers and the teachers were unaware of being in either the control or the treatment group. The validity of evaluating seven teaching clarity skills as suggested by Metcalf (1989) and Schulman’s (1987) pedagogical reasoning and actions is clear (as cited in Bowman & McCormick, 2000, p. 256). Pre service teachers need to combine both of these skill areas to succeed on the classroom. The Likert scale used to evaluate participants experiences, is a common scale used to rate respondents’ feelings and attitudes towards certain situations. While the participants may have been swayed to respond favorably to the questions because they were evaluating a program for authority figures, we think it unlikely. However, we think the random assignment of the pre-service elementary teachers to cooperating teachers within schools was not a very good idea. This is because we think that it could have been better if they had assigned them teachers with close backgrounds and majors.<br />The literature review was good and sufficient, although all of the sources in the article are from the 80’s or early 90’s, which are at least 15 years old. While a similar study by Joyce and Showers in 1983 is referred to, this past study and its findings are not summarized in detail. Moreover, although Bowman and McCormick made good use of existing literature and previous research, most of the reviewed literature is tied to the benefits and advantages of peer coaching. In other words, we were wondering if all the existing literature related to peer coaching showed it as more beneficial than traditional coaching. Also, the claim that traditional university supervision methods are untimely is not tied to any past research or data. <br />The overall design of the research is strong, contributing to its credibility. The procedures for collecting data and rating the participants are clear. In our opinion there is some researcher bias with the lack of support for the claim that traditional pre service teacher supervision is untimely and inadequate. We would have liked to see data from past research illustrating this to be summarized along with this claim. <br />The results of this study are presented with mean and standard deviation scores in the same breakdown as the original research question. The mean and standard deviation for each clarity skill variable and each pedagogical reasoning action variable were calculated for both groups in the pre and post assessments. Pretest mean score differences for the control and experimental group were not statistically significant for any of the variables, and the post test results favored the experimental, peer coaching group for all clarity skill variables. The results are shown in appropriate tables that break down the frequency, quality, and overall demonstration of the teaching clarity skills (stating objectives, repeating points, using examples, repeating items, asking questions, student questions, and practice time). Pedagogical reasoning and action results are shown broken down into comprehension, transformation, instruction, evaluation, reflection, and new comprehension. <br />In addition to the data on clarity and pedagogy, the participants’ views and feelings of either the peer coaching or the traditional pre-service teacher program were clearly depicted through the use of a Likert scale. This type of scale is “one of the most widely used techniques to measure attitudes” (Ary, et al., 2006, p. 227). In this study the Likert attitude scale results are shown broken down into collegiality, technical, feedback, analysis of application, adaptation to student, and personal facilitation. Depicting the results in this manner clearly conveys the findings of this research study.<br />However, there are some weaknesses about this study. First, a major weakness of this study is the small sample size, which consequently leads to questioning the generalization of the results of this study. Second, as admitted by Bowman and McCormick, that a weakness of this study is the time required for this type of pre-service experience in other universities. Though previous studies used six or more months to implement peer coaching, the authors felt teachers could acquire the skills for effective peer coaching in a shorter time. Third, the authors did not provide any sample examples for the audio-taped analysis done by the control group, like the sample they provided for the experimental group. <br />Fourth, there are some reasons that could have lead to the fact that the peer coaching had more positive outcomes than the traditional coaching other than that the former is better than the latter. Firstly, the university supervisor did not attend all the teaching sessions performed by the pre-service teachers. Thus, for three sessions, the post-conferences were held without direct observation from the university supervisor, but it relied only on the pre-service teachers’ description of their instruction. Secondly, sometimes the post-conference sessions were occasionally delayed, that is to say, they did not occur immediately after the teaching session. <br /> <br />Conclusion<br />After critically examining the research article, we found many strengths in the research. The purpose of the research is significant to the field of teacher education because it is contributing to the betterment of this process. The descriptions in the article of control and experimental groups, their demographics, and the data collection process are clear and easily understandable. All of these factors along with the fact that the skills the raters were looking for were highly identifiable by observing and the fact that selection bias was eliminated through random group assignment, all contribute to a high level of credibility. The level of inter-rater agreement leads us to believe the research had a high level of reliability and that this was valid through the random assignments of groups.<br />While the research results were depicted in clear and organized tables, the literature review in this research article seems to be quite dated. Along with the small sample size and the short time frame, this brought up questions about the research. We feel that the research could have been stronger if not only the time frame were longer, but also if the sample size were larger. <br />Instead of randomly pairing pre service teachers with cooperating teachers, possibly the researchers could have paired them with cooperating teachers of the same backgrounds and majors. Another weakness that overshadowed some of the research results were that the pre service teachers could have been driven to give bias responses to the Likert scale survey based on the fact that they knew they were reporting to authority figures. <br />It could have also made the research and the results stronger had there never been any delayed post conferences for either the experimental or the control groups. It was hypothesized that the peer coaching model, combined with immediate feedback in this type of peer interaction would positively affect the experience. If the researchers truly wanted to pinpoint and test this hypothesis, both groups should have had immediate post conferences. <br />Overall, the research seems to have brought up many points to ponder about the traditional method of pre service teacher education. The fact that social learning and peer coaching can positively contribute to this type of education is interesting in and of itself. There are certainly elements of peer coaching that seem to lend to bettering the pre service teacher education program. <br />References<br />Ary, D., Jacobs, L. C., Razavieh, A., & Sorenson, C., 2006, Introduction to research in <br />education (7th ed.). Belmont, CA: Thomson Wadsworth.<br />Bowman, C.L., & McCormick, S. (2000). Comparison of peer coaching versus traditional supervision effects. The journal of educational research 93(4), 256-261.<br />McMillan, J. H., & Wergin, J. F., 2006, Understanding and evaluating educational <br />research (3rd ed.). Upper Saddle River NJ: Pearson Education, Inc.<br />