Proof
Proof
1
ARTICLE IN PRESS Version 7.51e
3
Computers & Education & (&&&&) &–&
4
[Link]/locate/compedu
F
5
OO
7
10
perceived student learning in technology-using classrooms?
PR
11
12
Amy L. Baylora,*, Donn Ritchieb
a
13 Department of Educational Psychology and Learning Systems, Instructional Systems Program,
14 Florida State University, Stone Building No. 307, Tallahassee, FL 32306, USA
b
Department of Educational Technology, San Diego State University, USA
15
16
17
ED
Received 15 April 2001; accepted 4 June 2002
18
19
CT
20
21
Abstract
22
23 Based on a comprehensive study of 94 classrooms from four states in different geographic regions of the
RE
24 country, this quantitative study investigated the impact of seven factors related to school technology
25
(planning, leadership, curriculum alignment, professional development, technology use, teacher openness
26
to change, and teacher non-school computer use) on five dependent measures in the areas of teacher skill
(technology competency and technology integration), teacher morale, and perceived student learning
27
(impact on student content acquisition and higher order thinking skills acquisition). Stepwise regression
R
28
resulted in models to explain each of the five dependent measures. Teacher technology competency was
29
predicted by teacher openness to change. Technology integration was predicted by teacher openness to
CO
30
change and the percentage of technology use with others. Teacher morale was predicted by professional
31 development and constructivist use of technology. Technology impact on content acquisition was predicted
32 by the strength of leadership, teacher openness to change, and negatively influenced by teacher non-school
33 computer use. Technology impact on higher-order thinking skills was predicted by teacher openness to
change, the constructivist use of technology, and negatively influenced by percentage of technology use
UN
34
35 where students work alone. Implications for the adoption and use of school technologies are discussed.
36 # 2002 Elsevier Science Ltd. All rights reserved.
37 Keywords: Technology integration; Teacher education; Improving classroom teaching; Teacher morale; Technology
38 literacy
39
40
41
42
0360-1315/02/$ - see front matter # 2002 Elsevier Science Ltd. All rights reserved.
PII: S0360-1315(02)00075-1
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 1. Introduction
2
3 Identifying the value of technology in schools has challenged educational researchers for more
4 than 20 years. Part of the problem is our evolving understanding of how technology accentuates
F
5 student learning. Rapid changes in the technology itself also hamper research. Finally, the inter-
6 twining of complex variables in such a rich environment as a school precludes the pure isolation
OO
7 necessary to determine cause and effect.
8 Over the past decade, many articles have appeared in popular and educational journals pro-
9 viding anecdotal evidence of changes that educational technology can make in schools. Even
10 though other empirical articles have provided quantitative and qualitative evidence of these
PR
11 changes, most schools rarely base their technology decisions on specific published research find-
12 ings. Instead, school leaders often start by thinking about the intended results that technology
13 should provide within their school environment. Next, these leaders take certain actions regarding
14 the attainment, allocation, use, and support of technology. Consequently, this study was framed
15 to consider the question ‘‘What actions can school personnel take that most effectively lead to
16
17
ED
their desired results regarding the integration of technology in schools?’’
We considered seven factors (planning, leadership, curriculum alignment, professional devel-
18 opment, technology use, teacher openness to change, and teacher non-school computer use) and
19 five outcomes in the areas of teacher skill (level of teacher technology competency and technology
CT
20 integration), teacher morale, and perceived student learning (impact on content acquisition and
21 impact on higher order thinking skills). While there undoubtedly are a large number of possible
22 variables that may affect the complexity of technology integration within the schools, we limited
23 our factors to those that were most supported in the literature while including both teacher-related
RE
24 and school-related factors. Data was collected through structured interviews with teachers and
25 administrators, teacher surveys, and examination of school technology use plans. In the next
26 sections, we discuss the previous research and literature that led us to define the seven factors and
27 five outcomes used in our study. We begin with the seven independent variables.
R
28
30
31 Schools that are successful in integrating technology into the curriculum are often guided by a
32 comprehensive technology use plan (TUP). These plans do more than provide a blueprint to the
33 sequence of events the school hopes to achieve. The plans also describe the overall philosophy of
technology use and explore how technology will improve teaching and learning.
UN
34
35 In this study, technology planning was operationalized from the teacher’s perspective to include
36 the teacher’s role in creating the technology use plan, familiarity with the published vision, and
37 the belief that the plan considers his/her needs. From the administrator’s perspective, it was
38 operationalized to include three components: strategic, teaching/learning and operational. In
39 terms of the strategic component, this included the extent to which the plan stated a vision and
40 involved stakeholders, which may be the most important action regarding technology planning
41 (Anderson, 1996). The teaching and learning component of the TUP covered instructional inno-
42 vation. The operational component of the TUP included technology maintenance and support,
43 the presence of an action plan and timeline, and facility infrastructure, configuration, and funding
44 issues. Also included were the extent to which technology decision are based on the official TUP,
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 the extent to which records are kept regarding the type and number of technology activities, and
2 the extent to which purchase and use of technology has closely followed the details as described in
3 the TUP.
4
F
5 1.2. Technology leadership
6
OO
7 Although grass-roots movements within schools can be successful, more common in technology-
8 enhanced learning environments are leaders who have the leadership ability and vision to direct
9 changes. Modeling technology use, planning and articulating a vision, rewarding teachers as they
10 strive to incorporate technology, and sharing leadership are common characteristics of successful
PR
11 technology leaders. The leadership underlying a school is of critical importance because the
12 school culture begins to reflect new ways of teaching and learning (Maurer & Davidson, 1998).
13 An important component of this change is a school leader who is dedicated to fostering a new
14 culture with shared leadership and technology use.
15 In this study, technology leadership was operationalized from the teacher’s perspective to
16
17
ED
include the presence of positive technology-using role models, such as the principal, and the pre-
sence of incentives for teacher use of technology. From the administrator’s perspective, it was
18 operationalized to include the principal’s ability and work with the school community to for-
19 mulate, articulate, and communicate a school’s vision (Dede, 1994; Raizen, Sellwood, Todd, &
CT
20 Vickers, 1995; Rhodes, 1994; Sergiovanni, 1995). The principal’s use of technology is also inclu-
21 ded as part of the leadership component since principals foster credibility and respect by engaging
22 in technology activities such as communicating to the staff via email, demonstrating the use of
23 desktop presentation to the faculty, showing a student how to keep a writer’s journal with a word
RE
28 schools’ technology knowledge and leadership is shared by a variety of faculty is also important
29 given that a successful technology leader shares leadership by empowering other school members
CO
30 (Maurer & Davidson, 1998). Also included in the operational definition were the vision of the
31 technology use plan to promote technology for teaching and learning, and the presence of an
32 action plan and timeline within the technology use plan.
33
34
35
36 Given that a school’s curriculum provides instruction that results in student progress towards
37 the stated learning objectives, it would follow that technology activities should be aligned to that
38 curriculum. A variety of researchers conducting meta-analyses (e.g., Kulik, 1994; Liao, 1992;
39 Ryan, 1991) have found that technology can improve scores on national and state tests. But when
40 the technology is aligned to support specific curriculum goals, the increase in student capabilities
41 may not appear in national standardized test results.
42 In this study, curriculum alignment was operationalized from the teacher’s perspective to
43 include teacher perception as to whether technology activities are covered through the curriculum
44 documents.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
3 Regardless of the amount of technology or its sophistication, technology will not be used unless
4 faculty members have the skills, knowledge, and attitudes necessary to infuse it into the curricu-
F
5 lum. Generally this comes through self-education or professional development. Schools can assist
6 by providing in-service training that meets the needs of the faculty, and by promoting continual
OO
7 growth both within and outside the school boundaries.
8 Professional development was operationalized from the teacher’s perspective to include the
9 applicability of the professional development programs, incentive provided to attend programs,
10 access to technical support, and appropriateness of technology equipment. From the adminis-
PR
11 trator’s perspective, it was operationalized to include the extent to which the school supports
12 faculty to attend workshops or conferences, the listing of professional development activities in
13 the TUP, and the support of school activities to learn to use technology. The latter is important
14 given that teachers require prolonged exposure to new ideas and skills before classroom behaviors
15 change. It has been found that for teachers to feel in command of educational technologies and to
16
17
ED
know when and how to use them, it can take as long as five to six years (Brunner, 1992; Elmer-
Dewitt, 1991). In addition, Rubin (1989) found that the extent to which teachers assist in deter-
18 mining in-service technology training topics relates to how well they embrace the concepts deliv-
19 ered during the workshop. Given that there are varying abilities and knowledge of faculty in a
CT
20 school, seldom will there be agreement on the need for any one in-service topic; consequently, the
21 best strategy would be to identify multiple topics and then involve only people who have needs in
22 the specified content domain (Picciano, 1998). The presence of incentives for incorporating tech-
23 nology was also considered as part of professional development.
RE
24
27 Once schools obtain technology, questions arise as to how best to use it. Some schools opt to
R
28 place computers in labs, whereas others use group techniques in the classroom. Some teachers
29 focus on learning about computers while others focus on learning with computers. Even when
CO
30 teachers focus on learning with computers, some limit the use of technology to helping students
31 learn basic skills whereas others stress higher order thinking.
32 There is a basic dichotomy in which the computer is used as the subject matter for study or as
33 an instructional tool to teach other content. About half the time students spend on computers
involves learning ‘‘computer-specific skills’’ such as keyboarding, and spreadsheets (Becker, 1991;
UN
34
35 President’s Panel on Educational Technology, 1997). For this study, technology use was deli-
36 neated according to nine subcomponents, each of which were considered separately in the
37 regression models: (1) how often technology was used for preparing for or during classroom
38 instruction; (2) the percentage of time that subject-matter content was the focus of the technology
39 use; (3) the percentage of time that higher order thinking skills (HOTS) were the focus; (4) the
40 percentage of time technology literacy was the focus; (5) the percentage of time technology was
41 used alone by students, responding to questions, (6) the percentage of time technology was used
42 alone by students, creating; (7) the percentage of time technology was used with others; (8) the
43 percentage of constructivist use of technology; and, (9) the perceived success of technology use by
44 teachers.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
3 Teacher openness to change influences teachers’ willingness to integrate technology into the
4 classroom. Although it is generally viewed as an internal prerequisite to success, it is closely tied
F
5 to external factors such as professional development and a supportive climate. Although an atti-
6 tude of openness to change facilitates a teacher’s acceptance of technology, critical to this accep-
OO
7 tance is the need to see relevance in the process.
8 In terms of the propensity for innovation, Marcinkiewicz (1994) found that self-competence
9 and innovativeness were most closely related to the level of computer use, concluding that more
10 research is necessary to further quantify the role of teacher motivational factors. Pedagogical
PR
11 approach is also related to teacher innovation. Niederhauser and Stoddard (1994) conducted a
12 quantitative investigation of the relationships between constructivist teaching style, the use of
13 constructivist-oriented software, and teacher beliefs about technology. Results split along two
14 lines: teachers who saw computers as a tool to be used in collecting, analyzing and presenting
15 information (correlating with the constructivist approach) and those who saw them as teaching
16
18 Based on this research, in this study teacher openness to change was operationalized from the
19 teacher’s perspective to include predisposition for trying new instructional innovations, and the
CT
20 belief that they can take risks in teaching. From the administrator’s perspective, teacher openness
21 to change was operationalized to include whether the technology use plan promotes instructional
22 innovation with technology implementation.
23
RE
26 The extent to which teachers use technology outside of the classroom may be an indicator of
27 their interest and corresponding skill in using technology. Evan-Adris (1995) identifies three pat-
R
28 terns of technology use among teachers. The first is ‘‘avoidance’’, including teachers who assign
29 computer time to the students but do not use the technology for their own purposes. The second
CO
30 pattern she labels ‘‘integration’’ and these teachers spend time experimenting with and learning to
31 use hardware and software and structure learning time to promote effective and increased use of
32 technology by their students. The third pattern is ‘‘technical specialization’’ and includes teachers who
33 have strong computing skills and their use of the computer is more organized and purposeful than
average teachers. These classifications, which indicate the effectiveness of teacher technology use, are
UN
34
35 indirectly supported by the amount of non-school computer-use in which the teacher is engaging.
36 Teacher non-school computer use was operationalized in this study from the teacher’s per-
37 spective to include the number of times technology (e.g. word processing, database, spreadsheet,
38 graphics, multimedia, telecommunications) was used at home for non-school activity.
39
40
41 2. Outcome measures
42
43 In the previous sections, we examined the factors that may determine the success of technology
44 in a classroom. But what are the areas that schools want to impact? The dependent measures are
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 explained in the next sections. Note that several of the factors described previously could be
2 considered interchangeably with the desired outcomes (e.g. it could be argued that high teacher
3 morale should be an independent variable that affects the degree of success in using technology).
4 However, the factors were purposely selected to serve as predictors, and the outcome measures as
F
5 dependent variables.
6
OO
7 2.1. Impact on content acquisition
8
9 Teachers assess students frequently during the course of a year. Often this assessment is based
10 on the amount of knowledge that a student can demonstrate. This focus on content originates
PR
11 from two areas—most teachers were educated by teachers who focused on content acquisition
12 (and teachers tend to teach the way they learned), and schools are increasingly being held
13 accountable for students’ performance on factual recall. In response to this perceived need, the
14 majority of technology-based instruction has focused on the acquisition of factual information
15 rather than on higher-order thinking and problem-solving (Grabe & Grabe, 1998). However, this
16
17
ED
focus on factual knowledge is not spread evenly across all demographic groups (George, Mal-
colm, & Jeffers, 1993). Teachers of less able and lower-SES students are more likely to use tech-
18 nology for drill and practice of isolated skills such as math facts, phonics, and grammar rules, or
19 tutorials. In this way technology serves as an effective way to provide remediation when it is
CT
20 assumed that basic skills and knowledge are missing.
21 Impact on content acquisition was operationalized from the teacher’s perspective to include the
22 relative impact/importance of technology in terms of the content acquisition, in other words, the
23 extent to which the use of technology added to the class performance in content acquisition.
RE
24 From the administrator’s perspective, it was operationalized to include the role of the Technology
25 Use Plan’s vision in promoting technology for teaching and learning, and the reflection of the
26 plan in describing the use of technology by students to enhance learning based on current
27 knowledge of cognition.
R
28
30
31 Computer technology may also serve as a cognitive tool by supporting, guiding, and extending
32 the thinking processes of students (e.g. Lajoie & Derry, 1993). When used as a tool to help stu-
33 dents analyze, compare, contrast, or evaluate resources, the computer facilitates the student’s
internal cognitive processes by serving as an extension to their intellectual capacity. This heigh-
UN
34
35 tened capacity helps students think more critically as they manipulate information. Traditional
36 teaching tends to focus on imparting skills and knowledge that have been found to play an
37 important role in creating an educated society. Today, however, many educators are stating the
38 need to go beyond the basics to prepare our students for a life that will be drastically different
39 from that which the educator experienced. These educators stress the need for students to become
40 creative problem-solvers, able to analyze a wealth of information to draw valid conclusions.
41 These higher-order thinking skills were sometimes alluded to in traditional teaching, but have
42 been more commonly associated with teaching styles developed in the past two decades. Salomon
43 (1986) suggests that computers may be the ideal mechanism for teaching higher order thinking
44 skills, and other researchers are beginning to validate that claim.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 Impact on HOTS was operationalized from the teacher’s perspective to include the relative
2 impact/importance of technology in terms of higher order thinking (i.e. thought processes); spe-
3 cifically, the extent to which the use of technology added to the class performance in higher order
4 thinking. From the administrator’s perspective, it was operationalized to include the role of the
F
5 technology use plan’s vision in promoting technology for teaching and learning, and the reflection
6 of the technology use plan on current knowledge of cognition in describing the use of technology
OO
7 by students.
8
PR
11 The way in which technology is used in a classroom is a critical measure of its success. As stated
12 by the Office of Technology Assessment (1995, p. 57), ‘‘. . .it is becoming increasingly clear that
13 technology, in and of itself, does not directly change teaching or learning. Rather, the critical
14 element is how technology is incorporated into instruction.’’ When students and teachers perceive
15 computers as a separate subject, unassociated with the context of the lesson or classroom, the
16
17
ED
content or concepts studied are often left fragmented in the learner’s mind. But if a technology-
enhanced lesson is integrated into the larger curriculum with direct tie-ins, students are more likely
18 to infuse the knowledge into existing cognitive structures. Technology integration requires teachers
19 to alter their teaching processes, no longer being the sole distributor of information. This change in
CT
20 role requires support from many sources in order for the teacher to make the transition.
21 Given that the integration of technology in education should have the goal of changing the
22 nature of instruction rather than just using technology to perpetuate traditional teaching and
23 learning methods (Hawkins & Collins, 1992), technology integration was operationalized from
RE
24 the teacher’s perspective to include the extent to which the use of technology fits into the overall
25 unit of instruction, whether there are transitions before and after the activity with the rest of
26 instruction, and the extent to which technology use is not a separate activity from other instruc-
27 tional activities.
R
28
30
31 Because technology opens new avenues for instruction, and because its use is often linked to
32 professionalism, some schools have intended for technology implementation to improve teacher
33 morale. Hadley and Sheingold (1993) conducted a survey of 608 teachers in 576 schools
throughout the country that were known for their efforts to integrating computer technology into
UN
34
35 their teaching. They found that when teachers were asked to identify incentives for integrating
36 computers in their teaching, two trends emerged: student accomplishment, rather than their own
37 external rewards, was most motivating for the teachers, followed by students’ being able to use
38 computers as a tool for their own purposes. As they state, ‘‘in the daily professional life of these
39 teachers, it is the psychic payoff of student’s learning and engagement that appears to matter
40 most’’ (p. 281). Teachers also cited increased self-esteem, through recognition, advancement,
41 development, and financial reward, as a motivating factor. When asked to identify barriers, three
42 factors were considered as barriers in the past and persisted as barriers: too few computers and
43 peripheral equipment, not enough time to prepare computer-based lessons, and challenges with
44 scheduling enough computer time for different teachers’ classes.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 In this study, teacher morale was operationalized from the teacher’s perspective to include
2 enjoyment of using technology, perception of colleagues’ morale regarding technology use,
3 opportunities for collegial sharing of technology ideas and uses, satisfaction with work environ-
4 ment, and extent to which the position provides professional growth and is satisfying. From the
F
5 administrator’s perspective, it was operationalized to include the extent to which faculty are rewar-
6 ded for intent to use technology, the promotion of innovation/creativity within the technology use
OO
7 plan, specific plans for teacher technology maintenance and support, and incentives to participate in
8 professional growth and for incorporating technology as stated in the technology use plan.
9
PR
11
12 Because technology in education is a relatively new phenomenon, most veteran teachers were
13 not technologically proficient when they entered the profession. Yet to effectively broaden the
14 range of instructional opportunities that can be offered to students, teachers must reach and
15 maintain a certain degree of technological competence. Additionally, technology competency
16
17
ED
allows them to become more efficient in dealing with everyday tasks such as communicating with
parents, keeping records, doing research in their subject domain, and preparing presentations. A
18 1997 report by the Present’s Committee of Advisors on Science and Technology (PCAST) found
19 that teachers currently receive little technical, pedagogical, or administrative support and that few
CT
20 colleges of education adequately prepare their graduates to use educational technologies in their
21 classrooms. One of the biggest obstacles to teacher technology proficiency has been the lack of
22 technology training in teacher education program. Teachers commonly report that they have not
23 received adequate preparation to effectively use computers in the classroom (Office of Technology
RE
24 Assessment, 1995).
25 Teacher technology competency was operationalized to include teacher-perceived confidence in
26 the following areas: using a variety of software programs (e.g. word processing, database/
27 spreadsheet, email, internet), file management, solving general software or hardware problems,
R
28 use of terms associated with computers, identifying and explaining basic computer components,
29 operating technology equipment, selecting and implementing appropriate technology to support
CO
34
35 follows: Which combination of factors best predicts each of the five desired outcomes?
36
37
38 3. Methods
39
40 3.1. Sampling
41
42 Through purposive sampling we selected schools that were known to be effective users of tech-
43 nology. A set of highly technology-integrative schools was nominated from each of four regions
44 of the country by expert researchers. Specifically, we selected public schools that met the following
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 requirements: (1) the school made a significant effort to implement technology for at least 2 years
2 across many, if not all, of its classes; (2) the key administrator had been in place since the pre-
3 vious year and intended to be there the following year; (3) selected members of the teaching staff
4 were willing to cooperate with data collection activities during the spring and fall; and, (4) the
F
5 school had implemented a technology use plan.
6 From this process, 12 schools were selected: five from California, two from Florida, three from
OO
7 Virginia, and two from the state of Washington. In return for their assistance, the schools were
8 provided with a grant for the purchase of instructional materials and a report of the data col-
9 lected from the school.
10 Of the 12 schools, five were urban, four were suburban, and three were in rural settings. The
PR
11 percentage range of white to non-white students ranged from 5 to 95% with the mean percentage
12 of non-white students as 32%. Five of the schools were elementary, five were middle schools, and
13 two were high schools. Once the schools were selected, the principal of the school provided the
14 researchers with list of ‘‘classrooms’’ meeting the three following requirements: (1) the teacher is
15 the primary provider of instruction (not part of a teaching team); (2) the teacher is using tech-
16
17
ED
nology in teaching; and, (3) the teacher has been teaching the class since at least the prior school
year and is intending to teach at the school through the next year.
18 From this list of classrooms, we selected at random 10–12 classrooms for each school. Follow-
19 ing this, the principal asked each of the randomly selected teachers whether they would be willing
CT
20 to participate. If a teacher declined to participate, a replacement from the pool was randomly
21 selected, until there were at least 10 participating teachers per school.
22 The resulting sample included a total of 94 teachers and correspondingly 94 classrooms from
23 the 12 schools, with the classroom as the unit of study.
RE
24
25 3.2. Instrumentation
26
27 Given that there were no existing instruments that matched the sources of the data with the
R
28 independent and dependent variables, we developed four new instruments to collect school,
29 classroom, teacher, and administrator information. The four instruments included the following:
CO
30 administrator structured interview, teacher structured interview, technology use plan evaluation,
31 and teacher survey. Each instrument was created to consist primarily of Likert items on a five-
32 point scale. As appropriate, each instrument was evaluated by a panel of experts for content
33 validity, reliability, and usability, as will be described below. Triangulation was an important
component to gather data from multiple sources to gain multiple perspectives. See Table 1 that
UN
34
1 Table 1
2 Instrument coverage, by variable
3
F
5
OO
7
10
PR
11
12
13
14
15
16
17
ED
18
19
CT
20
21
22
23
RE
24
25
26
27
R
28
29
CO
30
31
32
33
34
35 A structured teacher interview was conducted with each of the 94 teachers to identify the tea-
36 cher’s perception of technology in the classroom regarding the following variables, as shown in
37 Table 1: curriculum alignment, technology use, impact on content acquisition, impact on HOTS,
38 and technology integration. Part of the interview required the teacher to list all activities in the
39 prior school year that involved technology. From this list, three activities were randomly selected
40 to focus upon and were scored according to Likert- scaled items. All researchers in the four geo-
41 graphic areas were trained via video together with detailed structured interview questions, all on a
42 Likert 1–5 scale. For example, the interviewer’s guidelines for assessing the level of technology
43 integration for an activity is as follows: ‘‘To determine the level of integration of the activity, ask
44 questions such as the following: In a typical day in which this activity was conducted, what would
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 normally be taught right before and right after this activity? How did the activity fit into the unit
2 of instruction? What are transitions like between the different components of the activity with the
3 rest of instruction? Is the activity separate from other instructional activities?’’ Following the
4 interview questions, the interviewer would then rate the activity according to the following scale:
F
5 ‘‘The technology-related aspects of this activity are integrated into classroom instruction.
6 (1—strongly disagree; 2—disagree; 3btrneutral; 4—agree; 5—strongly agree).’’
OO
7
PR
11 the school’s faculty. Analysis of the plan contributed to the following variables, as shown in
12 Table 1: planning, leadership, professional development, teacher openness to change, technology
13 use, integration, content acquisition, HOTS acquisition, and teacher morale. The TUPs were
14 scored by a trained team including one researcher in each of the four geographic areas. Inter-
15 rater reliability was conducted by evaluating sample TUPs until the researchers scores reached a
16
17
ED
reliability coefficient of 0.9. At that point the researchers each independently evaluated the
technology use plans in their geographic areas. A sample rating question regarding instructional
18 innovation is as follows: ‘‘The plan promotes instructional innovation and/or creativity with
19 technology implementation. (1—strongly disagree; 2—disagree; 3—neutral; 4—agree; 5—
CT
20 strongly agree).’’
21
28
29 3.3. Procedure
CO
30
31 Data was collected according to the following schedule: Administrator interview (Spring,
32 Summer), Technology use plan analysis (Spring, Summer), Teacher survey (Fall), and Teacher
33 interviews (Fall). The instruments were implemented by researchers in each of the four geo-
graphic regions. Additionally, there were several classroom observations at each school that were
UN
34
35 conducted throughout the Fall and following Spring that served to cross-check the data collected
36 through the other instruments.
37
38
39 4. Results
40
41 Using the four instruments described above (with a combined total of 148 items), data was
42 collected from 94 classrooms, comprising a potential dataset of 13 912 data points, of which 1988
43 data points (14%) were incomplete or missing. Specifically, there were 735 missing data points on
44 administrator interview; zero missing on technology use plan; four missing on teacher interview;
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 and, 1249 missing on the teacher survey questionnaire. These data points were left as blank for
2 the data analysis.
3 A total of 15 independent variables (technology use was broken down into nine sub-
4 components) were considered as predictors for regression models for each of the five dependent
F
5 variables. The independent variables were as follows: technology planning, technology leadership,
6 professional development, teacher openness to change, teacher non-school computer use, curri-
OO
7 culum alignment, technology use (broken into nine subcomponents: amount technology used,
8 percentage of use focused on content acquisition, percentage of use focused on HOTS, percentage
9 of use focused on technological literacy, percentage of use focused on activities where students
10 worked alone responding to the technology, percentage of use focused on activities were students
PR
11 worked alone creating with technology, percentage of use focused on activities working colla-
12 boratively, constructivist use of technology, and success of technology activities). The dependent
13 variables included the following: technology integration, level of teacher technology competency,
14 level of teacher morale, impact on content, and impact on HOTS.
15 Given that nearly all of the items were on a 1–5 Likert scale, a process of aggregation was used
16
17
ED
to average the scores and provide one score ranging from 1 to 5 for each classroom, where 1=low
and 5=high for a given variable. Aggregation was performed by averaging all related item values
18 so that each variable was reduced to one score per variable per class. For example, Mr. Lang’s
19 classroom could be rated 1.24 on technology planning, 2.45 on leadership, 4.56 on curriculum
CT
20 alignment, etc. A strength of this method was that multiple sources of information (via the four
21 instruments) contributed to the overall variable scores. Variables that were not on a 1–5 Likert
22 scale (e.g. the technology use subcomponents) were not aggregated, but entered into the model
23 separately as-is. Descriptive statistics are listed in Table 2.
RE
24 The primary means for analyzing the data was through stepwise regression analyses to identify
25 what combination (if any) of the independent variable(s) predicted the results of the dependent
26 variables. A statistically significant model of independent variable(s) predicted each of the five
27 dependent variables via a forward stepwise regression.
R
28 Technology impact on student content acquisition was predicted by (1) the strength of technol-
29 ogy leadership at a school; (2) openness to change; and (3) (negatively influenced by) teacher non-
CO
30 school computer use. (R2=0.589). All independent variables were entered in a forward stepwise
31 regression model. See Table 3.
32 Technology impact on higher-order thinking skills was predicted by (1) the degree of teacher
33 openness to change; (2) (negatively influenced by) the amount of technology use by students
working individually in situations where they were ‘‘creating’’; and (3) the level of constructivist
UN
34
35 modes of technology use (R2=0.608). All independent variables were entered in a forward step-
36 wise regression model. See Table 4.
37 Teacher morale was predicted by professional development and constructivist use of technol-
38 ogy. (R2=0.559). All independent variables were entered in a forward stepwise regression model.
39 See Table 5.
40 Teacher technology competency was predicted by teacher openness to change. (R2=0.164). All
41 independent variables were entered in a forward stepwise regression model. See Table 6.
42 Technology integration was predicted by teacher openness to change and technology use with
43 others. (R2=0.391). All independent variables were entered in a forward stepwise regression
44 model. See Table 7.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 Table 2
2 Descriptive statistics
3 N Minimum Maximum Mean S.D.
4
F
Factors
5
Technology planning 94 1.79 4.18 2.9531 0.6188
6
OO
Technology leadership 94 1.25 4.63 3.4467 0.5777
7 Curriculum alignment 89 1.16 5.00 4.2018 1.0026
8 Professional development 94 2.08 4.31 3.0261 0.4987
9 Professional development: number of_times 59 0.00 870.00 78.6102 132.3846
10
Technology use: number of times 60 0.00 56.00 14.4308 10.5259
Technology use: content% 89 0.00 0.82 0.4040 0.1793
PR
11
Technology use: HOTS% 89 0.00 0.80 0.2932 0.1495
12
Technology use: literacy% 89 0.00 0.78 0.3011 0.1597
13 Technology use: alone responding% 89 0.00 0.95 0.1692 0.2806
14 Technology use: alone-creating% 89 0.00 1.00 0.3382 0.2854
15 Technology use: with others% 89 0.00 1.00 0.4782 0.3280
Technology use: constructivist 94 1.58 4.50 3.3895 0.6923
16
17
Technology use: success
Teacher openness to change
89
94
ED
2.07
2.00
5.00
4.50
4.2742
3.6436
0.6365
0.5917
18 Teacher non-school computer use (no. times) 66 0.00 7.50 1.3182 1.5634
19
CT
20 Outcome measures
21
Technology impact on content 94 2.00 4.75 3.8654 0.5160
Technology impact on HOTS 94 2.00 4.75 3.7791 0.5649
22
Teacher morale 94 2.50 4.03 3.3793 0.3423
23
Technology integration 94 1.02 4.74 3.5320 1.0408
RE
26
27 Table 3
R
28 Summary of regression analysis for factors predicting technology impact on student content acquisition
29 Variable Unstandardized coefficients Standardized coefficients
CO
30
B S.E.
31
34
35 R2=0.589.
36 * P< 0.05.
37 ** P< 0.01.
38
*** P< 0.001
39
40 5. Discussion
41
42 The regression analyses identified that each of the five dependent variables were predicted by a
43 different combination of independent variables (with R2’s ranging from 0.164 to 0.608). Although
44 these predictions do not establish a cause and effect relationship, they do indicate that in the
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 Table 4
2 Summary of regression analysis for factors predicting technology impact on student higher-order thinking skills
3
Variable Unstandardized coefficients Standardized coefficients
4
F
5
B S.E.
6 Teacher openness to change 0.413 0.120 0.433**
OO
7 Technology use: alone-creating% 0.612 0.246 0.288*
8 Technology use: constructivist 0.244 0.110 0.290*
9
R2=0.608.
10 * P< 0.05.
PR
11 ** P< 0.01.
12
13
Table 5
14
Summary of regression analysis for factors predicting technology impact on teacher morale
15
21 R2=0.559.
22
* P< 0.01.
** P< 0.001.
23
RE
24
25
Table 6
26
Summary of regression analysis for factors predicting technology impact on teacher technology competency
27
R
30
31
Teacher openness to change 0.455 0.174 0.405*
32 R2=0.164.
33 * P< 0.05.
UN
34
35
36 Table 7
37
Summary of regression analysis for factors predicting technology impact on technology integration
38 Variable Unstandardized coefficients Standardized coefficients
39
B S.E.
40
1 schools and classrooms studied, higher scores on the independent variables were related to higher
2 scores on the dependent variables. The following sections describe the prediction of the five out-
3 comes and the implications for schools, teachers, and students.
4
F
5 5.1. The impact of technology on content acquisition
6
OO
7 School communities often regard content acquisition as a rationale for earmarking educa-
8 tional technology funds even though previous research has shown that simply placing technol-
9 ogy in schools does little to increase student learning. This study found three other variables
10 to be important: (1) the strength of technology leadership at a school; (2) teacher openness to
PR
11 change; and, (3) negatively influenced by teacher non-school computer use. These three vari-
12 ables had the greatest predictive value for student content acquisition as measured by teacher
13 perceptions.
14 Strong technology leadership, indicated through both administrative and teacher interviews,
15 may influence student content acquisition through a variety of ways. Administrators may serve as
16
17
ED
positive role models for both students and faculty. If teachers and students perceive that the
administrator values and uses educational technology, it may be more widely incorporated in the
18 classroom and more conscientiously used by students. In addition, strong technology leaders
19 tend to promote technology by providing acknowledgements and incentives. These activities
CT
20 may reinforce the importance of technology, thereby influencing its use by both students and
21 faculty.
22 Teacher openness to change was also found to impact student content acquisition. As teachers
23 progressively integrate computers into their curriculum, they consciously and inextricably dele-
RE
24 gate some of their duties to the computer and as a result are aware of the changes in their role.
25 The more that the teacher remains sensitive to, is prepared for, and is able to adapt to change, the
26 greater the impact of the technology (Rieber & Welliver, 1989/1990). Further, Marcinkiewicz
27 (1994) found that self-competence and innovativeness were most closely related to the level of
R
30 impact on content acquisition. Perhaps the more the teacher used technology out of the class-
31 room, the more s/he was a higher-end user and may have focused on the technology itself rather
32 than the application of the technology in the classroom.
33
34
35
36 The development of HOTS is often stated as an expressed goal of schools, but few studies have
37 examined which variables help reach this goal. We found that the impact of technology on
38 higher-order thinking was predicted by (1) the degree of teacher openness to change, (2) nega-
39 tively influenced by the amount of technology use by students working individually in creative
40 situations, and (3) the level of constructivist modes of technology use.
41 We found a strong positive relationship between teachers who had a higher degree of openness
42 to change and the impact of technology on students’ higher-order thinking skills (R=0.519,
43 P<0.001). This relationship may be because teachers who are innovative and adaptive are better
44 able to implement new teaching strategies that nurture these skills.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 The negative influence of activities where the individual is using a computer in creating mate-
2 rials may be because students working in isolation may be less conducive for the development of
3 these skills than working in collaborative groups.
4 It is apparent that not all computer technologies augment higher-order thinking skills. Teachers
F
5 who desire this development in their students need to extend their use of technology to areas
6 beyond simple drill and practice or tutorial software (which have their strength in accentuating
OO
7 basic skills). To accentuate the development of higher-order thinking skills, the computer needs
8 to be used in activities that incorporate the skills desired to build. Often these activities fall into
9 the realm of constructivist activities, in which students are required to examine and manipulate
10 resources, then collaboratively construct artifacts of their knowledge. Consequently, the rela-
PR
11 tionship of constructivism as the third predictor would be expected.
12
15 We found that the level of teacher morale was predicted by two variables: professional devel-
16
17
ED
opment and constructivist use of technology. Teacher morale influences all aspects of the teaching
and learning environment within the school setting. Morale can be made up of many commitment
18 and satisfaction elements, including availability of role models, rewards, recognition, encourage-
19 ment, professional development, incentives, empowerment in terms of support (e.g. technical) and
CT
20 the ability to demonstrate creativity in the school setting. These elements provide the foundation
21 upon which a teacher can make a positive difference in the learning environment.
22 It may be that building a strong base of foundational knowledge and skills through well-
23 designed professional development provides teachers with more confidence in their abilities. This
RE
24 confidence is manifested in higher levels of morale. A 1997 report by the President’s Committee
25 of Advisors on Science and Technology found that teachers currently receive little technical,
26 pedagogical, or administrative support in the area of technology competency. The committee also
27 found that few colleges of education adequately prepare their graduates to use educational tech-
R
28 nologies. As school districts increase expectations for technology in the classroom, the gap
29 between current and required skills becomes critical. Without adequate support, teachers may be
CO
30 unsure of best practices, leading to unclear expectations and an inability to cope with change and
31 a corresponding loss of morale. Consequently, proactive school districts are advised to implement
32 professional development programs early and often. Successful technology development pro-
33 grams offer tips, techniques, best practices, and models for classroom implementation. These
programs give teachers the opportunity to learn about and observe new teaching methods, share
UN
34
35 questions and problems with others, and to explore new ideas with experts (Hadley & Sheingold,
36 1993).
37 Improved teacher skills in the classroom, in turn, help facilitate improved student performance.
38 Through both personal improvement as well as their students’ performance, it stands to reason
39 that teacher morale increases. A critical factor, however, is that the professional development
40 program serve the needs of teachers with relevant examples and instruction. Simply providing off-
41 the-shelf workshops designed by external sources will not have as great an impact as when tea-
42 chers are surveyed and workshops are tailored to their needs.
43 We also found that the introduction of technology and constructivist learning philosophies into
44 the classroom environment affects the level of teacher morale (either positively or negatively) as
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 these factors facilitate a fundamental shift in the traditional environment, demanding that tea-
2 chers alter their styles and expectations. No longer is the teacher the only provider of information
3 or the ‘‘sage on the stage’’. Rather, the teacher becomes a coach, guide, and mentor, providing
4 student with the tools that they need to research, explore, and make meaning. This new role for
F
5 teachers may positively affect their morale.
6
OO
7 5.4. Teacher technology competency
8
9 It stands to reason that teachers who are more open to change will also be more willing to try
10 new ideas in the classroom as well as in their personal life. When these teachers are provided the
PR
11 opportunity to learn new technology skills and techniques, it appears that they avail themselves
12 of the opportunity with a resulting increase in their competence (r=0.329, P<0.01).
13 A key element, however, is that the school environment recognizes this initiative and provides
14 positive feedback, training, and technical support. Teachers are strongly attuned to what Mar-
15 cinkiewicz and Regstad call ‘‘the professional milieu’’ (1996, p. 31) consisting of students, peers
16
17
ED
and superiors. An innovative teacher who is using technology in the classroom will be sensitive to
both positive and negative responses from these sources. Administrators and policymakers seek-
18 ing to maximize technology use in the classroom are advised to recognize this interrelationship
19 between their leadership, teacher responsiveness, and school results.
CT
20
23 We found that technology integration was predicted by two variables: teacher openness to
RE
24 change and the percentage of technology activities with others. Technology integration refers to
25 how transparently the technology was blended into the lesson, and whether it was used to convey
26 content in ways not easily done without technology. In contrast to activities that automate direct
27 instruction (for example, computerized drill and practice used in place of pencil-and-paper tests),
R
28 integrative lessons often provide students with greater challenge in the form of research,
29 exploration, or expression. These alternative modes of instruction, frequently implemented as
CO
30 collaborative activities, may allow students to construct deeper meanings of the content.
31
32 5.6. Limitations
33
One issue that merits attention is the dependent measures of content acquisition and HOT
UN
34
35 skills. These measures were obtained through focusing on the teacher’s technology-related activ-
36 ities and the teacher’s assessment of student performance. While we could have had each teacher
37 make a global assessment of student performance based on overall recollection of activities over
38 the year, it would be significantly less precise than going through each activity and mathemati-
39 cally computing the assessment. Consequently, we limited the elaboration of activities to three,
40 which were randomly selected. Importantly, the activities were discussed in an interview format
41 so that the interviewer could probe the teacher to evaluate the task in a way that was appropriate
42 for the analysis.
43 Along this line, we did not obtain teacher assessment of student proficiency given that there is
44 nothing for the teacher to compare the class to (except to previous classes which may not have
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
F
5 The teacher’s assessment of class ‘‘performance’’ per se is questionable, given that there is no
6 external source to compare the assessment. (We would theoretically need a ‘‘super‘‘ teacher who
OO
7 could normatively rate the class performance relative to all the other classes in the study.) Con-
8 sequently, there is a subtle but important difference in the data we collected for the impact of
9 technology on content acquisition and HOTS: we obtained data on the teacher’s perceived
10 improvement in performance by the class as a result of the technology-related activity. According to
PR
11 the interview protocol, the teacher referred to their gradebook to refresh memory about class
12 performance on each technology-related activity. Specifically, they assessed the perceived
13 improvement in class performance on the two dimensions of content and HOTS.
14 Further, given that the selected schools were highly technology-integrative, the generalizability
15 of results regarding factors such as teacher morale may be limited to similar schools that are
16
17
ED
highly supportive of technology. Future research should include a more diverse sample of schools
in terms of technology support in order to address this limitation.
18
19
CT
20 6. Conclusion
21
22 The degree of teacher openness to change was repeatedly found to be a critical variable as a
23 predictor in our study. Teachers who are open to change, whether this change is imposed by
RE
28 and its subsequent influences are embraced in a classroom, administrators and policymakers may
29 wish to encourage further development in this area. Unfortunately, given that it is a personal
CO
34
35 identify those who actively use, model, and reward teachers who infuse technology into their
36 classroom. The bottom line appears to be that administrators who wish to nurture a technology
37 culture need to figuratively ‘‘roll up their sleeves and join in’’ rather than sitting by the side.
38 Although we found that administrators contribute to the positive interactions of technology in
39 a school, of greater importance were teacher attributes. Long-range planning for software devel-
40 opers and schools of education should include a vision that nurtures decision-making and devel-
41 opment by teachers, rather than implementing systems solely from the level of policymakers.
42 Currently many technology initiatives rely upon policymakers to communicate the value of
43 technology to teachers, instead of involving teachers from the start. By helping teachers find ways
44 to actively infuse technology, investments in time and money will pay off in greater content
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 acquisition and higher-order thinking skills for students and greater teacher competence and
2 morale.
3
F
5 Uncited reference
6
OO
7 Scrogan, 1989
8
9
References
10
PR
11 Anderson, L.S. (1996). Technology planning at state, district, and local levels. ERIC Digest, (ERIC Document Repro-
12 duction Service No. ED 393 448).
13
Becker, H. J. (1991). How computers are used in United States school: basic data from the 1989 I.E.A. Computers in
Education Survey. Journal of Educational Computing Research, 7(4), 385–406.
14
Brunner, C. (1992). Integrating technology into the curriculum: Teaching the teachers (No. CTE-TR-25). New York:
15 Bank Street College of Education, Center for Technology in Education.
16
17
ED
Dede, C. J. (1994). Leadership without followers. In G. Kearsley, & W. Lynch (Eds.), Educational technology: leader-
ship perspectives (pp. 19–28). Englewood Cliffs, NJ: Educational Technology Publications.
Elmer-Dewitt, P. (1991). Education: the revolution that fizzled. Time, May, 48–49.
18
Evans-Andris, M. (1995). An examination of computing styles among teachers in elementary schools. Educational
19
Technology Research and Development, 43(2), 15–31.
CT
20
George, Y. S., Malcolm, S. M., & Jeffers, L. (1993). Computer equity for the future. Communications of the AMC,
21 36(5), 78–81 Retrieved from the World Wide Web: [Link]
22 Grabe, M., & Grabe, C. (1998). Integrating technology for meaningful learning (2nd ed.). Boston: Houghton Mifflin Co.
23 Hadley, M., & Sheingold, K. (1993). Commonalities and distinctive patterns in teachers’ integration of computers.
RE
24
American Journal of Education, 101, 261–315.
Hawkins, J., & Collins, A. (1992). Design-experiments for infusing technology into learning. Educational Technology,
25
32(9), 63–67.
26 Kulik, J. A. (1994). Meta analytic studies of findings on computer based instruction. In E. L. Baker, & H. F. O’Neil Jr.
27 (Eds.), Technology assessment in education and training. Hillsdale, NJ: Lawrence Erlbaum.
R
28 Lajoie, S., & Derry, S. (1993). Computers as cognitive tools. Hillsdale, NJ: Lawrence Erlbaum Associates.
29
Liao, Y. (1992). Effects of computer-assisted instruction on cognitive outcomes: a meta-analysis. Journal of Research
on Computing in Education, 24(3), 367–380.
CO
30
Marcinkiewicz, H. R. (1994). Computers and teachers: factors influencing computer use in the classroom. Journal of
31
Research on Computing in Education, 26(2), 220–237.
32 Marcinkiewicz, H. R., & Regstad, N. G. (1996). Using subjective norms to predict teachers’ computer use. Journal of
33 Computing in Teacher Education, 13(1), 27–33.
Maurer, M. M., & Davidson, G. S. (1998). Leadership in instructional technology. Columbus, OH: Merrill.
UN
34
35
Niederhauser, D. S., Stoddart, T. (1994). Teachers’ perspectives on computer-assisted instruction: transmission versus
construction of knowledge. Paper presented at the Annual Meeting of the American Educational Research Associ-
36
ation, New Orleans, LA.
37 Office of Technology Assessment (OTA). (1995). Teachers and technology: making the connection. Washington, DC: US
38 Government Printing Office.
39 Picciano, A. G. (1998). Educational leadership and planning for technology (2nd ed.). Upper Saddle River, NJ: Prentice
40
Hall.
President’s Committee of Advisors on Science and Technology, Panel on Educational Technology. (1997). Report to
41
the President on the Use of Technology to Strengthen K-12 Education in the United States. Retrieved 15 January, 1998
42
from the World Wide Web: [Link]
43 Raizen, S. A., Sellwood, P., Todd, R. D., & Vickers, M. (1995). Technology education in the classroom: understanding
44 the designed world. San Francisco: Jossey-Bass Publishers.
CAE 593 Disk used No. pages 20, DTD=4.3.1
ARTICLE IN PRESS Version 7.51e
1 Rhodes, D. C. (1994). Sharing the vision: creating and communicating common goals, and understanding the nature of
2 change in education. In G. Kearsley, & W. Lynch (Eds.), Educational technology: leadership perspectives (pp. 29–38).
3
Englewood Cliffs, NJ: Educational Technology Publications.
Rieber, L. P., & Welliver, P. W. (1989). Infusing educational technology into mainstream educational computing.
4
International Journal of Instructional Media, 16(1), 21–32.
F
5
Rubin, L. (1989). Curriculum and staff development. In M. F. Wideen, & I. Andrews (Eds.), Staff development for
6 school improvement (pp. 170–181). New York: Falmer.
OO
7 Ryan, A. W. (1991). Meta-analysis of achievement effects of microcomputer applications in elementary schools. Edu-
8
cational Administration Quarterly, 27(2), 161–184.
Salomon, G. (1986). Educational technologies: what you see is not (always) what you get. Educational Psychologist, 20,
9
207–216.
10
Scrogan, L. (1989). The OTA report: teachers, training, and technology. Classroom Computer Learning, January, 80–84.
PR
11 Sergiovanni, T. J. (1995). The principalship: a reflective practice perspective. Boston: Allyn and Bacon.
12
13
14
15
16
17
ED
18
19
CT
20
21
22
23
RE
24
25
26
27
R
28
29
CO
30
31
32
33
UN
34
35
36
37
38
39
40
41
42
43
44