Perceived Effectiveness of Asynchronous Online Instruction: An
Evaluation of Teachers' Motivation, Teaching Skills and Satisfaction with Web-based Resources
By
Francisco Garabitos
Cluster
15
An Applied Dissertation Presented to the
EdD Program in Instructional Technology
and Distance Education
in Partial Fulfillment of the Requirements
for the Degree of Doctor of Education
Nova Southeastern University
2000-2005
Pre-Approval
Draft
This applied
dissertation was submitted by Francisco Garabitos under the direction of the persons listed below. It was submitted to the
EdD Program in Instructional Technology and Distance Education in partial fulfillment of the requirements for the degree of
Doctor of Education at Nova Southeastern University.
Todd A. Curless, PhD
Committee Chair-Adviser
Jared Bucker, EdD
Soledad Arguelles, PhD
Committee Member
Katherine
Tsamasino, PhD
Dissertation
Adviser
Lisa Dillengerr PhD
Dissertation Adviser
Margarite Foxon
Dissertation Adviser
Beverly Trash, EdD
Cluster XV Coordinator
Sharon Smaldino, EdD
Michael Simonson, PhD
Lya Viser, PhD
Fabio Chacon PhD
Program Professors
Barbara
Packer, EdD
Director of
Applied Research
Maryellen Maher, PhD
Executive Dean for Technology
and Research
Jaime Arango
Nova Institutional Review Board
Dr. Mark A. Seldine
Director, Student
Judicial Affairs
Henry Solomon, EdD.
New York City Department of Education
Office of Research and Accountability
****
Abstract
Perceived Effectiveness of Asynchronous
Online Instruction: An Evaluation of Teachers' Motivation, Teaching Skills and Satisfaction with Web-based Resources.
Garabitos, Francisco. 2003 Field data based: Applied Dissertation Proposal, Nova Southeastern University, EdD Program Instructional
Technology and Distance Education. Staff development/ Motivation/ Satisfaction/ Computers/ Internet/ Online Learning/ Instructional
Technology.
The New York City Department of Education
published a website to provide asynchronous online resources for staff development activities. Despite efforts to introduce
the site and the provision of e-mail accounts to all teachers, there was a lack of data on how many of them were using the
web-based resources provided, and with what effect.
This dissertation is the result of an evaluation study to assess the effectiveness of these web-based resources for staff
development. The method of research employed a quasi-experimental design with two existing groups of teachers. The website
(nycenet.edu/oit) was introduced to one group of teachers in New York City and its effects were assessed using two surveys.
A pre-intervention survey measuring attitudes and expectations was hand-delivered to the two groups. Teachers in the experimental
group were encouraged to access the new website as part of their professional staff development activities. The control group
did not have access to the website. After an eight-weeks period, a post survey was administered to teachers in both groups
and their responses were analyzed to determine the effects of the web-based resources.
This study
addressed a twofold research question: (a) how effective is asynchronous online instruction and (b) to what extent do the
web-based resources affect teachers' motivation, teaching skills and satisfaction. It was hypothesized that teachers,
who use the web-based resources, would report greater levels of motivation, teaching skills and satisfaction. The results
of this study rejected the null hypothesis. Although the web-based resources did not meet teachers’ expectations to
improve teaching skills, analysis of the data revealed that the online intervention did have a significant effect on teachers’
motivation and satisfaction.
A copy of the results was provided to participant schools in the NYC Department of Education and a follow-up study is recommended
to further facilitate access to current web-based resources. Nova Southeastern University is authorized
to include abstracts of this study in the university publications and to disseminate the applied dissertation upon request.
Chapter 1: Introduction
The convenience, flexibility and up-to-date content available online
today, is generally expected to facilitate technology integration into classrooms and administrative areas. The NYC Department
of Education established the Office of Instructional Technology (OIT) to provide web-based resources and promote integration
of technology into the classroom. Technology teachers were enthusiastic about exploring the Internet and OIT began expanding
their website to include asynchronous, online staff development resources for all teachers (Gura 2002). The NYC Department
of Education allocated substantial resources for technology training by negotiating a $500,000 contract with Classroom Connect™
(2001) to expand the professional development website (nycenet.edu/oit).
The Office of Instructional Technology
owns the instructional content enhanced by educational links and offers several modules of self-paced lessons with open access
to all teachers. The site provides asynchronous, online introductory courses and mini-lessons to integrate technology into
the classroom. The OIT also offers online courses for (a) teachers who are new to elementary, middle, and high schools, (b)
classroom management, and (c) specific content areas such as Math, Science and Social Studies (Gura, 2002). These steps taken
are indications that the NYC Department of Education is committed to provide effective web-based staff development resources
and this study examines supporting data on their success in meeting that goal.
Research Question
This investigation provides answer for a
twofold research question: (a) how effective is asynchronous online instruction and (b) to what extent do the web-based resources
affects teachers' motivation, teaching skills and satisfaction. The overall effectiveness of web-based resources provided
by the NYC Department of Education was further examined with additional survey items such as: how much time teachers spent
learning online and to what extent they integrated online learning into their teaching activities.
Statement
of the Problem
According
to the United Federation of Teachers (UFT), the high rate of staff turnover driven by early retirement, exodus to better paying
locations and the influx of new teachers heightened the need for a continuous cycle of face-to-face and online staff development
(Campanile, 2001). To promote online instruction and technology, the Office of Instructional Technology organized two citywide
conferences in 2001 and 2002 featuring web-based resources. The NYC Department of Education also assigned e-mail accounts
to all teachers (New York Teacher, Nov. 7, 2001). Since September 2001, it began offering online professional development
courses for teachers, supported in part by Enterprise Training Solutions (ETS). Teaching Matters, Inc. and Classroom Connect™
have also been conducting some forms of face-to-face training to provide online support to city schools. The NYC Department
of Education has committed itself to promote face-to-face and web-based professional development for the administrative and
teaching staff (Levy, 2001). However, during the researcher's visits to the district office and the target schools, many
teachers complained of not being properly informed about the available training opportunities. Personal interviews conducted
by the researcher during the current school year also indicated that very few teachers participate online.
Despite the efforts of the Office of Instructional
Technology (OIT) to introduce the web-based resources and the Chancellor's provision of e-mail accounts to all teachers,
there is a lack of data on how many of them are using the web-based resources, and with what effect. As the NYC Department
of Education continues the purchase and deployment of online technology, this research was needed to assess the effects of
asynchronous online instruction and measure the impact of the web-based resources on teachers' motivation, teaching skills
and satisfaction.
Rationale for the Study
The purpose of this study was to assess the effectiveness of using web-based resources as an alternative method
to provide staff development for teachers. After reviewing the Chancellor's Report (Levy 2001) and making personal visits
to several schools and the district office, the researcher found a need to examine the web-based staff development resources
provided by the New York City Department of Education. In one particular district (CSD #12, Bronx, NY), there was no comprehensive
technology plan and no senior person championing the integration of technology; one person was in charge of Math, Science
and Technology with no clear communication channels opened to the schools. Thus, many teachers and principals were on their
own to provide staff development and facilitate the integration of technology.
A report on these technical and instructional difficulties was submitted to the Office of Instructional
Technology (OIT) by our school staff developers. The OIT started as a small program to install computers in middle school
classrooms as part of Project Smart Schools. Today, OIT is an established support center, which includes teacher workshops,
citywide conferences and web-based resources (Gura, 2002).
The applied
research objective of this study was to determine if the use of asynchronous online resources (as provided by the Department
of Education) has any impact on motivation, teaching skills and satisfaction. The study hypothesizes that if the use of the
website improves teachers’ satisfaction in general (asynchronous online resources in particular), more teachers in similar
work-settings eventually may enjoy the benefits of additional web-based courses. In fact, additional online services have
been purchased from Enterprise Training Solutions (ETS) through its e-Learning system titled “My Skill Source”.
The website allows selected users to choose from hundreds of self-paced and instructor-led online courses, including a full
library of digitized e-books. Teachers, technical staff and administrators can access these training courses and e-books over
the Internet, any time of the day or night (ETS, 2001).
Instructional designers and educational institutions have the implicit
responsibility to provide the means for interactivity between instructors, learners and resources (Dick & Carey, 1996).
Both synchronous (at the same time) and asynchronous interaction (communication at different times) has been cited as critical
to student satisfaction and success with online courses (King & Doerfert, 1996). To this end, the New York City Department
of Education is taking the necessary steps to provide an interactive learning environment in which web-based resources are
made available on a regular basis.
The website
(portal) created by the NYC Department of Education is an enhanced site intended to facilitate Internet access and to centralize
other services. These kinds of sites are becoming the largest assortment of software that an educational institution might
deploy online. For example, Gettysburg College educational portal has evolved since 1997, adding lesson modules for students,
parents and faculty in 1998, alumni references in 1998, students’ portfolio in 2000, administrative tools, help-desk
and real timesheet forms in 2001 (Gleason, 2001).
The NYC Department
of Education is heading in a similar direction by investing citywide funds through the federal E-Rate program (Levy, 2001).
The fund provided city schools with Internet access to the NYC Department of Education's new portal and equipped every
classroom with the capability of participating in online staff development activities. Consequently, teachers and educators
in general need to examine the extent to which these online technologies are effective.
Since 1994, the National Center for Education Statistics (NCES)
has reported a significant increase in access to the computer and the Internet in the nation's public schools. Using the
Fast Response Survey System (FRSS), NCES found in 1999 that nearly 99% of full time regular public school teachers have increased
access to computers or the Internet in their schools (Scales, 1994; Williams, 2000).
Although there are hundreds of web pages and educational sites
available free on the Internet (See, for example, www.teachers.net,
www.school.aol.com), little was known
about the extent to which, K-12 teachers were using web-based resources to increase their teaching skills and integrate technology
into their classrooms. While this evaluation did not compare face-to-face with online instruction, it helped to determine
the extent to which the Department of Education website and its resources are being used for staff development.
Hypotheses
The directional hypothesis for this study stated that asynchronous online instruction is an effective method
and predicted that teachers who access the website would increase their levels of motivation, teaching skills and satisfaction.
The null hypothesis stated that the use of asynchronous, web-based resources would make no difference on motivation, teaching
skills or satisfaction. The result of this study (Chapter 4) rejected the null hypothesis.
Variables
The independent variable was: teachers contact with the website and the use of web-based resources
for staff development. The effect of the online intervention was determined by examining three dependent variables: (a) motivation
(b) teaching skills and (c) satisfaction. A combined analysis of these variables and additional survey items (Appendix A-B)
was used to assess the overall effectiveness of asynchronous online instruction and measure the effects of the web-based resources
on teachers.
Theories
Several theories
were examined in the review of the literature. The most influential theoretical implications for this study were based on
the theory of transactional distance (Moore, 1989, 1993) with an emphasis on computer-mediated communication and the types
of interaction that occurred between learners, instructors and resources. This study built upon previous quasi-experimental
and survey-based research studies conducted to assess the effectiveness of technology and web-based instruction (Giguere,
2002; Fadden, 2000; Lesh, 2001; Kurubacak, 2000; Murray & Starr, 2001; Sener, 2001; Wegner, Holloway & Garton, 1999).
Limitations of the Study
There are
several limitations to the generalization of the results of this study. The sample was a relatively small number of teachers
selected from only two of NYC public schools. Since all subjects were elementary school teachers working for the New York
City Department of Education, their perceptions may be a reflection of public education in two schools, within a large city,
and not of the general teaching population. The voluntary nature of teachers’ participation and the brevity of time
(eight weeks) during their summer vacation may have affected the subjects’ perception on satisfaction, since those who
were already motivated are more likely to volunteer and learn on their own. The limited web-resources used as treatment in
this study may have also contributed to its limitations. Furthermore, the online intervention relied on resources listed by
one specific website and findings may not be generalized to other online staff development initiatives.
Definition
of Terms
Asynchronous discussion.
Two-way communication that occurs with a time delay, allowing participants to respond at their own convenience (different
time, different place).
Asynchronous Learning Network (ALN). A distributed community of learners who, by having
access to a computer network, could communicate with each other and access learning materials at any time and from any place.
Asynchronous implies that most online learning activities do not occur in real time.
Computer-based training (CBT).
An umbrella term for the use of computers in both instruction and management of the teaching and learning process. CAI (computer-assisted
instruction) and CMI (computer-managed instruction) are included under the heading of CBT. Some people use the terms CBT and
CAI interchangeably.
Distance education. Educational situation in which time, location,
or both separate the instructor and the learners. Education or training courses are delivered to remote locations via synchronous
or asynchronous means of instruction which include written correspondence, graphics, videotape, CD-ROM, facsimile, hypertext,
audio-videoconferencing, interactive TV, Internet, etc.
Effectiveness. The
extent to which a given online program is capable to achieve the intended goal and objectives. In the context of this study,
effectiveness was determined by examining teachers' perceptions on motivation, teaching skills and satisfaction.
Instructional
designer. An individual who applies a systematic methodology based on instructional theory to create content
for face-to-face and online learning events.
Interactive. The process allows for a two-way interaction or exchange
of information between participants (student-teacher or student-student) in an instructional event.
Media. The collective
use of any medium as a channel of communication, by which instruction is created, stored, manipulated or presented to the
learner.
Online learning. The process of learning
new skills and acquiring knowledge via the Internet, without needing to be physically present in the learning environment.
Portal.
A centralized gateway to the Internet. An enhanced website that offers learners and organizations a consolidated access to
different learning and training resources from multiple providers.
Synchronous discussion. Interaction
between two or more people where the exchange of information occurs at the same time; that is, with no appreciable delay between
the end of one message and the beginning of another.
Web-based resources. Software, lessons, courses and hyperlinks provided
online to support instructional activities.
Summary
The use of technology
at all levels of the school system was considered as an important aspect of the organizational structure supporting public
education. This evaluation study was proposed to compare two existing groups of elementary school teachers, one of which had
access to resources posted on a website hosted by the New York City Department of Education. Although the NYC Department of
Education traditionally relies on face-to face instruction to train its staff, a high turnover of new teachers heightened
the need for additional online staff development. The New York City Department of Education invested a substantial amount
of money to help teachers in the integration of technology into their classrooms. To this end, it continues to provide a variety
of asynchronous web-based resources to support staff development activities. Although Department of education promoted
the integration of technology into the classroom, there was a lack of data on how many of teachers were using the resources,
and with what effect. The accessibility to web-based resources lead this study to ascertain the extent to which asynchronous
online instruction affects teachers' perceptions on motivation, teaching skills and satisfaction. The results (Chapter
4) rejected the null hypothesis by reporting a significant change on motivation, teaching skills and satisfaction.*
*For complete document contact: fgart44@gmail.com