Course Management System Utilization and Implications for Practice: A National Survey of Department Chairpersons


Dr. Charles F. Harrington
Assistant Vice President for Academic Affairs
Associate Professor of Business Statistics
University of Southern Indiana
cfharrin@usi.edu

Dr. Scott A. Gordon
Associate Dean of the Pott School of Science and Engineering
Associate Professor of Biology
University of Southern Indiana
sgordon@usi.edu

Dr. Timothy J. Schibik
Director of the Center for Teaching and Learning Excellence
Professor of Economics
University of Southern Indiana
tschibik@usi.edu

Overview

Over time, higher education has seen a number of innovations, some revolutionary, others having minimal to no impact (Katz, 2003). Over the last decade, the development of computer software and hardware directed toward education and the teaching and learning process has had tremendous impact on course delivery (Glahn and Gen, 2002; Katz, 2003). During this period, higher education has been witness to fundamental changes from courses delivered in the traditional face-to-face method to those delivered via video cassette and television, to a proliferation of courses and course content delivered via computer technologies. In recent years, the use of Internet resources (i.e. web pages) in course and curriculum development has made a significant impact on teaching and learning. The use of the Internet has evolved from the display of static, dull, and lifeless information to a rich multimedia environment that is both engaging, dynamic, and user friendly (Powel and Gill, 2003).

During this period, the Internet has become an important component in the teaching and learning process. As a result, the use of the Internet in higher education settings has become a more accepted and widely used tool in academia (Angelo, 2004; Glahn and Gen, 2002; Hawkins, 2004; Katz, 2003; Maslowski, et al., 2000). With the advent of web editing tools and other programs, the need to learn HyperText Markup Language (HTML) and other programming languages has diminished. Most recently, the development and refinement of university and commercially developed course management systems (CMS) like Blackboard , WebCT,and Prometheus, have resulted in the proliferation of web use in higher education (Angelo, 2004; Morgan, 2003). These technologies have made it possible to easily and efficiently distribute course information and materials to students via the Internet and have allowed for greater online communication and interaction (Gray, 1998 and1999; Stith, 2000). While these tools were initially developed for use in distance education pedagogies, their use in on-campus classroom settings to compliment traditional courses is now considered a viable and often preferred option. As a result, many academic units (i.e. departments) are struggling to keep pace with the demand for CMS supported course sites for traditional, face-to-face courses.

CMS have shown to significantly increase student involvement in multiple aspects of courses (Stith, 2000). The ability of instructors to control access to a variety of course materials – syllabi, lecture notes, outlines, images, etc. allows students access to such material from virtually any location. For the instructor, a multitude of options exist for developing, implementing, revising, and delivering course content. At the department level, these tools can have a profound effect on faculty teaching and student learning, departmental communication, and faculty workload.

However, there has been little meaningful work devoted to the assessment of CMS adoption. To what degree has the emergence and development of CMS led to improved teaching and learning? More importantly, to what degree has CMS utilization contributed to student engagement, student learning, and the overall quality of teaching? The purpose of this article is to examine the perceived long-term impact of CMS on the work of the academic department and departmental faculty.

History of CMS

Today, according to the 2003 Campus Computing project, more than 80% of universities and colleges in the United States utilize CMS (Morgan, 2003). Perhaps no other innovation in higher education has resulted in such rapid and widespread use as the CMS. In the early to mid 1990's, faculty utilized a variety of web-based tools to supplement course content and curriculum. Many faculty began using email and basic HTML functionality in an attempt to increase interaction and enhance the teaching and learning process. Many universities, in an attempt to lessen the burden on faculty, hired webmasters and instructional designers to assist faculty in putting together more dynamic learner-friendly sites. During the mid 1990's, several higher education institutions and commercial (for-profit) companies foresaw the need for more user-friendly approaches to putting course materials on the web and the need for increased availability to learners via the Internet. Simultaneously, these entities began developing systems that would be relatively easy to use, requiring little or no knowledge of programming language (HTML, java), and with the tools necessary to be useful for instruction. Subsequently between 1995 and 1997, several university and commercial CMS applications were launched in the higher education market.

These early CMS saw only slight variation in available tools (Gray, 1998 and 1999; Katz, 2003). Over time, a core group of tools were available with essentially all CMS. These core components included tools for synchronous and asynchronous communication, content storage and delivery, online quiz and survey tools, gradebooks, whiteboards, digital dropboxes, and email communications. While the majority of these tools are seen in the most commonly used CMS today, the robustness, flexibility, and ease of use have generally all been refined. Additionally, a vast array of additional components have been added, including mechanisms for “just in time” delivery and integration to front and back-office administrative computing systems.

Adoption Strategies

Technology adopters can be split into two broad groups: early adopters and followers. Early adopters are characterized as faculty members who are intrigued by and comfortable with new technologies and want to try new and innovative approaches in their teaching. These technology “geeks” take great pleasure in trying the newest tools. The Wisconsin survey (Morgan, 2003) asked faculty why they adopted a CMS and the most common reason stated was classroom management. Early adopters may also be driven to improve the level of faculty-student communications.

Followers are more apprehensive and cautious individuals, perhaps not so technological literate, who need to wait and see how something works. These individuals will adopt a CMS because of pressure from colleagues and students, the willingness to follow the example of early adopters, or because of administrative pressure (Morgan, 2003).

Cost and Benefit Issues

When trying to make any choice, academic department chairpersons must consider both the costs and the benefits of making the choice. Choosing whether, how, or not to utilize a CMS is no exception. Surprisingly, there is a startling void in the research literature on the cost and benefits of CMS utilization in higher education. Much of this work is hampered by the exceedingly difficult prospect of gathering accurate measures of costs. Many institutions bury the actual operational costs of CMS by spreading the expense across multiple cost centers ( Computer Center , Instructional Technology, Distance Education, and Academic Computing). Furthermore, the proliferation of technology-enhanced instructional support has added additional costs in personnel and non-personal services, that has likewise, been spread across numerous cost centers.

However, an understanding of direct and indirect costs can be of significant benefit to department chairpersons in their efforts to assess effectiveness and utility of course management systems. It is posited that many colleges and universities make decisions concerning CMS using very little cost and benefit information beyond to basic level of data (if any is utilized at all). How do CMS impact the bottom line? Do the benefits of CMS usage exceed the costs? How is departmental work affected by the choice of a CMS?

Typical cost/benefit analysis involves the splitting of both costs and benefits into broad categories of direct and indirect effects. Direct effects are those things that the utilization of a CMS is known to impact (e.g. the dollar cost of a software site license impacts university budget directly). The indirect effects are the impacts not typically thought to be impacted by the choice (e.g. the addition of another layer of campus bureaucracy). In order to determine whether or not a correct choice has been made, both direct and indirect affects must be considered.

The employment of a CMS involves several direct costs. The choice of a CMS is not without significant monetary cost for the actual software package site license, payments for support contracts, and in many cases the computer hardware necessary to operate the system in-house. These direct costs are typically well-known, budgeted and widely reported across campus during the decision-making process. Monies for course development may fall into the category of direct cost, but surveys reveal that many faculty do not believe that sufficient dollars have been budgeted for either course or faculty development (Morgan, 2003).

Indirect costs, however, are much harder to quantify and may even be purposely hidden to minimize the true cost of the adoption. Faculty in a recent study at the University of Wisconsin (Morgan, 2003) noted that the adoption of a CMS increased their feeling of a loss of control over their courses. Software and hardware advisors on campus related to the chosen CMS would be in a position to tell the faculty member what they could and could not do with their course materials. This additional level of bureaucracy was more than likely not included in the original cost figures. The level of support staff for a CMS can be quite high with each individual requiring not only salary and benefit packages, but space. In addition, the monies spend on CMS support staff are no longer available for usage in other areas of the academic enterprise. Each of these indirect costs should be included in the value calculations and could impact monies available to the department.

Another commonly overlooked indirect cost involves the time commitment by both the university and faculty. Movement to a CMS involves a significant amount of time on the part of both existing and any newly hired staff. This time must be taken away from other existing tasks. Faculty must not only learn how to properly utilize the CMS, but most also learn how to manage any new hardware and must do so while maintaining their normal teaching, scholarship and service activities. Many faculty feel discouraged and believe that CMS are too time consuming (Morgan, 2003).

Departments might also experience an additional cost from their students based on the transferal of some traditional costs. CMS allow faculty to place considerable amounts of materials on the site that would have traditionally been distributed in class (e.g. handouts and readings). Uploading documents to the CMS may afford the department a short-term cost savings (and thus an indirect benefit) in the short-run, but the backlash from students from this transfer of both effort and printing costs may prove costly in the long-run. Students may very well resent the additional burden placed on them to print documents stored on the CMS and this may create a dilemma for the department. Additionally, the loss of student “good will” may more than offset any printing expense savings.

The benefits of CMS adoption may be more easily quantified both in direct and indirect forms. The reason stated by departments and faculty for adopting CMS technologies is that the CMS provides a means to address pedagogical challenges. Whether the challenge stated is the need to teach students at a distance to tap into emerging markets or the need to do a more efficient job providing for the needs of students in large classroom setting, the CMS is viewed as a means to solve the problem. CMS provide the academic department a means to move beyond the boundaries of their existing capital stock of buildings and classrooms. Student markets that were unreachable because of space or distance constraints can be tapped with the utilization of CMS. With expanded markets come the benefits of increased institutional exposure and additional sources of tuition revenue.

CMS also afford departments and faculty more reliable means to communicate with traditional students outside of the classroom setting. The communication features of CMS provide greater means of conveying classroom and departmental materials. Chat features and discussion boards allow department chairs, faculty, and students to broaden both the type and amount of communication both across and between groups. Faculty have an additional method of communicating with students and students have alternative means of communicating not only with their faculty but with each other. The benefit to having students avail themselves of greater contact with the faculty member may be substantial.

CMS communication technologies may also enhance traditional faculty student interactions. Faculty can use the CMS to conduct electronic advising session, administer research surveys, to distribute a variety of student evaluations of teaching and to conduct a variety of classroom assessments. The increased communication methods inherent in CMS can provide departments and universities with substantial benefits, including increases in student satisfaction, retention, and graduation rates.

CMS also serve as document repositories giving both the faculty and the student the ability to develop just-in-time strategies for learning. Students no longer need worry about losing an important assignment or reference and faculty know that students can gain access to relevant classroom materials 24/7.

In general, there are two main challenges in evaluating the costs and benefits of CMS adoption. First, the benefits that might be easier to identify are very difficult to put into meaningful dollar terms. How does one put a value on increased possible communication between faculty and students? Second, the cost figures are typically underestimated due to the omission of the indirect costs of adoption. While these two issues may actually offset each other in the calculation of benefits and costs, to make decisions based on incomplete information is not good departmental practice. Greater effort should be placed on quantifying CMS benefits and greater care needs to be taken to truly represent the costs of adoption.

A National Survey of CMS Utilization

During the spring of 2004, the authors conducted a survey of a national random sample of 350 academic department chairpersons. The sample was constituted from a list of institutional members of the American Association of State Colleges and Universities and private institutions identified on the Association of Governing Boards member list. One hundred fifty eight (158) respondents completed the survey for an effective response rate of 45.1%.

The survey instrument designed and field tested by the authors, collected data and information relative to the perceptions of academic department chairpersons and program directors on the life-cycle of web-based course management systems. The authors were also interested in the perceived degree of utilization of course management systems over time, and the perceptions of department chairpersons regarding the degree to which they perceived that the use of course management systems led to measurable increases to either student learning or quality of instruction. A copy of the survey instrument can be found in Appendix A.

The on-line survey was structured to collect data and information in four major areas (1) demographic data on institutional control, institutional type, location of institution (state), and the academic discipline represented by the respondent; (2) the length of time of CMS usage, the primary types of course for which a CMS is utilized; and the number of CMS course shells developed by departmental faculty; (3) an assessment of the degree to which CMS utilization over time has contributed to student engagement, student learning, quality of teaching; and (4) an evaluation of the reliance, time commitment, and enthusiasm of departmental faculty for CMS utilization over time. Analysis of survey data consisted of simple descriptive statistics.

Demographics

Sixty percent (60.1%) of survey respondents reported that their current academic assignment was at a public institution. Most reported being employed in a public, Master's comprehensive institution (29.5%), followed closely by department chairs at public Doctoral research institutions (23.1%). Nearly forty percent (38.5) of the respondents indicated employment at a private institution.

Department chairpersons at two year institutions comprised 3.8% of the sample.

Survey respondents came from 42 different U.S. states. The majority of survey respondents were from Midwest states, followed by the southeast, northeast, and Midwest .

Academic Affiliation

The academic disciplines reported by the survey respondents were very broad, ranging from Accounting to Zoology. The most frequently identified academic discipline was Business (9), followed by Social Sciences and Communications (6 each), and Mathematics (5). When disciplines were group by academic classification, the majority of respondents (42.2%) chaired departments in the Liberal Arts and Humanities, followed by Science and Technology (21.4%), Nursing and Allied Health (14.5%), Business (11.7%), and Education and Human Services (10.3%).

Departmental History of CMS Utilization

Nearly ninety percent (89.1%) of the respondents indicated that their department is currently using a web-based course management tool (i.e. Blackboard, WebCT, Prometheus, e-College, etc). CMS adoption patterns were consistent among both public and private institutions.

When asked about the primary types of courses for which the CMS was used, the majority of respondents indicated that CMS was used primarily to support traditional face-to-face courses (44.1%). Nearly one third of the department chairpersons indicated that hybrid courses (partial on-line to supplement face-to-face courses) were the primary courses for which the CMS was used. Only one in four respondents (25.5%) indicated that the primary use of their CMS was to support web-based distance education courses.

Department chairs in public institutions indicated a greater prevalence of using CMS for support of distance education courses, whereas those at private institutions had a higher proportion of utilizing CMS primarily to support hybrid courses. There were no discernable differences between public or private institutions in using CMS to support tradition face-to-face courses.

A significant number of academic departments have been using course management systems for more than five years (20.8%). Twenty two respondents indicated that they had first utilized the Internet to support their academic courses as early as the fall of 1995, pre-dating most current course management systems. The majority of respondents (79.8%) have been using CMS technology to support their academic courses for more than two full academic years. Approximately one in ten respondents (10.8%) are relative newcomers to the use of course management tools, indicating departmental adoption as late as the fall of 2003.

The average number of CMS course shells developed by departmental faculty was 13 (maximum=100). Of those, the average number of course shells developed by faculty that are no longer used was 4 (minimum=1, maximum =10).

When asked to comment on the reasons why departments chose to no longer offer previously available courses via CMS, the most frequently identified reason cited was the instability of the CMS. One respondent adequately captured the frustrations “the [CMS] is cumbersome and takes entirely too much faculty time to maintain. The [CMS] frequently goes down, which angers students, frustrates faculty, and generally puts a pall on the whole thing. The [CMS] was supposed to make our lives easier. If anything, it has placed additional and unnecessary burdens on our faculty”.

Others cited the instructor-specific nature of CMS course shells. As faculty retire or depart the institution, the course shells are taken offline.

When asked to speculate on the degree to which technical difficulties (system down, connectivity and access issues, system modifications) with their departmental course management system was a disruption to teaching, the majority of department chairs (64.1%) indicated that such difficulties were a “minor” disruption. One in five (21.8%) felt that itinerate technical difficulties posed no threat of disruption to departmental courses.

Most departments use their CMS primarily to support small (25 or fewer students) lecture only courses. Few indicated that they used course management systems to support laboratory of large enrollment (50 or more students) courses.

Effects of CMS Utilization over Time

Nearly eighty percent of the respondents (78.5%) indicated that departmental utilization of and reliance on their course management system has increased over time. Fewer than ten percent of the department chairs indicated that CMS utilization and reliance had decreased over time. However, just over half (51.1%) of survey respondents indicated that faculty enthusiasm increased concurrently over the same time period. 48.6% responded that faculty enthusiasm for using the course management system had either decreased or remained unchanged over time.

More than two thirds (69.1%) of department chairpersons perceived increased student engagement as a result of CMS utilization, however only 47.1% felt that there were corresponding increases in student learning. One in twenty respondents (5.8%) believes that course management systems actually decrease student learning.

Respondents felt strongly that the amount of time their department faculty devoted to updating and managing web-based course sites over time had increased (71.1%). Approximately one-forth (24.6%) of the respondents felt that the time necessary to update and manage course sites was roughly equivalent to that of face-to-face course preparations. Only six of the 158 respondents (4.2%) indicated that CMS utilization resulted in a decreased time commitment over time,

When asked to indicate the perceived degree to which the quality of teaching and instruction was affected, over time, by course management systems, the majority of respondents (51.4%) felt that CMS utilization had not affected the quality of teaching in the department.

Survey respondents were also offered the opportunity to provide additional comments relative to their departmental utilization or perceptions of web-based course management tools. Fifty-five respondents (34.8%) provided comments. The tenor of the comments provided both support for and criticism of course management systems. A sample of the qualitative comments provided by department chairs is provided in Appendix B.

Conclusions

There is no evidence from the survey findings to suggest that departmental utilization of a CMS leads to increases in student learning. Furthermore, there is no evidence to suggest that departmental utilization of a CMS leads to increases in the quality of instruction (teaching).

In that the majority of survey respondents have been utilizing a CMS for more than two years – their responses are considered to be informed and non-trivial. It is to be assumed that their perceptions and opinions concerning the effectiveness and efficiency of CMS are valid and reliable.

Since the majority of respondents are from (at least) baccalaureate institutions, the responses are not generalizable to department chairpersons from community colleges.

In that the majority of respondents indicated that departmental CMS utilization was directed primarily to support traditional face-to-face courses, there is a question of value-added educational gains. The cost per semester credit hour of content delivery undoubtedly increases as a function of the integration of CMS.

In that respondents indicated no or negligible perceived gains to student learning and quality of teaching (instruction), the primary purpose and/or advantage for continued CMS utilization is considered to be convenience to students.

There is an issue of faculty effort as a function of time devoted to CMS course shell maintenance. Department chairs indicated no or negligible gains to student learning and quality of teaching (instruction), but did indicate a perceived increase in time commitment, departmental utilization, departmental reliance, and (oddly enough) faculty enthusiasm for CMS (although only 51.4% indicated increased enthusiasm). If faculty are spending significantly more time attending to CMS issues, what areas of faculty responsibility are receiving less time? How is this affecting the work of the department? How is this affecting the work of junior faculty (especially those who are CMS conversant and “probably” more apt than their senior departmental colleagues to use CMS)?

A compelling question is “are there external (to the department) pressures to increase departmental adoption/utilization of CMS”? And if so, to what extent do these pressures interfere with the “work” of the department?

Relative to the indication that nearly one third (4 of 13) of departmental courses initially migrated to CMS are no longer used; there is a question to the degree that faculty may have been originally lured to CMS by the “bells and whistles”, but found the tools too challenging or simply non-contributory to effective teaching and learning.

There remains the question of CMS functionality issues. To what degree do faculty initially fully exploit the capabilities of the CMS (chat, email, assessment, testing, etc), but over time, decrease use of all the ‘bells and whistles”?

When considering the degree to which technical difficulties interrupt “teaching” through the use of CMS, it begs an additional question; “if the only real advantage to departmental utilization of CMS is convenience to students, if there are technical difficulties that cause a delay of “transmission” of course content, then how is this time “made up”?

Are we seeing, what in essence, may amount to nothing more than another “fleecing” of American higher education? Are colleges and universities being taken to the “cleaners” by adopting the CMS “du jour” at increasing expense (site license, internal increases in ITS/DE staff, increased time commitment), all the while driving up the cost of instruction, and ultimately, the cost of education? And, of course, we pass the cost onto the student by assessing the not-so-clearly-defined-but-ever-increasing technology fee. Why is there a paucity of work on the economics of CMS? Is it because getting a clear and accurate picture of actual costs is so difficult to obtain?

Recommendations

•  Need for the Development of a Departmental Strategy for CMS Utilization

•  Attention to Departmental Assessment of CMS Effectiveness and Efficiency on teaching and learning

•  Need for Department-level professional development for effective CMS utilization


Bibliography

Angelo, J. 2004. New Lessons in Course Management. University Business . Online: http://www.universitybusiness.com/page.cfm?p=616

Glahn, R. and Glen, R. 2002. Progenies in Education: The Evolution of Internet Teaching. Community College Journal of Research and Practice . 26: 777-785.

Gray, S. 1998. Web-based Instructional Tools. Syllabus , September 1989.

Gray, S. 1999. Collaboration Tools. Syllabus , January 1999.

Hall, J. 2003. “Assessing Learning Management Systems.” Chief Learning Officer . January. Online: http://www.clomedia.com/content/templates/clo_feature.asp?articleid=91&zoneid=29

Hawkins, B., Rudy, J., and Madsen, J. 2004. EDUCASE Core Data Service 2003 Summary Report. Online: http://www.educause.edu/ir/library/pdf/PUB8001.pdf

Katz, R. 2003. Balancing Technology and Tradition: The example of course management systems. Educause, July-August 2003.

Lamberson, M. 2003. Course Management Systems: Trapped Content Silos or Sharing Platforms. Technology Enhanced Learning Research . The Ohio State University .

Lembke, R., and Clemmer, J. 2002. Faculty Member's Use of Course Management Systems: Results of a Survey. Midwest Instructional Technology Center . Associate Colleges of the Midwest . Online: http://www.midwest-itc.org/Share/FacultyResults.pdf

Mann, B. 1999. Phase Theory: A Taxonomy of Web Course Management. Conference Paper. 10 th Annual Conference on Web-based Teaching and Learning.

Maslowski, R.; Visscher, A.; Collis, B; and Bloemen, P. 2000. The Formative Evaluation of a Web-based Course Management System Within a University Setting. Educational Technology . May-June 2000.

Morgan. G. 2003. Course Management System Use in the University of Wisconsin System. EDUCAUSE Center for Applied Research (ECAR). May 2003.

Powell, P. and Gill, C. 2003. Web Content Management Systems in Higher Education. Educause Quarterly. 2:43-50.

Spodark, E. 2003. Five Obstacles to Technology Integration at a Small Liberal

Arts University . Technological Horizons in Education. April 2003.

Stith, B. Web-Enhanced Lecture Course Scores Big with Student and Faculty. Syllabus. March 2000.

Ullman, C. and Rabinowitz, M. (2004). Course Management Systems and the Reinvention of Instruction. Technical Horizons in Education . October 2004. Online: http://thejournal.com/magazine/vault/A5070.cfm

Van de Pol, J. 2001. “A Look at Course Management Systems.” IT Times May/June. Online: http://ittimes.ucdavis.edu/june2001/cms.html

Wilson , W. 2003. Faculty Perceptions and Uses of Instructional Technology. EDUCAUSE Quarterly , Volume 26, Number 2. Online: http://www.educause.edu/ir/library/pdf/eqm0329.pdf

Young, J. 2004. When Good Technology Means Bad Teaching. Chronicle for Higher Education. November 12, 2004. Online: http://chronicle.com/free/v51/i12/12a03101.htm

APPENDIX A

Web-Based Course Management

Utilization Life-Cycle:

Perceptions of Academic Department Chairpersons
Spring 2004

The purpose of this questionnaire is to collect data relative to the perceptions of academic department chairpersons and program directors on the life-cycle of web-based course management tools.  We are interested in the perceived degree of overall utilization and satisfaction over time.  Please answer the questions with candor.  The survey should take approximately five minutes to complete.  Your responses will be held in strict confidence and only aggregate data will be reported in the findings of this inquiry.  Should you have any questions, please contact Scott Gordon, Associate Director, Center for Teaching and Learning Excellence, University of Southern Indiana at sgordon@usi.edu .  Thank you for your assistance with this project.  We would appreciate your response by February 20th. 

Demographics

Institutional Control
    Public
    Private
    Two-Year Institution

Institutional Type
    Associate's/Two Year
    Bachelor's
    Master/Comprehensive
    Doctoral/Research

State in Which Your Institution is Located: : 

Academic Discipline of which you are Chair/Program Director: 

1.  Is your campus currently using a web-based course management tool (i.e. Blackboard, WebCT)  in one or more of your courses?
        Yes         No

2.  Primarily, for what type of course does your department use web-based course management tools?
        Traditional face-to-face on campus course
        Hybrid course (i.e. supplementing face-to-face course)
        Web-based distance education course

3.  What was the first semester/quarter that your department used a web-based course management tool for your academic courses?  

4.  How many years has your department used a web-based course management tool to support your academic courses?  
        1 or less
        1 - 2 years
        3 - 5 years
        5 or more years

5.  How many web-based course management tool course sites have faculty in your department created? 

6.  Over time, to what degree has departmental utilization of web-based course management tools changed?
        Our utilization of a web-based course management tool has increased .
        Our utilization of a web-based course management tool has decreased .
        Our utilization of a web-based course management tool has not changed .

7.  Over time, to what degree have students embraced your utilization of web-based course management tools?
        The use of a web-based course management tool has increased student engagement .
        The use of a web-based course management tool has decreased student engagement .
        The use of a web-based course management tool has not changed student engagement .

8.  Over time, what amount of time has your department devoted to updating its web-based course sites?
        Increased Time Commitment
        Decreased Time Commitment
        About the Same Time Commitment

9.  How do you perceive your use of a web-based course management tool in relation to student learning in the courses in your department?
        The use of a web-based course management tool has led to an increase in student learning .
        The use of a web-based course management tool has led to a decrease in student learning .
        The use of a web-based course management tool has led to no change in student learning .

10. Over time, how has your departmental reliance on a web-based course management tool in teaching changed?
        Over time, our reliance on a web-based course management tool has increased .
        Over time, our reliance on a web-based course management tool has decreased .
        Over time, our reliance on a web-based course management tool has stayed about the same .

11. Are there courses for which your department has developed web-based course sites, but no longer use them in teaching?
        Yes        No            How Many? 

12.  If you answered "Yes" to the previous question, why have you decided not to use these sites?

13. Over time, to what degree has the quality of teaching in your department been affected by the utilization of a web-based course management tool?
        The web-based course management tool has increased the quality of our teaching
        The web-based course management tool has decreased the quality of our teaching
        The use of a web-based course management tool has not affected the quality of our teaching

14. To what degree have technical difficulties (system down, access issues, system modifications) with a web-based course management tool been a disruption to your department's teaching?
        Technical difficulties have been a major disruption to our courses .
        Technical difficulties have been a minor disruption to our courses .
        Technical difficulties have not been a disruption to our courses .

15. Over time, how would you describe your department's enthusiasm for using web-based course sites for your courses?
        Our enthusiasm for using web-based course sites has increased .
        Our enthusiasm for using web-based course sites has decreased .
        Our enthusiasm for using web-based course sites has remained unchanged .

16. For what size classes do you find a web-based course management tool most useful?
        Small classes (<25 students)
        Medium classes (25 - 50 students)
        Large classes (>50 students)

17. For what type of course do you find a web-based course management tool most useful?
        Laboratory course only
        Lecture and Laboratory course
        Lecture only course

18a.  Would you like a copy of the results of the survey?   Yes        No

18b.  If yes, please provide an e-mail address:

19.  Please provide any additional comments about your use or perceptions of web-based course management tools

APPENDIX B

Sample Qualitative Comments by Survey Respondents

“CMS were touted as the ‘next best thing' for higher education. I question the return on investment.”

“Our web-based courses have been popular with students that we are now offering entire undergraduate degree programs on-line. We have implemented an on-line internship and are considering putting a few of our graduate programs on-line.”

“Training of faculty in the use of [CMS] and on-line teaching strategies is critical. Also, prerequisites to being eligible to take on-line courses should be strongly considered.”

“The use of web-based course management tools to supplement traditional courses has shown to have some benefits. However, the use of web-based course management tools as the principle or sole mechanism for distance education courses has not proven successful for traditional-aged undergraduates. They tend not to be self-motivated nor self-organized enough to benefit.”

“If you require part-time and adjunct faculty to utilize course management tools, be certain that you provide opportunities for training and professional development when and where it is most convenient for them.”

“Our department used the course management tools in a number of ways – to deliver course content – to develop course content - to communicate with our majors – to communicate with our faculty (both full and part-time) – and to manage some of the “work” of the department (post meeting agendas, meeting notes, important announcements, etc). It is particularly useful if you administer a large department.”

“Students expect that we will make course materials available to them over the web. For some students, it is the only medium though which they'll communicate with faculty.”


Online Journal of Distance Learning Administration, Volume VII, Number IV, Winter 2004
State University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Content