A Faculty Evaluation Model for Online Instructors: Mentoring and Evaluation in the Online Classroom


B. Jean Mandernach,
Department of Psychology,
Park University
Parkville , Missouri
jean.mandernach@park.edu

Emily Donnelli, Department of English, Park University

Amber Dailey, Department of Education, Park University

Marthann Schulte, Department of Education, Park University


Abstract

The rapid growth of online learning has mandated the development of faculty evaluation models geared specifically toward the unique demands of the online classroom. With a foundation in the best practices of online learning, adapted to meet the dynamics of a growing online program, the Online Instructor Evaluation System created at Park University serves the dual purpose of mentoring and faculty evaluation. As such, the model contains two distinct phases of interaction: formative reviews and a summative evaluation. Beyond its critical role in instructor retention, program assessment, and accreditation, this faculty evaluation system signals the University's commitment to ongoing professional development. The Online Instructor Evaluation System maximizes the potential of faculty evaluation to inspire reflection and growth; encourages the persistent professional development needs of online instructors; emphasizes the process of teaching as well as product; incorporates multiple perspectives to capture a comprehensive view of instructor performance; and educates key on-ground university constituents about online learning.

In the infancy of online instruction, considerable emphasis was given to demonstrating equivalence between online and traditional face-to-face instruction. This movement extended from pedagogy to evaluation as many online programs mirrored established face-to-face processes for faculty evaluation when creating models for the virtual classroom. With the rapid growth of online learning, these early evaluation models have revealed limited relevance to the online environment both in content and implementation. To address the ineffectiveness of traditional faculty evaluation models for use with online faculty, as well as to contribute to the growth of online learning as a field (and not simply a practice), innovative faculty evaluation models that are geared specifically to the unique demands, expectations and requirements of modern online learning must be developed.

Institutional Context

The evaluation model for online faculty at Park University was created to meet the unique demands of an evolving online program. While Park University was founded as a small, private liberal-arts college in 1875, the original campus has grown to include graduate programs, 42 nation-wide campus centers, and an extensive online program supporting 45,000 annual student enrollments in seven online degree-completion programs and four fully-online graduate programs. Park University 's culture is as a teaching-oriented institution, with emerging expectations for faculty scholarship, research, and service. The institutional complexity at Park University samples challenges found across a host of institutions targeting 2- or 4-year degrees, public or private settings, and traditional or adult student populations. As such, the University's online faculty evaluation model is potentially translatable to an equally wide range of higher-learning institutions. With the increasing popularity and growth of online learning, it is essential to establish clear, direct, relevant guidelines for evaluating online faculty that maintain instructional quality and promote best practices in online education.

Rationale for Online Faculty Evaluation System

The legacy online faculty evaluation system at Park University was inadequate for evaluating the unique expectations and demands faced by online instructors within this system. Prior to the development of the new online evaluation system, online instructors were evaluated using a generic instrument designed for face-to-face distance instructors at the University's campus centers across the country. Because the evaluation was based on traditional classroom concerns, the instrument's evaluative criteria did not emphasize key competencies for effective online instruction, such as instructor response rate and availability; frequency and quality of presence in online classroom; facilitation of discussions in writing; usability of instructor-created supplemental content; and overall management of the administrative aspects of the course (approving final examination proctors, connecting students to University support resources, completing required administrative tasks, etc.).

Also in keeping with traditional face-to-face evaluation models, the evaluator completed his/her review at the end of the term in a singular exchange with the instructor. While this type of singular formal interaction may be effective in the more intimate environment of the on-ground campus, where evaluators have ongoing opportunities to interact informally with instructors based on proximity, this practice does not translate meaningfully to the virtual classroom. In the online environment, the evaluation process often proved the first and only time the instructor and evaluator interacted. The limitations of this model were compounded by the fact that many online instructors were relatively new to the virtual classroom. Not only were the evaluation and recommendations for these instructors' online teaching practice being provided after the point at which suggestions could be effectively implemented into the classroom, the legacy evaluation instrument did not account for the limited online teaching experience of many of the University's online instructors. Thus, the assumptions under girding face-to-face faculty evaluation models limited the effectiveness of an online evaluation system based on traditional models.

In addition to the flaws in the timing and content of the legacy evaluation system, there were growing concerns about the lack of peer support and professional guidance for instructors working within a geographically diverse and highly individualized environment. Larger University concerns present included the need to educate the key departmental leaders who were expected to work with online learning staff to evaluate online instructors. The legacy evaluation system did not provide these individuals, only some of whom had experience with online instruction, with the context and education needed to understand the differences between online and face-to-face methods. As such, the Online Instructor Evaluation System (OIES) was developed to reinforce key competencies and expectations unique to online faculty; provide significant professional development exchanges and resources for online faculty; and ensure a high-quality learning experience in all virtual classrooms. Moving outward, a tertiary goal of the OIES was to educate the broader University community about benchmarks and best practices of online learning to promote reciprocal exchange between online and face-to-face practitioners and practices.

Theoretical and Institutional Foundations

When evaluating the effectiveness of an online course, two distinct components are under review: the content (curriculum) provided within the online course and the instructor's facilitation of the learning experience. At Park University , the core content of each course is created by a content-expert. Once the core content is reviewed and approved by the relevant department chair, all instructors teaching that course utilize the same core content. In addition to ensuring compliance with University-wide learning outcomes and promoting academic equivalence and curricular coherence between delivery methods, this type of standardized curriculum allows the evaluation of online faculty to emphasize the facilitation and instructional skills of faculty members rather than their ability as instructional designers. This curriculum development model also acknowledges the realities of developing and facilitating content within an accelerated format by providing instructors with a full course of instructional materials they can then supplement as desired to reflect their expertise and to suit learner needs. Since Park University utilizes a standardized curriculum, there is less emphasis during evaluation on the content of a course. As such, the Online Instructor Evaluation System (OIES) was designed to focus exclusively on online pedagogy while relying on departmental oversight to ensure content quality.

The OIES was developed out of a comprehensive review of the literature on benchmarks and best practices of online pedagogy (for more detailed information on these standards, see Council for Regional Accrediting Commissions, 2001; Graham, Cagiltay, Lim, Craner, & Duffy, 2000; Luck, 2001; Fitch & Montambeau, 2000; Reeves, 1997; Romans, n.d.; and Weiss, Knowlton & Speck, 2004) to achieve the goals of educating online instructors concerning accepted standards in online education and holding them accountable to these best practices through evaluation and professional development. Based upon Chickering and Ehrman's guidelines for integrating principles of good practice in undergraduate education (1996) into technology-enabled learning environments, the OIES emphasized reciprocal student-faculty contact, inclusion of active learning strategies, prompt feedback, promotion of student time-on-task, clear communication of high expectations, and respect for diversity in student talents and ways of learning.

It is important to note that the OIES was not founded solely on the literature and research findings. Rather, these generalized best practices in online education were considered and adapted in ways that reflected Park University's institutional history, current context, and future goals, as synthesized in the University's “Online Course Standards and Principles” (Park University, 2004). The development of the OIES occurred simultaneously with the conversion of the online program to a different online course delivery platform, a platform which provided increased instructional resources and thus allowed evaluators to expect more rigorous and varied interaction in the online classroom from instructors. At the same time, the initial training of online instructors was streamlined and condensed into a self-paced, individualized online format, opening up an opportunity to extend the instructor training and orientation processes with an individualized first-term evaluation. All of these programmatic changes were implemented during a paradigm shift in which there was increased emphasis on faculty collaboration, communication and integration across all modes of course delivery. As such, the OIES at Park University was developed in light of institutionally-specific dynamics combined with an empirical basis of online pedagogy. The effectiveness of the OIES is a direct result of this type of tailoring; in order for an evaluation system to be effective, it must have a foundation in generally-accepted practices, but these best practices must be tailored to meet the needs of the particular institution.

Overview of the Online Instructor Evaluation System

The Online Instructor Evaluation System (OIES) serves the dual purpose of mentoring and faculty evaluation. As such, the model contains two distinct phases of interaction: formative reviews and summative evaluation.

The purpose of the formative reviews is to provide feedback and guidance to instructors on critical online learning components essential to successful facilitation of an online course. The formative reviews are “no stakes” evaluations in which the evaluator provides detailed feedback relevant to the instructor's online course as well as suggestions for overall enhancement of the online learning experience. While the formative reviews are shared with the instructor (and instructors are encouraged to use reviews as a starting point for further dialogue), they are not included in the overall course review sent to the instructor's academic department.

During an eight-week course term, the instructor is evaluated with five formative reviews, each focusing on a critical aspect of online instruction. The topics for each review were selected based on the guiding principles outlined in the best practices for online education and were sequenced according to logical use within the course/term. The formative reviews include:

•  Preterm review – The preterm review (see Appendix A) is completed prior to the start of the term and focuses on the set-up of the online course. While completing the Preterm Review, the evaluator ensures introductory items (i.e., course homepage, homepage content items and syllabus) are updated, personalized and reflect University requirements. These set-up activities are especially important in orienting learners in an accelerated format, where students often access their online classrooms prior to the start of the term to obtain textbook ordering information and other prepatory materials. In addition, the evaluator provides targeted suggestions for enhancements related to course organization and the utilization of course tools and features.

•  Review #1 – Review #1 (see Appendix B), which is completed at the end of the second week, examines community building and promotion of an interactive climate. The purpose of this review is to e xamine the use and implementation of discussion threads, including an overview of participation expectations, tips for grading discussion items, instructor availability, instructor presence, and student-to-student interaction.

•  Review #2 – Review #2 (see Appendix C) is completed during the fourth week and focuses on discussion facilitation, feedback and grading. During this review, the evaluator provides guidance on instructor interactions, specifically addressing feedback and participation in the discussion threads as well as comments and use of the grade book. Reviews #1 and #2 both emphasize discussion facilitation and feedback, as meaningful, consistent instructor interaction is an ongoing essential of effective online learning.

•  Review #3 – At the end of week six, Review #3 (see Appendix D) examines assessments and final exam preparation. The evaluator r eviews the implementation of formative and summative assessments as well as preparation for final exam and the proctored exam process. In addition, interactions include an opportunity to share ideas about supplemental formative assessments that may be added to the course. Although instructors are expected to use the core curriculum provided, they are also encouraged to add supplemental content to their courses.

•  Review #4 – Implemented during the final weeks of the term, Review #4 (see Appendix E) focuses on i nstructional materials and overall course climate/organization. As such, evaluators will r eview all supplemental information to ensure adherence to general instructional design principles and specific University curriculum standards and expectations.

These formative assessments provide an avenue for peer-mentoring and professional growth by promoting ongoing reflection and dialogue about practice. While the formative reviews are not explicitly presented as an informative/prescriptive resource, they serve as consistent and concrete mechanisms to educate instructors about best practices and serve as a valuable professional development resource.

In contrast to the low-stakes nature of the formative reviews, the final Summative Evaluation (see Appendix F) is an overall reflection of the course and is used to inform both the instructor and the academic department (who will use the summative evaluation along with the instructor's self-review, Appendix G, and student evaluations to make final decisions concerning instructor retention). While the Summative Evaluation is focused on an overall evaluation of the course, it emphasizes the instructor's ability to incorporate suggested changes and required modifications (including an evaluation of instructor responsiveness and adherence to administrative requirements) from the formative reviews; although the formative reviews, in keeping with their function, are not included in the summative package, the particulars of those reviews are generalized to reflect either the instructor's strengths as a reflective, improved practitioner or to note unresponsiveness and lack of participation in the process. The summative evaluation package is particularly effective as a means of online faculty evaluation as it incorporates multiple perspectives to present to the department a comprehensive portrait of the instructor. This integrative evaluation emphasizes an instructor's growth throughout a term rather than simply highlighting the mistakes made throughout the instructional process.

Implementation

The Online Instructor Evaluation System is implemented by designated instructor evaluators. The instructor evaluator position is a full-time, faculty-classified role, with 50% administrative release time granted to evaluate/mentor approximately 15 instructors per eight-week term.  The faculty classification of the evaluators allows them to be fully integrated into the life of the campus, participating on curriculum and other university-wide committees and meeting institutional expectations for research, scholarship, and service. A dual benefit of this classification is that the online learning program gains faculty representatives in the disciplines, faculty who can serve as ready sources of accurate information about the online learning program, its academic oversight, and the processes related to developing and teaching courses online. The representative/liaison function of the evaluator role enables stronger relationships between the online learning program and the disciplines, paving the way for increased collaboration. Pragmatic implications of structuring the position in these ways include working with university academic administration and candidates' departments in the hiring process to ensure that the candidates possess appropriate credentials and the hires respond to departmental needs in addition to needs within the online learning program. 

Since the inception of the OIES in the fall of 2004, 102 separate faculty evaluations have been completed. In order to ensure that the OIES was an effective evaluation system, responsive to programmatic needs and institutional culture, the initial pilot implementation was intentionally restricted to allow for necessary revisions and modifications. During the first academic year of implementation, the following evaluations were completed:

At the conclusion of the first year of the OIES pilot, extensive reflection on the evaluation system resulted in the creation of a University Review Board comprised of full-time faculty, administrators and adjunct instructors to provide annual feedback and guidance on the faculty evaluation system. As part of the first Review Board process, additional instructor evaluators were hired to meet the demands of the University's pool of approximately 300 active online instructors and to ensure sound evaluation loads. Based on pilot analysis, it was determined that five instructor evaluators were needed to mentor/evaluate 15 faculty members per eight-week term, four terms each academic year. This load would allow each instructor evaluator to complete 60 OIES evaluations in one academic year, resulting in a total of 300 evaluations completed across the University each year. While there exist ongoing budgetary considerations associated with the additional faculty members hired as instructor evaluators, these costs are offset by the economic value of retaining qualified instructors compared to the high-cost of faculty turnover.

Results and Reflections

Like all effective evaluation systems, the OIES is continually monitored and adjusted to adapt to the dynamic nature of higher-education and the emergent quality of online education in particular. The initial implementation of the OIES exposed a number of strengths and weaknesses related both to faculty readiness for and response to the evaluation process and administrative concerns over its implementation. Reflecting on these initial concerns has enabled continued refinement of the evaluation materials and process.

Instructor feedback to the OIES indicated a noticeable contrast between the perceptions and reactions of existing faculty and new instructors. While most new instructors were very receptive to and appreciative of the review process, noting that the additional resources usefully expanded their initial training, existing faculty members displayed some resistance to the evaluative dimension of the system. New instructors indicated an appreciation for the extensive guidance and the way in which the formative reviews provided them with a measure of their performance as facilitators of learning online. Representative of this category of feedback, one new instructor commented, “I love the constructive criticism and since this is my first time teaching online courses, it is greatly appreciated.” Furthermore, new instructors considered the evaluation to be a collaborative process between themselves and the evaluator. One respondent noted, “I really appreciate your mentorship and suggestions this semester as this has facilitated me to become a better instructor for future Park Classes. Again, thank you for your time and guidance over the past 8 weeks!” Overwhelmingly, new instructors valued the mentorship and suggestions for improvements offered through the evaluation process. As one first time instructor echoed, “This being my first course online [for Park University] has been a great experience, learning as I go as well as generating ideas for me on how to make changes in the course materials, supplements, etc.”

In contrast, many of the online instructors with experience in the University's program, while appreciative of the OIES formative feedback, expressed concerns about the evaluative component. The seasoned online instructors' suspicions often resulted in questions about the purpose of the evaluation, how the information would be reported to their academic departments, and the credentials of the instructor evaluators. One instructor suggested the evaluation was viewed by some as a sort of surveillance by stating, “Thank you, it has been a pleasure having you ‘look over my shoulder'; I promise to do better in the future.” Feedback from experienced instructors emphasized the reality that implementing a new process often involves enacting a cultural change. This realization led to the development of a more explicit introduction of the evaluation system in the University's online instructor training course among the development of other communications sent out pre-term to introduce instructors to the evaluation system and to their mentor-evaluator. As the existing instructors became familiar with the review process and participated in the mentoring discussions with the instructor evaluator, they became more receptive to the interactive nature of the formative reviews, recognizing their role in shaping the evaluation of their teaching.

Most important, the evaluation process yielded opportunities for instructor reflection and subsequent revision and improvement of teaching practice. At the broadest level, one instructor wrote, “I have enjoyed this faculty evaluation. You have given me a lot to think about, and some great ideas on how I might approach my teaching style and my classroom.” However, the summative instructor self-review also sought to facilitate more specific reflection through the prompt: “Create a priority list of your top 3 plans for improving your instruction in this course. How will you carry out these plans?” Characteristic of many responses, one instructor noted, “[Based on the feedback this term,] I want to provide additional communication opportunities between students and instructors.  I have added weekly virtual office hours where students can come “chat” with me.  They will be held 2 days a week during specified time periods.  I will continue to send weekly e-mail to my students.  I have asked that they send me an e-mail confirmation by the first week of the term letting me know they received the e-mail and that their [University email] is functioning and they know how to access it.”

In addition to feedback from instructors, the evaluators' reflections on the administrative aspects of the evaluation process provided valuable insight for continual enhancements of the OIES. The most prominent issues related to the OIES administration included the extensive time involved in completing each formative review; managing the schedule of reviews against the accelerated, eight-week format of the online courses; and maintaining consistency in the subjective marginal and end-commentary provided by evaluators to explain the criteria rankings. As additional instructor evaluators were hired, it became important to maintain consistency in the evaluators' interpretation of the evaluative criteria and thus the language used by the evaluators to explain the criteria rankings. The evaluators worked together to create a standardized approach to ensure consistency in criteria interpretation and, more importantly, to allow more time for adding custom, instructor-specific commentary.

Summary and Conclusions

Like many others', the history of Park University 's online learning program is distinguished by rapid growth and limited administrative resources and models. Mirroring established processes and paradigms for faculty evaluation at the University's traditional campuses, online faculty at Park University were evaluated from the program's beginning; however, that evaluation was of limited relevance to the online environment both in content and in implementation. Strategic planning led to identification of program goals and key competencies for online instructors, which were subsequently woven into the training, evaluation, and faculty development components of the online program.

The inclusion of a formative component in the faculty evaluation system allowed the online program to demonstrate an understanding of teaching as dynamic, reflective practice or, as more plainly stated by one of our online instructors, as an “action verb.” The low-stakes formative assessments promoted dialogue and sharing of best practices among instructor and evaluator as peers, with evaluator most prominently occupying the roles of colleague and mentor. The formative piece ensured regular interaction between instructor and evaluator, providing the instructor with easy access to an experienced online instructor for questions about course facilitation or administration.

Critical to instructors' ongoing growth throughout the term, dialogue was not confined to instructor and evaluator. The exchanges between instructor and evaluator regularly broadened to include the core content developer and academic department chair when curricular issues arose. Additionally, the promotion of formative exchanges reinforced the expectation that instructors would respond with flexibility and innovation to more effectively address their students' needs. The dialogic nature of the formative assessments was paralleled in the summative assessment, which also incorporated multiple perspectives (instructor, evaluator, student, departmental administrator) to obtain a comprehensive and balanced assessment of the instructor's performance.

Perhaps the most significant contribution the Online Instructor Evaluation System has to offer other institutions is in reinforcing the potential of faculty evaluation in online learning as a tool for building bridges between online and in-resident administrators. While most online faculty and administrators have experience with face-to-face instruction and evaluation, the converse often is not guaranteed. In writing about the unique constraints of evaluating faculty online, Tobin (2004) identifies as a primary marker of traditional models the singularity of evaluator's interaction with individual faculty member. Ostensibly, a traditional evaluator only needs one visit to a face-to-face classroom because the markers of effective classroom teaching are easily observable to someone who is also teaching on-ground. In contrast, in the online environment, academic administrators, many of whom only teach face-to-face, are tasked with the evaluation of online faculty. Providing these departmental leaders with a detailed packet of information about the online instructor not only enabled more sound decision making, but also served to educate these stakeholders about effective online pedagogy. Specifically, the evaluator's summative evaluation and the instructor's self-reflection forms, in reflecting best practices and program expectations, introduced and reinforced the academic quality goals of the University's online learning program.

Beyond its critical role in instructor retention, program assessment, and accreditation preparation, a faculty evaluation system can signal to faculty an institutional commitment to their ongoing professional development. In order to maximize the potential of faculty evaluation to inspire reflection and growth, innovative systems are needed that not only respond to the persistent professional development needs of instructors but that, more broadly speaking, emphasize process as well as product; incorporate multiple perspectives to capture a more comprehensive view of instructor performance; and serve a dual purpose of educating key on-ground university constituents about online learning.


References

Council of Regional Accrediting Commissions. (2001, March). Best Practices for Electronically Offered Degree and Certificate Programs . Retrieved June 22, 2005 from http://www.wcet.info/resources/accreditation/ .

Chickering, A. & Ehrmann, S. (1996, October). Implementing the seven principles: Technology as lever. American Association for Higher Education Bulletin, 49 (2), 3-6.

Finch, J., & Montambeau, E. (2000). Beyond Bells and Whistles: Affecting Student Learning Through Technology. Retrieved June 22, 2005 from http://www.cofc.edu/bellsandwhistles/# .

Graham, C., Cagiltay, K., Lim, B., Craner, J., & Duffy, T.M. (2000). Seven Principles of Effective Teaching: A Practical Lens for Evaluating Online Courses. Retrieved June 22, 2005 from http://ts.mivu.org/default.asp?show=article&id=839 .

Luck, A. (2001, January/February). Developing Courses for Online Delivery: One Strategy. The Technology Source . Retrieved June 22, 2005 from http://horizon.unc.edu/TS/default.asp?show=article&id=834.

Park University . (2004). Online Course Principles and Standards . Retrieved July 12, 2005 from http://www.park.edu/online/faculty/Best_Practices/principles_and_standards.html .

Reeves, T. (1997). Evaluating What Really Matters in Computer-Based Education . Retrieved June 22, 2005 from http://www.educationau.edu.au/archives/cp/reeves.htm .

Romans, C. (n.d.). Designing a Course for Online Delivery at Metropolitan State College of Denver . Retrieved June 22, 2005 from http://www.mscd.edu/~options/online.

Tobin, T. (2004). Best Practices for Administrative Evaluation of Online Faculty. Online Journal of Distance Learning Administration, 7 (2).

Weiss, R.E., Knowlton, D.S., & Speck, B. W. (Eds.) (2004). Principles of Effective Teaching in the Online Classroom. San Francisco : Jossey-Bass.

 

Appendix A

Park University Online Instructor Evaluation System

Course Preterm Review

 

Focus:

Course Setup & Organization

Areas/Topics Reviewed

•  Syllabus

•  Introduction Page

•  Online Classroom Set-Up

Course:

 

Instructor:

 

Evaluator:

 

 

This checklist and review will help you to ensure compliance with Park University policies and give you feedback on your course preparation and setup. Your assigned evaluator will complete this review and return it to you one week prior to the start of the term.

While you may not modify the basic content of the Introductory Page or Syllabus, you are required to update all information to reflect the current instructor, dates, and policies. The following checklist highlights the items that you must modify or add to your course.

Criteria

No

Yes

Comments and Suggestions

Introductory Page

Welcome announcement

 

 

 

Virtual office and discussion thread

 

 

 

Introductions link and discussion thread (including personal introduction)

 

 

 

Syllabus

Instructor Information

  • Instructor name, phone (time zone), email and fax
  • Term dates

 

 

 

Course Policies:

  • Course policies and procedures updated to reflect your own specific policies; may include methods/type of communication preferred, guidelines for online participation, etc

 

 

 

Grading Policy:

  • Grading policy updated to include personal grading policies; such as late work policy or personal participation policy

 

 

 

Academic Honesty:

  • Inclusion of personal policies (if any) for academic honesty; must be in agreement with Park University policy

 

 

 

Attendance:

  • Inclusion of personal policies for participation (if any); may include expectations for involvement, time investment, etc

 

 

 

Student Resources:

  • Personal additions (if any) to Park University resources

 

 

 

My Bio:

  • Includes your professional interest, background, accomplishments, educational philosophy

 

 

 

Overall:

  • Items ordered according to Park University syllabus template

 

 

 

Online Classroom Set-Up:

Syllabus is available for students one week prior to beginning of term

 

 

 

All information accessible to students updated with correct information one week prior to beginning of term

 

 

 

Information is clear and presented without grammar/spelling errors

 

 

 

Unit dates are set

 

 

 

Announcements have been updated and set to appear at relevant points throughout the term

 

 

 

•  Please provide recommendations and suggestions for smoothly and efficiently teaching this course.

 

Appendix B

Park University Online Instructor Evaluation System

Course Review #1

 

Focus:

Climate and Community Building

Areas/Topics Reviewed

•  Creating an interactive climate

•  Community building

•  Use of discussion threads

•  Course organization and navigation

Course:

 

Instructor:

 

Evaluator:

 

 

The beginning of the term is the best time to set the tone for all class interactions. There are many ways an instructor can create an interactive climate and a community of engaged learners; some of the most effective tools are discussion interactions, announcements and gradebook comments. Discussion threads are the primary tool used in an online course as they allow for the expansion and discussion of key course concepts.

Criteria Needs Attention Acceptable Exemplary Comments and Suggestions
Climate and Community Building

Instructor sets tone for interactions via office, introductions, announcements, discussion postings and gradebook comments

 

 

 

 

Instructor models the tone and quality of interactions expected of students

 

 

 

 

Instructor interacts in the classroom on a regular basis (roughly every other day per Park's instructor attendance policy)

 

 

 

 

Interactions are professional and promote a positive classroom climate

 

 

 

 

Instructor utilizes gradebook to provide information on progress in the course

 

 

 

 

Discussion Thread Interactions

Discussion directions clearly specify the number and type of responses required of students

 

 

 

 

Instructor sets guidelines and expectations for discussion interactions

 

 

 

 

Instructor interactions promote open-ended discussion

 

 

 

 

General grading criteria or grading rubrics are provided

 

 

 

 

Instructor interacts in discussions to promote higher-order thinking (i.e. critical thinking, synthesis, and application of concepts)

 

 

 

 

Instructor uses discussions to highlight key topics, terms, or information

 

 

 

 

Instructor encourages students to express multiple, even divergent, points of view

 

 

 

Organization

Navigational cues are provided to help students figure out where to begin and

how to best move through the course content

 

 

 

 

Instructor communicates expectations in a clear and consistent manner

 

 

 

 

Due dates are clear

 

 

 

 

Assignment directions clearly specify requirements and directions for submission

 

 

 

 

•  What are the instructor's strengths in creating an interactive climate?

•  Please list any suggestions or recommendations for enhancing interaction in the online classroom.

 

Appendix C

Park University Online Instructor Evaluation System

Course Review #2

Focus:

Instruction and Grading

Areas/Topics Reviewed

•  Interaction

•  Discussion facilitation

 

Course:

 

Instructor:

 

Evaluator:

 

 

There are a range of strategies and techniques for promoting learning in the Online classroom. Interactive instruction occurs through the facilitation of discussion threads, gradebook comments, course announcements, and email. By its nature, email conversations are private; thus the following review will focus on interactions visible through the gradebook and discussion threads.

 

Criteria Needs Attention Acceptable Exemplary Comments and Suggestions
Interaction and Discussion Facilitation

Instructor actively and meaningfully participates in discussion on a regular basis throughout the week

 

 

 

 

Instructor encourages students' continued interaction/engagement through the use of questions or comments

 

 

 

 

Instructor keeps discussion relevant and on topic

 

 

 

 

Instructor monitors and ensures that discussions are appropriate and respectful

 

 

 

 

Instructor grades discussion via a private format

 

 

 

 

Instructor promotes and encourages a range of viewpoints in the discussions

 

 

 

 

Instructor's discussion postings are professional, clear, precise and supportive of student learning

 

 

 

 

Instructor uses discussion postings to introduce information and to facilitate acquisition/application of course concepts

 

 

 

 

Instructor acknowledges all questions insofar as possible

 

 

 

 

Instructor provides a general set of criteria (rubrics) for grading discussion participation

 

 

 

 

Instructor uses gradebook comment feature to highlight reason for assigned participation grade and suggests specific ways students can improve

 

 

 

 

Instructor assigns grades that differentiate students' levels of discussion thread participation

 

 

 

 

Instructor assigns participation grades in a timely manner

 

 

 

 

•  List aspects of the instructor's interaction that contribute most to student learning.

•  Possible recommendations to improve instruction and grading.

 

Appendix D

Park University Online Instructor Evaluation System

Course Review #3

Focus:

Implementation of Assessments

Areas/Topics Reviewed

•  Implementation of assessments

•  Feedback

•  Grading

•  Final exam preparation

Course:

 

Instructor:

 

Evaluator:

 

 

While the content of the basic assessments is determined by the course developer, effective learning occurs through students' active engagement with course material. As instructor, your role is to facilitate this type of interaction, provide feedback and incorporate resources to encourage student mastery of course concepts. To maximize student learning, instructors must structure their activities around a grading system that effectively and explicitly measures course/learning goals and provides students with feedback and guidance to reach these goals. In addition, you will be responsible for coordinating the administration of the proctored final exam.

Criteria

Needs Attention

Acceptable

Exemplary

Comments and Suggestions

Implementation of Assessments

Instructor incorporates all assessments specified by the course developer

 

 

 

 

Instructor utilizes assessments in the manner designed by the course developer

 

 

 

 

Instructor clearly communicates assignment guidelines

 

 

 

 

Assignment due dates and submission instructions are clear and provide adequate advanced notice

 

 

 

 

Instructor schedules assignments in a manner amenable to an accelerated course while providing time for thoughtful feedback

 

 

 

 

Instructor utilizes dropbox, journal or doc sharing, rather than email, for submission of private assignments

 

 

 

 

When necessary, instructor includes additional resources to assist students in meeting assignment expectations

 

 

 

 

Feedback and Grading

Instructor clearly explains the grading system or method for assigning points (i.e. the weight of each assignment, the grading scale used, etc.)

 

 

 

 

Specific grading rubrics or general assessment criteria are provided in advance to assist students in completing assignments

 

 

 

 

Instructor utilizes the comment feature of the gradebook to give individual feedback that not only highlights reasons for assigned grade but also suggests strategies for improvement

 

 

 

 

Gradebook comments are clear, respectful and professional

 

 

 

 

Instructor assigns grades that reflects/differentiates the quality of student performance as well as the quantity

 

 

 

 

Instructor maintains a consistent and appropriate definition of “good” performance that reflects the level (100, 200, 300, etc) of the course

 

 

 

 

Instructor uses the gradebook in a timely manner to keep students informed of their progress

 

 

 

 

Final Exam Preparation

 

 

 

 

Instructor directs students to Proctor Request Form when it becomes available (usually Week 2 of the term) and to proctor selection guidelines in the syllabus

 

 

 

 

Instructor reinforces the Park University 's timeline for proctor selection: Students must select a proctor by the end of Week 6

 

 

 

 

Instructor provides general information concerning the nature and format of the final exam

 

 

 

 

Instructor provides resources to help prepare students for the final exam

 

 

 

 

•  List strengths of the instructor in facilitating course assessments.

•  Provide recommendations for enhancing the implementation and facilitation of course assessments.


Appendix E

Park University Online Instructor Evaluation System

Course Review #4

Focus:

Course Climate & Supplemental Materials

Areas/Topics Reviewed

•  Classroom climate

•  Rapport

•  Organization

•  Supplemental instructional materials

Course:

 

Instructor:

 

Evaluator:

 

 

In an Online course, instructors are unable to rely on visual cues or nonverbal behaviors to guide interactions; thus it is vital that the environment of the online classroom encourages is well-organized, and encourages thoughtful and respectful participation. Some instructors enhance the course climate by adding their own materials to the core content provided by the developer. These supplemental content items reflect the instructor's unique background and experience and allow the instructor to add his/her expertise to the course content. If you elect to supplement the content provided by the developer, it is important to ensure that all additional resources are relevant and follow principles of good Online course design.

Criteria Needs Attention Acceptable Exemplary Comments and Suggestions

Classroom Climate

Instructor is respectful and fair

 

 

 

 

Instructor appears enthusiastic about course material and modeled active learning

 

 

 

 

Off-topic or disrespectful interactions are quickly addressed

 

 

 

 

Instructor's responses to student postings are positive and encouraging

 

 

 

 

Classroom atmosphere is inviting and non-threatening

 

 

 

 

Organization

Expectations and flow of course activities are easy to understand

 

 

 

 

Guidelines are consistently enforced

 

 

 

 

Classroom activities are clear and relevant

 

 

 

 

Instructor is prepared (threads and assignments are posted in a timely manner)

 

 

 

 

Course is conducted according to designated schedule, with any deviations being communicated in advance to students via announcements or other course tool.=

 

 

 

 

Additional Readings and Resources (evaluated only if instructor has added their own supplemental content)

Appropriate length & difficulty level

 

 

 

 

Integrates well with other course material

 

 

 

 

Current and up-to-date

 

 

 

 

Links are updated and in working order

 

 

 

 

Students are given guidance on how to effectively use resources

 

 

 

 

Clearly relate to course and learning objectives

 

 

 

 

Clear and easy to understand

 

 

 

 

Interesting and reflect instructor personal comments/expertise

 

 

 

 

When possible, target a range of learning styles (visual aids, auditory)

 

 

 

 

Appropriately references outside sources

 

 

 

 

•  What do you consider to be the strengths of this professor?

•  Evaluate, from your standpoint, the performance of this professor in relation to this specific course. Do you have suggestions on how this course may have been handled more effectively?


Appendix F

Park University Online Instructor Evaluation System

Summative Evaluation

 

Instructor:

 

Term:

 

Course:

 

Course Number/Section:

 

Reviewer:

 

Reviewer Email:

 

 

The Summative Online Instructor Evaluation is a summary feedback form that reflects the instructor's facilitation of the course content and interaction with/assessment of students as of the end of the term. Instructors also received formative feedback throughout the term from their Online Instructor Evaluator; this review also takes into account any modifications or adjustments made by the instructor as a result of those weekly evaluations.

Criteria Needs Attention Acceptable Exemplary Comments, Concerns, and Suggestions
Course Organization

Instructor was effective in reinforcing the core learning outcomes.

 

 

 

 

Instructor conducted the course according to the expectations and schedule presented in the syllabus.

 

 

 

 

Instructor was consistently well-prepared and organized.

 

 

 

 

The course is conducted that students knew on a daily/weekly basis what was expected of them.

 

 

 

 

Instruction

Instructor was able to explain concepts clearly and effectively.

 

 

 

 

Instructor stressed important points in information resources (lectures, discussion, etc).

 

 

 

 

Instructor displayed his/her knowledge of the subject matter.

 

 

 

 

Instructor strengthened students' understanding of course concepts through various interactions (discussion, gradebook, feedback, etc).

 

 

 

 

Interaction and Discussion

Instructor participated actively in class discussions.

 

 

 

 

Instructor communicated clearly and meaningful in course discussions.

 

 

 

 

Instructor effectively lead Online discussions, synthesizing student posts and stimulating ongoing discussion.

 

 

 

 

Instructor was responsive to student questions.

 

 

 

 

Assessments, Grading and Feedback

Instructor was clear and specific in assignment directions and evaluation criteria.

 

 

 

 

Instructor provided helpful, individualized, constructive feedback on all assignments: correcting errors, highlighting strengths, and providing suggestions for improvement.

 

 

 

 

Grading and feedback completed in a timely manner.

 

 

 

 

Instructor kept students informed of their progress in the course.

 

 

 

 

Overall course grades accurately represented students' mastery of course objectives.

 

 

 

 

Classroom Climate

Instructor maintained a positive atmosphere in the Online classroom.

 

 

 

 

Instructor was sensitive to student difficulty with course work.

 

 

 

 

Instructor was easy to communicate with and available for consultation.

 

 

 

 

Professional Engagement

 

 

 

 

Instructor was responsive to Online Course Evaluator's formative feedback and enhanced his/her course facilitation/assessment throughout the term.

 

 

 

 

 

Overall Recommendation:

Do not retain instructor

Retain with contingency

Retain instructor

 

 

 

 

General Comments:

 

Appendix G

Park University Online Instructor Evaluation System

Instructor Self- Review

 

Instructor:

 

Course Title:

 

Term:

 

Course Number:

 

Section:

 

 

The Self-Review is a detailed feedback form completed by each instructor to provide reflective feedback on their experience with a course. The purpose of this self-review is to help you pinpoint your strengths and weaknesses as an instructor; thus, it is important that you honestly and carefully critique your own performance. This form will be included in your overall packet to be sent to the academic department. If you have comments to accompany any of the opinion statements below, please include them with your summative statements at the end of the document.

 

OPINION STATEMENTS

Needs

Improvement

Satisfactory

Strength

Course Organization

•  I was effective in reinforcing the learning outcomes of the course.

 

 

 

•  I explained all activities and assessments clearly.

 

 

 

•  I conducted the course according to the expectations outlined in the syllabus.

 

 

 

•  Work requirements, grading system, and feedback schedule were clear and consistently followed from the beginning of the course.

 

 

 

•  I was consistently well-prepared and organized for class.

 

 

 

•  My additions to the core course content were clearly related to the core learning outcomes and integrated logically with the rest of the course content.

 

 

 

•  The course was so conducted that students know on a daily/weekly basis what is expected of them.

 

 

 

Instruction

•  I made good use of examples and illustrations.

 

 

 

•  I reinforced important points in information resources (in lectures, discussion, etc).

 

 

 

•  I was able to explain concepts clearly and effectively.

 

 

 

•  I stimulated students to think critically about the subject matter.

 

 

 

•  I broadened students' understanding and ability to apply the subject matter.

 

 

 

•  I frequently discussed recent developments related to the subject matter.

 

 

 

Interaction and Discussion

•  I participated actively in class discussions and responded to all student inquiries (in the course or via email) within the 48-hr period expected of Park Online instructors.

 

 

 

•  I communicated clearly via email and discussion postings, modeling the types of interactions expected in an Online course.

 

 

 

•  I was able to lead Online discussions, synthesizing student posts and encouraging continued dialogue.

 

 

 

Assessments, Grading and Feedback

•  Directions for course activities were clear and specific.

 

 

 

•  I provided helpful, timely instruction on all activities and assessments, helping students prepare to successfully complete assignments.

 

 

 

•  My evaluation of students' work was constructive, focused on continued learning, and reflective of the assessment expectations/criteria communicated.

 

 

 

•  Exams and assignments were returned promptly so that learning was reinforced.

 

 

 

•  I acknowledged all questions in a timely manner.

 

 

 

•  I gave individual feedback to students in the class.

 

 

 

•  I suggested specific ways students could improve.

 

 

 

•  I identified strengths in student work.

 

 

 

•  I kept students informed of their progress via the gradebook an in email exchanges (if necessary).

 

 

 

•  The grading system was clearly explained in the syllabus and rubrics (or general grading criteria) were provided.

 

 

 

•  Students' grades accurately represented their performance in the course.

 

 

 

Classroom Climate

•  I maintained a positive atmosphere in the Online classroom by interacting regularly and meaningfully.

 

 

 

•  I was sensitive to student difficulty with course work, providing extra attention and resources when appropriate.

 

 

 

•  I was easy to communicate with and available for consultation.

 

 

 

•  I provided individual help when students needed it.

 

 

 

Instructors are one of the most valuable sources for information about the quality of the course curriculum and effectiveness of instructor training and support. As such the following questions will be used to provide insight on curriculum enhancement and creation of ongoing professional development opportunities. Please use these questions as a vehicle to critically examine your experiences in these areas.

Please consider these follow-up questions carefully:

•  Do you have suggestions about how to improve the content or presentation of the core course curriculum (provided by the developer)?
•  Describe one or two aspects of your instruction this term (interaction with students, facilitation of weekly threaded discussions, supplemental content additions, etc) that you feel are reflective of your strengths as an Online instructor.
•  Create a priority list of your plans for improving your facilitation of this course and encouraging your overall professional growth as an Online instructor. How will you carry out these plans?

 

Authors' Note: The authors would like to acknowledge and thank Ms. Linda Passamaneck for her contributions to the theoretical development of earlier versions of the faculty evaluation model.


Online Journal of Distance Learning Administration, Volume VIII, Number III, Fall 2005
State University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Content