What Online Students Want Compared to What Institutions Expect
Jeffrey L. Bailie
The purpose of this study was to examine whether a set of instructional practices commonly prescribed to online faculty in the higher education setting were consistent with the expectations of a group of experienced online student participants. Online faculty performance conventions were collected from 20 institutions of higher learning located in the United States. The collective practices yielded three primary domains related to administrative faculty performance expectations in online instruction: Communication, Presence/Engagement, and Timeliness/Responsiveness. Undergraduate participants representing a cross section of colleges and universities in the United States were surveyed to determine their expectations for online faculty as compared to scaled items derived from the lists of participating institutions. The results of this investigation offer practitioners insight into how administrative instructional guidelines relate to the user demands of an informed group of undergraduate online students.
The continued expansion of online learning throughout the world has prompted the education profession to identify effective instructional practices common to the online learning environment. The vast majority of such protocols have been administratively defined, ultimately becoming the basis for widely applied evaluations of instructional performance in the Web-based learning environment. At the same time, ubiquitous delivery creating an expanding array of higher learning options for today's online learners (coupled with the rising costs associated with post-secondary enrollment) foretells a future in which students will become ever more discerning consumers in search of learning options that are more connected to their own individual needs and interests (Allen & Seaman, 2013).
Perceptions of quality are said to be the result of a consumer's comparison of expectations to actual performance (Malik, 2012). O'Neill and Palmer (2004) defined quality in higher education as "the difference between what a student expects to receive and his/her perceptions of actual delivery (p. 42)." While the metaphor "student as consumer" has been a topic of debate in the higher education community for decades, today's institutions of higher learning seem to increasingly recognize the value of being responsive to the constituents they serve by exhibiting more of a focus on meeting the expectations of their student clientele. This is widely evidenced in institutional strategies aimed at improved student retention in what has become a progressively competitive market. After all, retention rates for the majority of institutional types are at their lowest level since mandated gathering of retention data first began in 1983 (ACT, 2009). Accordingly, and consistent with any consumer driven enterprise, student satisfaction with their post-secondary learning experience becomes a pervasive consideration that becomes particularly significant given the enrollment prospects of a growing online learner demographic.
An area of interest that is worthy of further review would include an investigation of whether the instructional guidelines, that online faculty are commonly held accountable to by their administration, correspond with the expectations that an increasingly demanding online learner population has for them. In particular, how do student expectations of their online faculty compare to those performance expectations set forth by the institution? The purpose of this quantitative investigation was to determine the extent to which a set of performance protocols prescribed to online faculty at various higher learning institutions were consistent with the expectations of a group of experienced online student participants seated for this inquiry.
Review of the Literature
For some time, administrators and faculty have recognize that understanding the needs and wants of students (and meeting their expectations) are important attributes to developing environments in which students can learn effectively (Seymour, 1993; Gerdes & Mallinckrodt, 1994). Many contemporary institutions have come to the realization that they have indeed become active participants in a service industry and have, therefore, placed added importance on student satisfaction given the increasing competition for enrollments. After all, student satisfaction has been positively correlated to advancements in student recruitment, persistence, and academic success (DeShields et al., 2005; Helgesen & Nesset, 2007).
One theoretical framework that considers how consumer expectation correlates with satisfaction and retention can be found in the Expectation Confirmation Theory (ECT). ECT contends that consumer expectations, together with perceived performance, lead to post-purchase satisfaction. This effect is mediated through positive or negative disconfirmation between expectations and performance in that, if a product outperforms expectations (referred to as positive disconfirmation) post-purchase satisfaction will result (Spreng et. al. 1996, Oliver, 1980). On the other hand, should a product fall short of expectation (e.g. negative disconfirmation) consumer dissatisfaction is the likely result. Further, ECT holds that people's intention to repurchase a product or service is largely determined by their satisfaction with prior use (Anderson and Sullivan, 1993). In response, higher education administrators have focused more attention on influences that reinforce their efforts to attract, support, and retain students.
The collecting of detail on student satisfaction has become a major factor in the assessment of quality in higher education institutions (Leckey and Neill, 2001) and student appraisals of faculty have become a common standard for producing useful feedback which may, in turn, be used to improve instruction. In their 2008 study that sought to identify common methods for the assessment of teaching effectiveness, Jahangiri, Mucciolo, Choi, & Spielman concluded that student evaluations encompass a preponderance of the feedback regarding instruction collected in the higher learning setting. The process of gathering student opinion about their satisfaction with teaching practices related to their learning experience, analyzing and interpreting this information, and then responding to the results are considered to be significant for several reasons (Rahman, 2006). Not only can instructors gain an appreciation for how others interpret their teaching methods, thereby offering feedback that might be deemed useful in improving their instructional delivery, but the information can be used by administration to make formative recommendations (e.g., identifying areas for improvement) and summative decisions (e.g., decisions about promotion) (Dunegan & Hrivnak, 2003). The use of student evaluations also affords students an important opportunity to effectively contribute to the teaching-learning process by outlining pre-consumption expectation and post-consumption disconfirmation, as defined by the ECT framework.
Although student evaluations tend to encompass a significant measure of how faculty effectiveness is viewed, their validity has been nonetheless challenged as a useful method for evaluating teaching excellence. While the extent to which students should be viewed as consumers has been (and will continue to be) questioned by some, student ratings do provide an important perspective that can be used for inclined faculty to reflect on their approach to instruction (Greenwald, 1997). Of the Sloan Consortium's "Five Pillars of Quality Online Education," student satisfaction is regarded as being a key to the decision to continue learning (Sloan, n.d.). The literature offers evidence that student satisfaction is positively correlated to retention and an individual's decision to take additional courses (Booker & Rebmon, 2005). A 2011 study conducted by Harris, Larrier, & Castano found that when student expectations are consistent with their learning experiences, they are more likely to persevere in online learning.
Yet in something of a paradox, while it might be suggested that learner satisfaction is held the highest regard, very few investigations have examined the degree to which student expectations of online instruction are being met. Harris, et. al. (2011) concluded that only a limited number of formal investigations have examined how student expectations of faculty performance influence satisfaction, retention, and persistence in online learning. Joseph et al. (2005) reported that investigations of quality in higher education have disproportionally relied on the views of academic administration, while apparently overlooking direct input from students. Their findings were that "decisions about what constitutes the quality of service (e.g. such as deciding what is 'most important' to students) are exclusively in the hands of administrators and/or academics" (p. 67). The authors contend that "traditional approaches have left decisions about what constitutes quality of service (e.g. such as deciding what is most important to students) exclusively in the hands of administrators and/or academics."They go on to suggest that academic administrators "focus more on understanding the needs of their students, who are the specific and primary target audience" (p.67).
Other investigations have established that the majority of student evaluations of teaching are developed based on faculty and administrators' knowledge and experience, supplemented by review of previous research (Marsh, 2007), but excludes students' input (Ory & Ryan, 2001). Oldfield and Baron (2000, p. 86) maintain that "there is an inclination to view service quality in higher education from an organizational perspective". They suggest that institutions of higher learning pay more attention to what their students want instead of collecting "data based upon what the institution perceives its students find important." Highlighting the necessity of academic institutions to consider the practices of businesses in the area of customer service, and applying it in the context of higher education, Sines and Duckworth (1994) concluded that "it's time for educational institutions to face two facts: they are in a competitive battle for students, and students are customers (p. 14)".
Having received the authorization to proceed with the study from the Institutional Review Boards (IRB) of the participating institutions from which the participant sample would be drawn, the investigator sought to seat a heterogeneous panel of experienced online students to assist in the determination of the how the expectations commonly placed on online faculty compared to the aggregated list of protocols collected for this investigation. Given the investigative design relied on the involvement of informed participants, a requirement for a minimum equivalency of five successful online course enrollments constituted an eligible candidacy for this investigation. Administrators asked faculty to voluntarily post an announcement describing the study in their online courses, and this announcement served as an invitation to students reflective of the designated qualification profile to participate. Enrollment in the study was closed when a sample of 62 participants representing a group of consenting adult online students meeting the qualification profile had been achieved.
This study employed a quantitative, non-experimental, exploratory research design. The research was non-experimental because participants were not randomly assigned. Instead, a purposive sampling scheme was employed in the recruitment of informed participants for this study. This sampling technique is commonly applied when the investigator is interested in specific seating a qualification profile of the participants (Trochim, 2007).
For this study, lists of online instructional protocols were solicited from 40 institutions of higher learning located in the United States, 20 of which responded to the request. The items included in each of the submissions were open-coded and similar codes were clustered together, recurring themes were linked, and the emergent domains were established. Three domains related to administrative faculty performance expectations in online instruction were formed: Communication, Presence/Engagement, and Timeliness/Responsiveness. Subcategories of each domain were formed when they were referenced by at least ten of the institutions. The ranges of expectations within each subcategory were then arranged, forming a scale according to intensity.
The construct of the thirteen item instrument presented the instructional requirements gleaned from the twenty institutions included in this investigation. The instrument adhered to a standard closed item response survey format, and was created using a Web-based application that has been widely acknowledged as offering a valid and secure method for data collection. Qualified participants were provided with an Internet address where they received the survey.
All of the qualified participants successfully responded to each of the 13 items in the two week timeframe allotted, and the survey was subsequently closed. Of the 62 respondents, 41 were female. The age range was 19 through 42, with the majority of participants (n = 35) ranging in age from 21 through 28. Forty percent of the participants were enrolled in Associate degree programs, and sixty percent were pursuing their bachelor's degree. The mean online course completion of the sample was nine courses.
Table 1 presents how the participants responded when queried about the expectations they have for online faculty when it comes to communication. The results indicate that the majority of the participants did expect contact from the instructor before the start of the term, with most favoring the day prior to (50%), closely followed by those expecting contact the week prior to the start of the term (40%). Student participants were dissimilar when it came to their expectations for individual telephone contact prior to the start of the term, with the majority (65%) suggesting that welcome calls should only be placed when requested by the online student. The students' surveyed overwhelmingly expected that new units of instruction would be prefaced with an announcement describing assignments and reiterating due dates, either the day of (45%) or the day prior to (55%) the start of the unit, and the majority (55%) felt that online faculty should only be expected to post pictures of themselves at their own discretion.
Institutional Online Instructional Expectations
Online Instructional Protocols: Communication
Online faculty should be expected to initiate email contact with each enrolled student:
One month prior to the term start
One week prior to the term start
One day prior to term start
Should not be expected
Online faculty should be expected to place a welcome telephone call to each student:
The week before the course begins
During the first week of the course
Only when requested by student
Online faculty should be expected to preface new units of instruction with an announcement delineating learning objectives and due dates:
A day prior to the start of the new unit
The day of the start of the new unit
Should not be an expectation
Online faculty should be expected to include personal imagery in their welcome messages:
For each course they teach
As they deem appropriate
Only when students are expected to do so
With regard to student expectations for online faculty presence and engagement, participants expected online faculty to access their course every day of the week (55%) and participate in discussions daily (45%) or two to three times during the week (55%), but engage in a manner that is consistent with the activity (43%). Finally, the majority of online students surveyed expect their faculty to maintain office hours. These results are presented in Table 2.
Institutional Online Instructional Expectations
Online Instructional Protocols: Presence & Engagement
Online faculty should be expected to access their course at a frequency of not less than:
Once a day, seven days a week
Once a day, except weekends
2 – 3 x per week
At least once a week
Online faculty should be expected to actively participate in discussions:
At least one day during the week
2 – 3 days per week
On a daily basis
Online faculty should be expected to engage in discussions:
Only when directly asked a question
With at least half of the class each week
With each student during each unit
Online faculty should be expected to maintain office hours:
Each business day
As deemed appropriate
Table 3 reveals the survey results of online student participants relative to their expectations for faculty timeliness and responsiveness. More than half of the participants (52%) expected online faculty to respond to email inquiries from students within 12 hours of receipt, and to voicemail messages within 12 hours (40%) to 24 hours (42%). Fifty seven percent of the students surveyed expected faculty to return a minor graded assignment within three days, and a major assignment within one week (63%).
Institutional Online Instructional Expectations
Online Instructional Protocols: Timeliness/Responsiveness
Online faculty should be expected to respond to email inquiries from students:
Within 72 hours of receipt
Within 48 hours of receipt
Within 24 hours of receipt
Within 12 hours of receipt
Online faculty should be expected to respond to student voicemail inquiries:
Within 72 hours of receipt
Within 48 hours of receipt
Within 24 hours of receipt
Within 12 hours of receipt
Online faculty should be expected to return a graded "minor" assignment
(discussions, 3-5 page papers):
"In a timely fashion"
Within two weeks
Within one week
Within three days
Online faculty should be expected to return a graded "major" assignment
(final project, 6+ page papers, team projects):
"In a timely fashion"
Within two weeks
Within one week
Within three days
As with any research endeavor, limitations will exist. The data for this study were collected over a span of 2 months in the Fall of 2012. Only 62 participants completed the data collection forms for this investigation. Additionally, the online instructional protocols of only 20 institutions were included in the survey. Accordingly, the results may not be generalized to the larger population. Finally, because the survey instruments used in this study required participants to self-report their perceptions, the results can only be considered valid to the extent that the subjects truthfully reported their own perceptions.
Implications and Recommendations
This survey study examined the extent to which performance protocols routinely assigned to online faculty in the post-secondary setting correlated with the expectations of a group of experienced online student participants. The investigators role in this study was not to add to the age old debate of whether students should be classified as customers. Instead, reflective of the rudimentary premise of the Expectancy Confirmation Theory which suggests that students enroll in the higher learning experience with certain expectations, and when they believe that their goals have been satisfied, and they persist.
It should come as no surprise that the vast majority of expressed protocols that govern delivery of Web-based courses at the post-secondary level have been administratively defined, simply because the nature of these rules are an extension of more widely applied evaluations of instructional performance. First devised as basic practices when online learning began to emerge, the protocols were more widely recognized and publicized as best practices in the literature which then, in turn, resulted in instructional expectations for many institutions. After all, online learning practitioners new to the field had to have a set of rules to operate by - how would we had known, for example, how often a faculty member should demonstrate a presence in their online classroom? Albeit perhaps difficult to fathom, there was a time when academic institutions were far less prescriptive regarding an educator's approach to online instructional delivery. Still, some might find the relatively wide variance of protocols between the twenty institutions included in this investigation to also be of interest, perhaps signifying that administrative expectations for online faculty (at least in the areas of communication, presence & engagement, and timeliness & responsiveness) are still very much institutionally determined.
Online learning now operates in an environment that a demanding and increasingly sophisticated consumer base is pursuing. With an ever-expanding array of available online learning opportunities, students are increasingly aware of what they are seeking in their online experience – perhaps now more than ever before – and learners will certainly be attracted to those options where they find the greatest satisfaction. Based on the findings of this investigation, practitioners should consider the expectations derived from the sample population participating in this endeavor as they contemplate their own institution's practices (as well as how those practices were devised).
Based on the survey responses, participants favored an email message being sent by the faculty member within one week prior to the start of the term. Beyond the courtesy of a welcome message, the benefit of such a correspondence could be in introducing the student to the learning experience by offering a syllabus, determining whether texts and other related materials have been accounted for, and providing contact information. According to the students participating in this study, and contrary to the requirements of some institutions, a telephone call to each enrolled student is not expected unless specifically requested by the student. Also, personal imagery of the instructor should be included as the faculty member deems appropriate.
Once the term is underway, respondents preferred that each new unit of instruction proceed with an announcement the day prior to opening the new module. Such an announcement might serve as an introduction that highlights the relevance of the goals of the unit, accentuates the relation between activities and assignments to the learning objectives, and offer strategies for efficient completion of the stated outcomes.
Presence & Engagement Expectations
Active involvement by the faculty member was upheld as an important expectation by the students participating in this study. The majority of the students surveyed support the expectation that faculty should access their online course at least once per day, seven days of the week. In addition, they should participate in discussions at least two to three days during the week, but this participation should be consistent with the activity. The group felt that regular office hours should not be required of faculty, instead favoring that faculty be available by appointment.
As the range of protocols collected from the institutions are deliberated, this is an area where considerable variance between institutions remains. After all, given the ubiquitous nature of today's technology, it would seem that an institutional expectation that faculty only access their online course a minimum of one time per week would be objectionable. It was interesting to note that, according to the prevailing view of the sample seated for this study, faculty participation in discussions need only be consistent with the 2-3 times per week engagement (which is consistent with the commonly prescribed expectation for student performance).
Of the three protocol areas included in this investigation, the most robust dialog received from colleagues attending conference presentations where the results of this investigation were reported came in the area of timeliness and responsiveness. Clearly, this is an area where the unsettled argument regarding student as consumer is amply revealed.
Students participating in this study indicated that faculty should respond to an email inquiry from a student within 12 hours of receipt, and a voicemail within 12 – 24 hours of receipt. Participants in this study also favored a response time of three days for the return of a minor assignment and one week for a major assignment. Again, it was interesting to note that (at least) one institution found "in a timely fashion" to be an acceptable measure. In the view of this investigator, this expectation is derived by advancements in technology that have provided greater connectivity by way of cell phones and electronic messaging. Coupled with the common marketing scheme of "anytime, anyplace" learning coined by the online learning industry, it would appear that online students desire a more instantaneous response than previously expected.
In general, the results of this investigation support the belief that for online faculty to meet the expectations of their students, they must demonstrate a timely and dependable presence their online courses. They must also communicate often with students through consistent feedback, widely engage in and promote opportunities for discourse, and be responsive to occasions for contact.
As practitioners, when we place ourselves in the role of a student, would it not be reasonable to assume that such qualities would also match our own expectations? Perhaps it would be beneficial for responsive intuitions to validate the expectations of their online students in comparison to the instructional protocols they have in place.
ACT. (2009, January 22). Fewer students returning to same school for second year of college. Retrieved from http://www.act.org/newsroom/releases/view.php?lang=english&p=731
Allen, I. E. & Seaman, J. (2013). Changing Course: Ten Years of Tracking Online Education in the United States. Wellesley MA: Babson College/Quahog Research Group
Anderson, E.W. & Sullivan, M.W. (1993). The antecedents and consequences of customer satisfaction. Marketing Science. 12(Spring), 125-143.
Booker, Q.E., & Rebman, C.E. (2005). E-Student Retention: Factors Affecting Customer Loyalty for Online Program Success. Issues in Information Systems, 1(1), 183-189.
DeShields, O. W., Kara, A., & Kaynak, E. (2005). Determinants of business student satisfaction and retention in higher education: Applying Herzberg's two-factor theory. International Journal of Educational Management, 19(2), 128-139.
Dunegan, K. J., & Hrivnak, M. W. (2003). Characteristics of mindless teaching evaluations and the moderating effects of image compatibility. Journal of Management Education, 27(3), 280–303.
Elliot, K. M., & Healy M. A. (2001). Key factors influencing student satisfaction related to recruitment and retention. Journal of Marketing for Higher Education, 10, 1-11.
Gerdes, H. & Mallinckrodt, B. (1994). Emotional, social, and academic adjustment of college students: a longitudinal study of retention. Journal of Counseling and Development, 72(1), 281-288.
Greenwald, A. G. (1997). "Validity concerns and usefulness of student ratings of instruction." American Psychologist, 52(11), 1182-1186.
Harris, S. M., Larrier, Y. I., & Castano-Bishop, M. (2011). Development of the student expectations of online learning survey (SEOLS): A pilot study. Online Journal of Distance Learning Administration, 19(5). Retrieved from https://www.westga.edu/~distance/ojdla
Helgesen, O. & Nesset, E. (2007). What accounts for students' loyalty? Some field study evidence. International Journal of Educational Management, 21(2), 126 – 143.
Heilman, J. D. & Armentrout, W. D. (1936). The rating of college teachers on ten traits by their students. Journal of Educational Psychology, 27, 197-216.
Jahangiri L., Mucciolo T., Choi M. & Spielman A. (2008). Assessment of teaching effectiveness in U.S. dental schools and the value of triangulation. Journal of Dental Education. 72, 707-718.
Joseph, M., Yakhou, M. & Stone, G. (2005). An educational institution's quest for service quality: customers' perspective. Quality Assurance in Education, 13 (1), 66-82.
Leckey, J. & Neill, N. (2001). Quantifying quality: The importance of student feedback. Quality in Higher Education, 7(1), 19–32.
Malik, U. S. (2012). Customer Satisfaction, Perceived Service Quality and Mediating Role of Perceived Value. International Journal of Marketing Studies, 4(1).
Marsh, H. W. (2007). Students' evaluation of teaching: Dimensionality, reliability, validity, potential biases and usefulness. In R. P. Perry and J. C. Smart (eds). The Scholarship of Teaching and Learning in Higher Education: An Evidence-Based Perspective, 319-383.
Oliver R. L, (1980). A cognitive model of the antecedents and consequences of satisfaction decisions. Journal of Marketing Research. 17(3), p. 460.
O'Neil, M. & Palmer, A. (2004). Importance-performance analysis: a useful tool for directing continuous quality improvements in higher education. Quality Assurance in Education, 12(1), 39-52.
Ory, J. C. & Ryan, K. (2001). How do student ratings measure up to a new validity framework? New Directions for Institutional Research, 109, 27–44.
Rahman, K. (2006). Learning from your business lectures: using stepwise regression to understand course evaluation data. Journal of American Academy of Business, Cambridge, 19(2), 272–279.
Seymour, D. T. (1993). On Q: Causing quality in higher education. Phoenix, AZ; Oryx.
Sines, R. G. Jr, Duckworth, E. A. (1994). Customer service in higher education. Journal of Marketing for Higher Education, 5(2), 1-15.
Sloan (n.d.). The 5 Pillars of Quality Online Education. The Sloan Consortium. Retrieved from: http://sloanconsortium.org/5pillars
Spreng R. A, S.B. MacKenzie and R.W. Olshavsky, (1996). A reexamination of the determinants of consumer satisfaction. Journal of Marketing 60(3), p. 15.
Trochim, W. M. K, & Donnelly, J. P. (2007). Research methods knowledge base (3rd ed.). Macon, OH; Thomson.
Online Journal of Distance Learning Administration, Volume XVII, Number II, Summer 2014
University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Contents