Can SPOC (Self- Paced Online Course) Live Long and Prosper? A Comparison Study of a New Species of Online Course Delivery
Clayton State University
Clayton State University
Clayton State University
Numerous formats exist for online course delivery: pure online, blended or hybrid, flipped and web-enhanced. The literature is replete with comparison studies on the efficacy of online, hybrid and traditional format courses. However, the self-paced online course, a relatively new and rare variation, has received very little coverage in the body of research on this topic. This study examines the components of a self-paced online course specifically designed to incorporate web-based pedagogy to create an engaging and dynamic learning environment. It compares student performance in a self-paced online course, a conventional online course and a traditional in-class course and reveals the potential for students to thrive in a wide variety of online course formats. This study provides useful information to administrators exploring online programming options and online instructors seeking to improve student performance.
Distance education has become a mainstream academic component of traditional institutions of higher education. (Schifter, 2004). The vast majority of institutions report that online education is a critical part of their long-term strategic plans. (Allen & Seaman, 2014). Online education offers increased flexibility to students and universities, serving as a bridge over time, space and distance obstacles. As some have observed, “[i]t is beyond doubt that distance education has progressed in concept and practice…from an “anywhere” to an “anytime” to an “any place” delivery method”. (Shachar & Neumann, 2010). Myriads of students are enrolling in online classes in record numbers for various reasons. Perhaps, the most appealing qualities of distance learning are the flexibility and convenience it offers to students, such as nontraditional students with inflexible work schedules or transportation challenges. (Daymont & Blau, 2008).
A recent study revealed that 7.1 million students were currently enrolled in at least one online course in the United States. (Allen & Seaman, 2014). The figure has increased steadily each year over the past decade. (Allen & Seaman, 2014). Given the proliferation in online course offerings, the landscape of instructional delivery has changed significantly and numerous species of online courses have evolved. The three prototypical classifications of course delivery are web-facilitated, blended/hybrid and pure online courses. (Allen & Seaman, 2014). In web-facilitated courses, instructors use technology to enhance a traditional face-to-face classroom experience, (1% to 29% of the content is delivered online). (Allen & Seaman, 2014). In blended or hybrid courses, instructors deliver a substantial portion of the content online (30% to 79%) and conduct the remainder of the class in person. (Allen & Seaman, 2014). In a pure online course, instructors deliver most or all of the content online (80% or more). (Allen & Seaman, 2014). All of these courses can be designed for synchronous or asynchronous content delivery and interaction. They are also typically structured around the same time parameters as the traditional face-to-face courses, as it relates to assignments, quizzes and exam deadlines.
Other variations of these traditional models of online education have developed in recent years. One such variation is the flipped classroom. This model is typical in a blended or hybrid course setting. It consists of interactive group learning activities inside the classroom, such as practical exercises, and direct computer-based individual instruction, including video lectures, which take place online. (Bishop &Verleger, 2013). Another more recent variation of the traditional online model is the Massive Open Online Course (MOOC). MOOCs are unique in that they are courses offered 100% online, free of charge to a very large number of people who are not registered students and will not receive completion credit at the institution offering the MOOC. (Allen & Seaman, 2014). While MOOCs have garnered sizable attention in recent years, very few institutions of higher learning have embraced them. (Allen & Seaman, 2014).
Yet another format, known as the self-paced online course, is also a relatively new and rare variation of an online course within the traditional university forum. (Rhode, 2009). In this type of 100% online course there is little or no synchronicity between the student and the professor (Gerlich, Mills & Sollosy, 2009). Instead, the student proceeds through the course at his or her own speed, regulated only by the deadline at the end of the term. This alternate delivery method allows the student to customize his or her learning to meet personal and educational preferences. For some students, this further enhances the ability to meet their educational goals. (Rhode, 2009).
The researchers developed a self-paced online course and compared student performance in the three different learning environments: 1) self-paced online course, 2) pure online course and 3) traditional face-to-face course. They collected and analyzed data relating to student achievement to develop a greater understanding of factors that positively influence student learning.
Abundant studies comparing student performance in online versus traditional face-to-face classes exist. One of the most all-encompassing studies is a summative meta-analysis, known as Twenty Years of Research on the Academic Performance: Differences between Traditional and Distance Learning. (Shachar & Neumann, 2010). In this study, the analysts quantitatively integrated the findings of 125 studies and 20,800 students over a 19-year period. They divided the study into four sub-periods from 1991 to 2009. The researchers utilized final course grades to measure academic performance. The meta-analysis revealed that each period the distance education students’ performance progressively increased over that of their traditional counterparts. The overall positive effect over the 19-year period was 70%. The researchers attributed this upward trend to technological advances, improved access to the Internet, increased acceptance of distance education in higher education, and greater participation of adult learners in higher education. Overall, the meta-analysis results echoed those of many studies that had come before and several that have followed (Allen & Seaman, 2014). The conclusion has been “that distance education not only is comparable to traditional instruction, but also…can outperform traditional instruction”. (Shachar & Neumann, 2010).
Similarly, researchers have compared traditional face-to-face courses to their hybrid counterparts and have noted equivalent or better student performance in hybrid classes. (Friedman & Joyce, 2014; McFarlin, 2008; Scida & Saury, 2006; Utts, Sommer, Arcredolo, Maher & Matthews, 2003). For example, in a recent study, a college instructor taught two sections of a statistics course, with each containing 75 students. (Friedman & Joyce, 2014). One section was a traditional seated class that met twice per week and the other was a 50% hybrid class, meeting only once per week for half the time and the remainder of the course took place online. For the hybrid class, the instructor required students to study the lecture online before class. During class, the instructor provided highlights of important or difficult portions of the lecture, worked through problems, and answered student questions. The instructor gave the same exams and course materials to students in both sections. The researchers used midterm and final exam grades to study student performance. Their findings revealed that students in the hybrid class were more likely to have grades of 80 or higher on both the midterm and final exams. Overall, the researchers observed, “no significant difference in student performance in a traditional versus a hybrid format…[S]tudents in the hybrid class did just as well as students in the traditional format.” (Friedman & Joyce, 2014).
Given the prevalence and success of hybrid classes, some researchers have examined their efficacy compared to fully online classes and found similarities in student performance and, in some instances, even greater student success in hybrid settings. (Larson & Sung, 2009; Lim, Morris, & Kupritz, 2014; Means, Toyama, Murphy & Baki, 2013). In one such study, 125 college students enrolled in a program evaluation course: 59 were in the fully online section and 66 participated in the blended delivery format, using both classroom and online instruction. (Lim et al., 2014). The study used a questionnaire including open-ended and closed-ended questions to link quantitative and qualitative data regarding student performance in the course. According to the researchers, their findings supported those of previous studies. “Data analysis revealed that the two learner groups in online and blended delivery formats did not show any significant differences in the mean scores for perceived and actual learning….” (Lim et al., 2014).
In other studies, researchers have undertaken a three-way comparison of delivery modes: online, blended and face-to-face. In one of these studies, each of the 168 student-participants took the class from the same instructor: 63 attended the class face-to-face, 22 enrolled in the online section and 83 participated in a hybrid format. (Larson & Sung, 2009). Researchers measured student performance by exam scores and final grades. The main observation that emerged was that no significant difference existed in student performance in the class across the three modes of delivery: face-to-face, online, and blended. More recently, a meta-analysis confirmed these findings and synthesized the empirical literature contrasting student performance in either fully online or blended conditions with those in a traditional classroom. (Means et al., 2013). The analysts reviewed scholarly literature on the subject published from 1996 (when web-based learning resources became widely available) to 2008. They then employed a selective screening process that allowed them to focus on extracting data from the most important and relevant of the 522 independent studies they initially found. In the end, “[t]he meta-analysis found that, on average, students in online learning conditions performed modestly better than those receiving face-to-face instruction. The advantage over face-to-face classes was significant in those studies contrasting blended learning with traditional face-to-face instruction….” (Means et al., 2013).
As indicated above, there exists a plethora of research analyzing and comparing student performance in traditional, online and hybrid classes; however, little to no empirical research and data exists on the efficacy of the self-paced online course compared to other modes of course delivery in higher education. The self-paced online course provides students with the most flexibility and convenience, so it would appear that more research in this area is warranted. Nonetheless, the only comparison study found measured the performance of teachers enrolled in a professional development course. (Russell, Kleiman, Carey & Douglas, 2009). In that study, researchers offered the course to instructors in a variety of online formats, including self-paced. The online section was highly supported, including an online facilitator and asynchronous peer interactions and participants worked through the course together. One section was self-paced, with no supports available. The other two online sections provided intermediate levels of support. To measure performance in the four sections, researchers employed six instruments: (1) a background survey to collect demographic data and information about prior technological and professional experiences; (2) a pedagogy survey to collect data about the beliefs and instructional practices of the participants; (3) a math assessment, designed to collect information about their understanding of the concepts; (4) logs, to capture information about day-to-day instructional practices; (5) student surveys from those in participants’ classrooms; and (6) a course evaluation, to gather information from teachers about their experiences in the faculty development course. Overall, the researchers found comparative positive outcomes across all four delivery methods. In other words, there was no significant difference between outcomes in the self-paced online course and those delivered in other formats. However, the researchers acknowledged that further research in this area is necessary to determine if the study’s findings are valid.
The findings in that study and the dearth of research in this area led the authors to conduct a comparison study of student performance in the university setting in three different mediums: a self-paced online course, a pure online course and a traditional seated course.
Development of the Self-Paced Online Course
The body of research has concludes, “[o]nline education can be an equally effective teaching format when the online course is designed using appropriate pedagogy”. (Driscoll, Jicha, Hunt, Tichavsky, & Thompson, 2012). The researchers in this study sought to determine whether this proposition held true for the self-paced online course. They strategically designed the course to incorporate a high degree of engaging content-student interactions to motivate the students to advance through the course in the absence of regular deadlines. This emphasis was consistent with the literature. A study of student perspectives in a self-paced online course revealed that the most important features were the variety of course activities and the technology employed to deliver the instructional content. (Rhode, 2009). Moreover, research shows that a learner-centered environment promotes high levels of motivation, learning and achievement. (McCombs & Vakili, 2005). Cognitive, motivational, and affective factors are instrumental in creating this environment.
In developing the self-paced online course, which is the subject of this study, the course designers researched the most innovative techniques and online pedagogy strategies to create a premium online course experience. The advanced level of technology employed in the course went well beyond that which is included in a conventional online course. In particular, the designers structured the course consistently and created a simple navigation process; developed high-impact introductions to each chapter; created engaging instructional content; and constructed interactive online activities. (Gaytan & McEwen, 2007).
The course was highly structured and organized. Good design and structure are must-haves for a successful online class. (Abel, 2005; Grant & Thornton, 2007). Consistent structure is vital to online student success and greater consistency among course modules leads to greater student satisfaction. (Grandzol & Grandzol, 2006). Online course structure refers to “standardized layout, design, arrangement of material, location of information and use [of] communication tools to enhance and facilitate learning and course navigation and ambiance”. (Lee, Dickerson & Winslow, 2012). The layout and navigation of this highly structured course was both logical and intuitive. Three main columns of information appeared in the learning management system to allow the students to proceed from administrative matters to content delivery to assessment. The first column contained administrative content, such as professor contact information, orientation materials and a schedule with proposed completion dates for each module. The middle column contained the instructional content. The designers created a webpage for each module to allow students to access the material content for each chapter in a centralized location. The final column contained the assessments, assignments and discussions. In addition, the course home page included widgets and images to enhance navigation, layout and asthethics. Lastly, the learning modules, which contained the course content, were designed identically.
The course featured high-impact introductions. Previous research has indicated that instructor presence and quality course content are essential elements to facilitate online student engagement (Dixson, 2010; Gaytan & McEwen, 2007; Moore, 2014; Rhode, 2009; Swan & Shih, 2014). Online students need to feel connected to the instructor, to other students in the course, as well as to the course content. (Dixson, 2010). Therefore, researchers in the instant study developed high-impact introductions to each topic in an effort to promote a strong professorial presence and to cultivate a student’s interest in the topic. For instance, the instructor filmed his introduction to the civil rights chapter at the Martin Luther King Center for Social Change in Atlanta, Georgia. In the video, he narrated the lesson as he navigated to the major sites at the King Center and explained the role each played in the Civil Rights Movement. The instructor ended the video with a catch phrase “Learn this chapter, live this chapter.” The researchers also developed stop motion videos to introduce some of the chapter content. In these videos, the instructor brought static objects to life and they appeared to move as he narrated the key concept covered in the chapter. The students came to know these videos as “Big Picture” pictures as they served as a short overview of the chapter to put the material in context. Finally, the instructor used props to illustrate the chapter highlights.
Rich and Dynamic Instructional ContentAll of the content was visual to promote engagement with the course concepts. (Ulbig, 2009). Designers translated concepts visually to make the material more interesting, intelligible and memorable. This is consistent with the notion that “a picture is truly worth a thousand words”
To create added student-content interaction, course designers employed a variety of technologies to create a high degree of robust content. (Chickering & Ehrmann, 1996; Dixson, 2010; Friedman & Friedman, 2014; Gaytan & McEwen, 2007; Rhode, 2009). Each learning module contained diverse multi-media content, such as instructional videos embedded with picture-in-picture images of the material referenced by the instructor. As shown in Figure 1, cartoon vignettes with content-rich material provided fun and interesting variations of the instructional content. Students also viewed relevant Internet video clips, which gave real world meaning to the learning content. (Zhang, Zhou, Briggs, & Nunamaker, 2006). Course designers used advanced presentation software with zoom and animation features to elevate the quality of the materials.
Figure 1. This figure depicts a cartoon animation of a class discussion on the line of succession of the presidency.
The researchers embedded interactive quizzes, review exercises, crossword puzzles, and hotspots throughout each module to allow the students to review their understanding of the material and improve engagement. Numerous studies have shown that learning takes place through active engagement rather than passive transmission. (Chandler, Park, Levin & Morse, 2011; Dixson, 2010; Friedman & Friedman, 2014; Gaytan & McEwen, 2007; Rhode, 2009). As shown in Figure 2, course designers incorporated a wide assortment of other activities, such as ordering, matching, labeling, pairing, and drag and drop activities throughout the material to provide variation. They intended for these self-assessment activities to foster active learning to build confidence, lower fear and improve retention.
Figure 2. This figure depicts a drag and drop activity.
RQ: After methodically developing the course as described above, the researchers queried whether a self-paced online course designed using appropriate pedagogy would positively impact student performance as compared to student performance in the same conventional face-to-face and online courses.
Participants in the study consisted of undergraduate students enrolled in three sections of Introduction to American Government courses delivered in the three different formats. A total of 96 students enrolled in the three sections (n=96): 26 students in the pure online section, 37 students in the self-paced online section (SPOC), and 33 in the traditional in-class section.
Given the extremely flexible nature of the SPOC course, its designers took one additional measure to improve student likelihood of success in the course. The literature reveals a student’s GPA is a significant positive predictor of student success in the online learning environment as it is a measure of the student’s cumulative academic effort and a solid predictor of future academic outcomes (Gerlich, Mills & Sollosy, 2009). Hence, only students with a 3.0 or higher GPA could register for the SPOC course.
The same instructor taught all three sections of the course, administering the same exams with the same time restrictions. Each of the three exams consisted of 50 multiple-choice questions related to the course PowerPoint lectures, discussion topics, and the textbook. The instructor allotted 50 minutes for students to complete each exam. In addition, he required the self-paced and conventional online students to download an internet lockdown browser, or a browser that allowed the student to access the course only via Desire2Learn, an integrated learning platform. The 50-minute time limit, equivalent to one minute per question, hindered online students from researching answers in the text. Lastly, none of the students knew that they were completing the same exam to prevent cheating. Researchers compiled the results, namely the overall number of points scored on each exam, from all students, and separated them by section to allow for empirical analysis.
To empirically examine the results of the exams, the researchers employed difference of means testing. Difference of means testing, or t-testing, is an empirical method that allows the investigator to examine whether the means (averages) of each sample are statistically different from each other (Fadem 2008; Zimmerman 1997). Specifically, the researchers in this study used a two-sample method, which tests the null hypothesis that the means of the two samples are statistically equal. The researchers were attempting to uncover whether there was an empirical difference in averages on the exams.
Table 1: Difference of Means Test: SPOC v. In-class
The first t-test compares the class averages on exams 1, 2, and 3 between the traditional in-class and SPOC sections of the course. Table 1 exhibits the results of this t-test. Of most interest to the researchers was whether student performance in the SPOC course was comparable to traditional face-to-face instruction. The researchers indeed found initial evidence of this phenomenon. Specifically, the difference in average for exam 1 for the SPOC and in-class section of the course was only 1.67 points. This suggests that the class averages in each section on exam 1 were statistically the same. Exams 2 and 3, where the difference was 5.34 points and 1.58 points respectively, repeated the findings of exam 1. Taken together, the estimates of the model imply that the course instruction in the in-class and SPOC settings worked equally as well and produced equal results, or the goal of online instruction. These findings indicate that student performance in SPOCs can be comparable to traditional face-to-face courses.
Table 2: Difference of Means Test: SPOC v. Conventional Online
The second t-test compared the class averages on exams 1, 2, and 3 between the traditional online and SPOC sections. The results of this t-test can be seen in Table 2. Again, of most interest was whether student performance in the SPOC and traditional online classes was comparable; and, the evidence suggests that student performance in the SPOC was superior. Specifically, the researchers found the difference in average for exam 1 for the SPOC and the traditional online section of the course was 14.15 points, which was statistically significant with 95% confidence. This implies, with 95% certainty, that a SPOC student scored 14-15 points higher on exam 1 than traditional online student. Moreover, the results of exams 2 and 3 return difference of means of 10.59 points and 13.23 points, respectively, which are significant with 90% confidence. Again suggesting, with 90% certainty, a SPOC student scored approximately 11 and 13 points higher than traditional online students did on exams 2 and 3, respectively. Taken as a whole, the results from this series of t-tests suggest that the pedagogy employed in the SPOC fosters an academically rigorous experience producing higher rates of achievement than traditional online course study.
This study examined whether a strategically designed SPOC would positively influence student performance as compared to its conventional online and traditional face-to-face counterparts. It represents a significant addition to the literature as no prior study examines the impact of student performance across these three mediums. In addition, very little research has been published on student performance in SPOCs.
Given the lack of intermittent assignment, quiz and exam deadlines in the SPOC course, the students could be more prone to procrastination or non-completion of the course. Course designers developed the SPOC with certain controls to offset these potential negative consequences. They employed advanced online pedagogical strategies, which included consistent structured, high-impact introductions, engaging instructional content, and interactive online activities. Also a minimum GPA of 3.0 was the single selection criterion for enrollment in the course.
Overall, this study demonstrated that the structure, design and selection criterion of the SPOC positively impacted student performance as compared to students in the same conventional face-to-face and online courses. When comparing the SPOC to a traditional face-to-face class, two interesting findings emerged. First, there was no statistical difference in exam average between the two course formats. Second, the SPOC students outperformed the in-class students on two of the three exams. Although the performance differences were minute, it is still interesting to note that initial evidence suggests that SPOC is not only an adequate course format, but also SPOC student performance can equal or exceed that of their traditional face-to-face counterparts.
When comparing the SPOC to a conventional online class, even more surprising results emerged. In each of the three exams, the SPOC students outperformed the conventional online students. Since the difference in averages in all three exams was very large and empirically significant, it is safe to assume that the instruction in a SPOC can affect the students more positively than traditional online classes. The researchers speculate that the maximum flexibility afforded the SPOC students enhanced their ability to achieve maximum performance. The SPOC students were able to select the optimal days and times throughout the semester to productively engage in the course. Unlike their traditional classroom counterparts, they were not constrained by scheduled meeting, assignment and exam dates and times. So although the maximum flexibility afforded in the self-paced environment could cause some students to fall prey to procrastination and non-completion of the course, students with a track record of academic success tend to be the type of self-disciplined students that could thrive in the self-paced learning environment.
This study provides empirical evidence of the learning effectiveness of the self-paced online course format in higher education under the prescribed conditions, namely a course developed with advanced course design and populated with high-achieving students. The selection criterion for the course supports the proposition that student success in this extremely flexible online learning environment is correlated to the student’s cumulative academic effort. The course and schedule flexibility may allow certain students to meet or exceed the learning that would take place in the traditional face-to-face or online course. Due to the self-regulated nature of the course, this course format may not provide an optimal learning experience for all students, yet it could have a very positive learning impact on students with an established record of academic success.The findings of this study have implications for curriculum design, course development and student performance. It provides insight that a course strategically designed to attract students with the appeal of the purely anytime and anywhere format can result in maximum student performance. This study provides distance education administrators with a model that can be used to increase distance learning enrollment and participation, as well as improve student success and retention.
There are two limitations to the scope of this study. The first limitation is the topic. Student performance was measured within the context of a single subject, American Government. Student success in the SPOC learning environment may vary by content. Some subjects may be better-suited or ill-suited for this learning environment. The second limitation is the breadth of the study. This study examined course performance over a single semester. Additional longitudinal studies examining performance over a greater time period may be provide more insight.
Abel, R. (2005). Achieving success in internet-supported learning in higher education: Case studies illuminate success factors, challenges, and future directions. Retrieved from http://home.fau.edu/musgrove/web/Achieving%20success%20in%20internet%20supported%20learning%20in%20higher%20education.pdf
Allen, I. & Seaman, J. (2014). Grade change: Tracking online education in the United States. Retrieved from http://www.onlinelearningsurvey.com/reports/gradechange.pdf.
Bishop, J. & Verleger, M. (2013). The Flipped Classroom: A Survey of the Research. Retreived from http://www.studiesuccesho.nl/wp-content/uploads/2014/04/flipped-classroom-artikel.pdf
Chandler, T., Park, Y., Levin, K., & Morse, S. (2011). The incorporation of hands-on tasks in an online course: an analysis of a blended learning environment. Interactive Learning Environments 2011, 1-13.
Chickering, A. & Ehrmann, S. (1996). Implementing the seven principles: technology as lever. AAHE Bulletin, October, 3-6.
Daymont, T. & Blau, G. (2008). Student performance in online and traditional sections of an undergraduate management course, Institute of Behavioral and Applied Management, 275-294.
Dixson, M. (2010). Creating effective student engagement in online courses: What do students find engaging? Journal of the Scholarship of Teaching and Learning, (10)2, 1-13. Driscoll, A, Jicha, K. Hunt, A, Tichavsky, L. & Thompson, G. (2012). Can online courses deliver in-class results? A comparison of student performance and satisfaction in an online versus a face-to-face introductory sociology. Teaching Sociology, (40)4, 313-331.
Fadem, B. (2008). High-Yield Behavioral Science (High-Yield Series). Hagerstwon, MD: Lippincott Williams & Wilkins. Friedman, L., & Friedman, H. (2014). Using Social Media Technologies to Enhance Online Learning. Retrieved from http://126.96.36.199:8080/xmlui/bitstream/handle/123456789/2163/EJ1004891.pdf?sequence=1
Friedman, L., & Joyce, T. (2014). Comparing Student Performance in a Hybrid vs. Traditional Section of STA 2000. Retrieved from http://ctl.baruch.cuny.edu/comparing-student-performance-in-a-hybrid-vs-traditional-section-of-sta-2000/.
Gaytan, J., & McEwen, B. (2007). Effective online instructional and assessment strategies. American Journal of Distance Education, 21(3): 117-132.
Gerlich, R.N., Mills, L.H., & Sollosy, M. (2009). An evaluation of predictors of achievement on selected outcomes in a self-paced online course. Research in Higher Education Journal, 4.
Grandzol, J.R., & Grandzol, C.J. (2006). Best practices for online business education. International Review of Research in Open and Distance Learning (7)1.
Grant, M., & Thornton, H. (2007). Best practices in undergraduate adult-centered online learning: Mechanisms for course design and delivery. Merlot Journal of Online Learning and Teaching, 3(4), 346-356.
Larson, D., & Sung, C. (2009). Comparing student performance: Online versus blended versus face-to-face. Journal of Asynchronous Learning Networks, (13):1, 31-42.
Lee, C., Dickerson, J. & J. Winslow (2012). An analysis of organizational approaches to online course structure. Online Journal of Distance Learning Administration, Volume XV (1). Retrieved from https://www.westga.edu/~distance/ojdla/spring151/lee_dickerson_winslow.html.
Lim, D., Morris, M., & Kupritz, V. (2014). Online vs. blended learning: Differences in instructional outcomes and learner satisfaction. Retrieved from http://files.eric.ed.gov/fulltext/ED492755.pdf
McCombs, B. and Vakili, D. (2005). A learner-centered framework for e-learning. Teachers College Record (108)8, 1583-1600.
McFarlin, B. (2008). Hybrid lecture-online format increases student grades in an undergraduate exercise physiology course at a large urban university. Advances in Physiology Education 32, 86-91.
Means, B., Toyama, Y., Murphy, R., & Baki, M. (2013). The effectiveness of online and blended learning: a meta-analysis of the empirical literature. Teachers College Record 115.
Moore, R. (2014). Importance of developing community in distance education courses. TechTrends 58(2), 20-25.
Rhode, J.R. (2009). Interaction equivalency in self-paced online learning environments: An exploration of learner preferences. The International Review of Research in Open and Distance Learning, 10(1).
Russell, M., Kleiman, G., Carey, R., & Douglas, J. (2009). Comparing self-paced and cohort-based online courses for teachers. Journal of Research on Technology in Education, 41(4), 443-466.
Schifter, C. (2004). Compensation modules in distance education: National Survey Questionnaire revisited. Online Journal of Distance Learning Administration, 7(1). Retrieved from https://www.westga.edu/~distance/ojdla/spring71/schifter71.html.
Scida, E., & Saury, R. (2006). Hybrid courses and their impact on student and classroom performance: a case study at the University of Virginia. CALICO Journal, 23(3), 517-531.
Shachar, M., & Neumann, Y. (2010). Twenty years of research on the academic performance differences between traditional and distance learning: Summative meta-analysis and trend examination. Merlot Journal of Online Learning and Teaching, 6(2): 318-334.
Swan, K., & Shih, L. (2014). On the nature and development of social presence in online course discussions. Retrieved from http://anitacrawley.net/Articles/Swan%20and%20Shih2005.pdf
Ulbig, S., (2009). Engaging the unengaged: Using visual images to enhance students’ “Poli Sci 101” experience. Political Science and Politics, 42(2) 385-391.
Utts, J., Sommer, B., Acredolo, C., Maher, M., & Matthews, H. (2003). A study comparing traditional and hybrid internet-based instruction in introductory statistics classes. Journal of Statistics Education, 11(3).
Zhang, D., Zhou, L., Briggs, R., & Nunamaker, J.F. (2006). Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Information and Management 43, 15-27.
Zimmerman, Donald W. (1997). A Note on Interpretation of the Paired-Samples t Test. Journal of Educational and Behavioral Statistics 22(3), 349–360.