Effects of an Open Educational Resources Initiative on Students, Faculty and Instructional Designers
Anthony A. Piña
Kenneth A. Moran
* This paper was one of three selected as a "Best Paper" among DLA 2018 proceedings, Jekyll Island, Georgia, June 24 - 27, 2018.
A research and development project was undertaken to document, analyze and report the effects of implementing a free/open educational resources (OER) initiative within Sullivan University’s College of Business Administration. The project involved the redesign of courses using no-cost textbooks and other educational resources, in place of resources costing students hundreds of dollars per course. Quantitative data were gathered from 311 students enrolled in sections of two widely-used undergraduate courses over four quarters. Qualitative data were gathered from interviews of faculty and instructional designers involved with the project. Results indicated that students in the OER courses were more satisfied with the cost and quality of the courses and the instructional materials and were more likely to recommend the course to others, compared to students in Pre-OER courses. The use of OER based on student learning outcomes was judged by faculty subject matter experts and instructional designers to have a positive effect on the quality of course design and the relationship between subject matter expert and instructional designer, compared to courses based on textbooks.
The United States Government Accountability Office reported that the cost of textbooks in the decade of 2002-12 increased 82%, while overall consumer prices during the same period rose 25% (USGAO, 2013). Estimates of average annual course material spending by students ranges from $638 (National Association of College Stores, 2014) to $1,200 (Baum & Ma, 2013). It is estimated that, on average, students are using over $300 of their federal financial aid each semester for purchasing textbooks, while some students are paying more than $400 for a single textbook (Senack & Donoghue, 2016).
An analysis was undertaken comparing the cost of textbooks used in fully online courses at Sullivan University. During the spring quarter, 230 unique online courses were offered. Of these courses, 72 (31%) required students to purchase a textbook costing more than $200. Fourteen of these courses required a textbook costing more than $300. In some of these courses, additional books or materials were also required (Piña & Moran, 2017).
Due to enrollment management strategies implemented during the summer term, the number of individual courses decreased by 38. However the percentage of courses with a textbook above $200 rose to 36%. Even more distressing was that the number of textbooks at the $300 or above price point doubled to 28. Also, for the first time, three books were priced above $400. Many of these increases were brought on by a single publisher, who increased the cost of its books by $30-$65 from spring quarter to summer quarter. The price increases were for the same editions of the textbooks currently in use—rather than for new editions of these books (Piña & Moran, 2017).
The analysis also revealed that, in addition to printed textbooks, students in many courses were also required to purchase software and other online content from various vendors and that more than 30 different products (each with different interfaces) were being used. Some of this content was bundled with textbooks, but would have to be purchased separately if the students opted to rent or purchase a used textbook. One publisher had eight different products--with eight different interfaces—that required students to purchase course codes and/or register accounts on the vendor’s site and leave the learning management system to use (Piña & Moran, 2017).
Free and Open Educational Resources
One strategy for addressing the high cost of textbooks and other education materials has been the use of free and open educational resources (OER). The William and Flora Hewlett Foundation, one of the primary promoters of OER have described them thusly:
“OER are teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge” (Atkins, Brown, & Hammond, 2007, p. 4).
These resources include textbooks, presentations, audio recordings, videos and other learning objects produced by faculty and subject matter experts. OER advocates often maintain that access to information and content is a fundamental human right, which should be available at no cost to the public in order to shrink the digital divide (Piña, 2015). Whereas traditional copyright is employed to limit the distribution of materials, open content licensing uses the power of copyright to make materials more widely available (Piña, 2015, Seaman & Seaman, 2017).
A recently survey of 2,700 faculty conducted by the Babson Survey Research Group (Seaman & Seaman, 2017) revealed that faculty awareness of OER is rising, but is still very low, with just 10% of respondents indicating that they were “very aware” of OER and another 20% reporting that they were “aware.” Two years earlier, 5% of faculty reported being “very aware” and 15% reported being “aware” of OER (Allen & Seaman, 2015).
Research into the effects of replacing traditional textbooks and instructional materials with OER is in its infancy (Farrow, Pitt, de los Arcos, Perryman, Weller & McAndrew, 2015). However the emerging studies published thus far are encouraging. Besides obvious cost savings (e.g. Hilton, J., Robinson, T., Wiley, D., & Ackerman, J. (2014), adoption of OER in place of costly materials has been associated with higher standardized test scores (Robinson, Fisher, Wiley & Hilton, 2014), higher scores on course assessments, lower failure rates and decreased attrition (Pawlyshyn, Bradlee, Casper & Miller 2013) and decreased student costs without a decrease in the quality of the instructional materials (Bliss, Robinson, Hilton & Wiley, 2013).
Several institutions have adopted major free/open educational resource initiatives. Tidewater Community College, the 11th largest community college in the U.S. (44,000 students) launched its first “Z-Degree” (for “zero textbook cost”), by replacing printed textbook with free and open alternatives. The “Z-Degree,” an associate of science in business administration, began in 2013 and has experienced a 6% decrease in student attrition, compared to the same degree offered with costly textbooks (Wiley, Williams, DeMarte & Hilton, 2016).
The University of Maryland University College (UMUC) is one of the country’s largest online degree providers, with more than 85,000 students (UMUC, 2016). In 2013, UMUC began implementing a major curriculum redesign to replace traditional textbooks with free/open educational resources in its more than 700 undergraduate courses (Schaffhauser, 2016).
As the concept of OER is being developed, the definition of truly “open” resources has been defined by David Wiley (2014) using “The 5Rs of Openness”:
- Retain – the right to make, own, and control copies of the content
- Reuse – the right to use the content in a wide range of ways (e.g., in a class, in a study group, on a website, in a video)
- Revise – the right to adapt, adjust, modify, or alter the content itself (e.g., translate the content into another language)
- Remix – the right to combine the original or revised content with other open content to create something new (e.g., incorporate the content into a mashup)
- Redistribute – the right to share copies of the original content, your revisions, or your remixes with others (e.g., give a copy of the content to a friend)
It should be noted that in this article, the term OER will be used to denote any materials that are deployed within a course at no cost to students, whether or not they meet all 5Rs.
Purpose for the Project
The project involved the strategic redesign of courses using no-cost textbooks and other educational resources in place of resources costing students hundreds of dollars per course, the training of faculty subject matter experts in the selection and use of OER and open content licensing and the collection and analysis of data on the effect of the OER initiative on students, faculty and instructional designers. This project sought to answer the following research questions:
- R1: Does the use of free/open education resources have an economic effect on students?
- R2: Does the use of free/open education resources influence student satisfaction?
- R3: Does the use of free/open education resources influence student retention and grades?
- R4: What are the effects of free/open education resources on faculty subject matter experts?
- R5: What are the effects of free/open education resources on instructional designers?
Data were gathered from 311 undergraduate students enrolled in two courses that were required for multiple degree programs: MGT304 Principles and Management and MKT304 Principles of Marketing. Fifty-six percent of respondents identified as female and 44% identified as male. NOTE: Other courses were originally intended to be included in the study and were developed using free and open resources in place of costly materials. However enrollments in these courses were deemed to be not high enough for a reliable sample of students.As shown in Figure 1 below, student experience in purchasing their textbooks was overwhelmingly via their institutions’ bookstores or outside bookstores or rentals. Only 8% of students had any experience using free textbooks or library e-books in their courses.
Figure 1: Prior Student Experience in Purchasing Textbooks
SMEs trained on OER and Creative Commons. Both the MGT304 and MKT304 courses were redesigned and developed during the winter and spring 2016 quarters. Course redesign and redevelopment focused on removal and replacement of the respective “paid” course textbooks with free/open educational materials. Full-time faculty from the College of Business Administration (COBA) were assigned as subject matter experts (SME) to work with the University’s Instructional Designer Team to redesign and develop both the online and on-campus (face-to-face) versions of the courses. Each SME had experience teaching their respective courses, but no prior experience in course design with OER. Training was developed and delivered to the SMEs on Creative Commons licensing and on instructional design and the selection, evaluation and implementation of free and open educational resources in online, hybrid and on-campus (face-to-face) courses.
Course Mapping. Perhaps the most significant adjustment made to the instructional design/development process transitioning from courses designed using cost-textbooks to OER course materials was to shift responsibility of approving the course map from the SME to the COBA Dean. Traditionally, the SME would develop the course map during the first weeks of the 12 week contracted course development period. This process would often involve negotiations with the assigned instructional designer if the course objectives and/or weekly learning objectives were not stated in terms of measurable student learning outcomes and if the assessments were not sufficiently aligned with the objectives. This often resulted in strained relationship s between SMEs and IDs.
The course mapping development and approval processes were adjusted to accommodate the OER project. Specifically, the adjustment meant that course mapping for the targeted OER courses would be completed at the academic department level (in this case the College of Business Administration) prior to the start of the contract course development period; thus allowing the full quarter for the SME to develop or select the course. Traditionally, the course map approval process was delegated by the COBA Dean to the SME as part of the course redesign/redevelop assignment. However, the approval process was adjusted for implementation of the OER grant to be overseen directly by the Dean, in order to ensure that the course and student learning outcomes driving course development would be established before the start of the contracted course development period.
Course Development. SMEs were permitted to redesign/redevelop the course as they felt best given the stated course objectives/student learning outcomes identified in the course. Interestingly, the MGT304 course identified a variety of OER resources, assembling them to address the course objectives/student learning outcomes. While the MKT304 course used a free, creative commons textbook. In both cases, the resources used in the course redesign/redevelopment process added no cost to the student; thus fulfilling the research grant objective for OER.
Each SME worked collaboratively with an assigned instructional designer who served as the development project manager (Piña & Sanford, 2017). Each course was initially developed as an online course using course development tools available through the learning management system. The face-to-face course outline was developed in reference to the online course to ensure continuity for course delivery. After course development, both courses were reviewed and approved by the COBA Dean and administered to online and face-to-face students in the summer 2016 and following quarters. Additionally, each SME was paired with a Sullivan University librarian to assist them in locating high quality OER course materials.
Comparison of Pre-OER and Post-OER student satisfaction was measured using a pre/post survey design. The Pre-OER data collection period occurred in the winter and spring 2016 quarters and resulted in 156 student responses. The Post-OER data collection period occurred in the summer and fall 2016 quarters, resulting in 155 student responses. Student economic impact was measured by calculating the costs of existing MGT304 and MKT 304 textbooks. Data on Pre- and Post-OER final grades and retention rates were gathered from the University’s student information system.
Instrumentation and Analysis
The Pre-OER data collection instrument consisted of nine questions. The first six questions used a 5-point Likert-type scale to collect student perceptions regarding their satisfaction with course costs, the quality of course materials, the quality of non-textbook readings, the quality of video materials, the overall quality of the course, and whether the student would recommend the course to another student. One question allowed the student to designate the course being taken (e.g., MGT304). One question asked if students to identify methods used to acquire their textbook. The last question collected student gender data.
The Post-OER data collection instruction consisted of eleven questions. The first six questions likert-scale questions and the demographic items were identical to those in the Pre-OER instrument. The three additional questions collected student perceptions related to their level of satisfaction/dissatisfaction with OER versus printed textbooks, whether they would desire more classes that used OER materials, and preference for enrollment in colleges/universities that used OER.
A t-test was undertaken to compare the mean values for the separate Pre- and Post-OER groups and to assess significant differences in any of the variables. Specifically, the study compared seven variables: i.e., textbook costs, material quality, non-textbook quality, video quality, course quality, course recommendation, and total comparisons.
An additional focus of the research project was the effects of the OER initiative on faculty and instructional designers. Qualitative data collection included semi-structures interviews with faculty who served as subject matter experts for course development and who taught the Pre-OER and OER courses and with the instructional designers who worked with the faculty subject matter experts to design and develop the courses.
R1: Does the use of free/open education resources have an economic effect to students?
There was a significant economic effect on students in OER courses versus those in Pre-OER courses, with students in OER saving between $172 and $336 per course. Is it estimated that the total savings for students in the summer 2016 MGT304 and MKT304 courses was $51,186 and estimated annual savings of students taking these courses would be at least $200,000. If all of a student’s college courses were redesigned with OER, then the savings for each COBA student could be upwards of $1,000 or more per academic term; $4,000 or more per academic year. Extrapolated out for a student’s entire college stay could be as much as $16,000 or more for a four year degree program. As previously noted, this level of student savings would be very appealing to any institutions faced with increasing costs, lower student enrollment, and facing a radically changed higher education marketplace. Therefore, the research grant was not only timely, and its findings a value add to any college or university administer searching for interventions capable of increasing their competitiveness to attract and retain students.
R2: Does the use of free/open education resources influence student satisfaction?
Table 1 displays the results from a t-test indicating significant differences between the Pre- and Post-test groups for all of the individual variables examined and combination of variables for each group. Results show that Post-test group perceptions for all variables tested were significantly different from Pre-test group perceptions related to course design using textbooks versus OER resources. Specifically, Post-test results showed significant differences for Textbook Costs (t=-20.144, p=.000), Materials Quality (t=-8.806, p=.000), Non-Text Quality (t=-7.901, p=.000), Video Quality (t=-6.234, p=.000), Course Quality (t=-2.878, p=.004), and Recommend Course (t=-2.878, p=.001). Additionally, the combined totals for each of the study variables showed significant differences in Pre- and Post-test group perceptions related to course design using textbooks versus OER resources (t=-12.426, p=.000).
Results of student surveys indicated that students in the OER courses were more satisfied with the cost of their courses and judged the free/open materials in their courses to be equal to or higher in quality than the commercial materials used in their other courses. More OER than Pre-OER students would recommend their courses to others and more students would prefer OER –based courses than course using traditional print textbooks.
Students in the Post-OER courses were asked three additional questions that were not in the Pre-OER survey. The first was to determine whether students in Post-OER courses would like to take more courses that utilized free/open educational resources. Results displayed in Figure 2 below showed overwhelming support for additional OER-based courses with 88% of students in agreement (38%) or strong agreement (50%) that they wish to take more courses that use OER, while only 4% disagreed or strongly disagreed.
Figure 2: I would like to take more courses that use free/open educational resources.
Additionally, students in Post-OER courses were asked whether they would be more likely to enroll at a particular college or university if they were aware that the institution featured the use of OER. As shown in Figure 3 below, 80% either agreed (36%) or strongly agree (44%) that OER would be a significant motivator in selecting a college or university.
Figure 3: I would be more likely to enroll in a college or university if I knew that it used free/open educational resources for its courses.
The last survey item assessed the anecdotal claim that a large number of students would prefer to take classes that utilized printed textbook, rather than online/digital materials. Results shown below in Figure 2 indicated that 19% of students surveyed agreed (14%) or strongly agreed (5%) that they would rather take courses with printed textbooks. Sixty-two percent of students, indicated agreement (25%) or strong agreement (37%) with the use of OER-based materials.
Figure 4: I would prefer to take classes that use printed textbooks instead of free/open educational resources.
R3: Does the useof free/open education resources influence student retention and grades?
An analysis of final grades and end-of-term retention data was undertaken for winter and spring (Pre-OER) sections of MGT 304 and MKT 304 compared to summer and fall (Post-OER) sections of the same courses. No significant difference was found for either grades or retention, indicating that switching from paid materials to free/open materials did not have a detrimental effect on these student outcomes.
R4: What are the effects of free/open education resources on faculty?
Interviews with the faculty who served as subject matter experts for the OER courses yielded some very interesting data. Faculty stated that they were initially reticent to switch from textbooks to OER, as textbooks provided a sense of security and familiarity. Once they had a chance to learn about and utilize OER, the nervousness gave way to excitement and a sense of liberation. The following comments are representative:
“At first, I felt like I was flying blind without a textbook to dictate the content and course sequence; but I soon discovered that focusing first upon the student learning outcomes and then selecting the most appropriate materials is a much better way to go.”
“Converting my course to OER was a revelation. I had previously thought of the print textbook as a security blanket. What I came to realize was that it had become a straightjacket which told me what to teach, when to teach it and how to teach it.”
“I now feel a sense of liberation and freedom to build courses in a way that meets our desired student learning outcomes and to use a much wider variety or resources to meet those outcomes.”
“The commitment to use OER in classes takes some preparation and involves some searching to find materials, but I am able to mix-and-match different parts of books, videos, website and other materials that are much more interesting to my students and to me as a teacher.”
R5: What are the effects of free/open education resources on instructional designers?
Interviews with the instructional designers revealed that the OER project had resulted in improved working relationships with faculty SMES and overall higher course quality. The following comments are representative:
“Relationships with some SMEs could get adversarial. SME’s would sometimes fight us about making certain that learning objectives were measurable, that assessments aligned with the objectives and that the content was robust enough to meet the University standards for online courses and federal credit hour requirements. Using the course maps and OER and starting with learning outcomes, rather than textbooks, has nearly eliminated these problems.”
“We used to require that 80% of the course content be the SME’s original work. As a result, most of the content was just multiple pages of text, some of which was not properly attributed and required a significant amount of editing by IDs. The OER course content is much higher quality and the courses are much richer with multimedia content.”
“Our biggest problem has always been that too many SMEs do not deliver their course content on time, which limited the time available for us to do the course development and QA. Replacing the mandates for original content to having them select the best OER for their course has resulted in much higher on-time delivery
Both faculty and instructional designers involved in the project unanimously observed that the move to free/open resources based on student learning outcomes improved the course development process and improved the working relationship between the faculty and instructional designers. The use of course maps by the academic department to move the formulation of course objectives and primary assessments to the department level, eliminated the “tug of war” between ID and SME, allowing the SME to focus almost exclusively on course content. Faculty and instructional designers also agreed that the use of OER in place of original, primarily text-based content from the SME resulted in overall higher quality course content that was more multimedia in nature and was delivered on time.
The findings of this study affirm that the effects of a Free/Open Educational Resources (OER) initiative on students, faculty and instructional designers--plus an academic department and the institution’s course development process—are overwhelmingly positive. There was a significant positive economic effect for students. Contrary to the conventional wisdom that materials sold by publishers must be superior to materials made freely available, students judged the OER materials to be superior to traditional textbooks and found courses that used OER materials to be superior to the same courses using traditional costly publisher-produced materials. Additionally, more students in OER-based courses reported they would recommend these courses to other students, would rather enroll in more courses that use OER and would be more likely to enroll in an institution that used OER in their courses. This study found no evidence that students were educationally disadvantaged or were more likely to drop courses in which free and open educational resources were used.
Faculty and instructional designers were also unanimous in their assessment of the effects of the OER initiative. They concluded that the implementation of OER, coupled with the subsequent changes in the course development process resulted in higher quality courses and a higher quality working relationship among those who developed the courses.
As a result of the OER initiative, all undergraduate courses in the College of Business Administration are shifting from costly textbooks and instructional materials to free/OER textbooks and other materials. Other academic units within the University are now following suit. In addition the COBA has shifted its basic course design paradigm from “putting face-to-face courses online” to “online drives the bus” --using the online course mapping procedures to drive the development of on-ground courses. In 2018, the Provost approved a plan by the Dean of COBA and the Associate Provost for Instruction and Online Learning to train all academic leadership in the use of course maps and OER and implement the COBA course development paradigm university-wide.
The Authors wish to acknowledge individuals whose participation was critical to the success of this study. These include faculty members Dorothy Day, Bobby Dean, Dr. Terrance Kibiloski, Dr. Christopher Hughes, Richard Routt and instructional designers Diane Curtis, Peggy Muller and Barry Sanford. We also wish to express gratitude to the Sullivan University Office of Institutional Research and its Director, Dr. Mark Wiljanen, for providing a research grant which allowed us to pursue and disseminate the findings of this study.
Allen, I. E., & Seaman, J. (2015). Opening the textbook: Educational resources in U.S. higher education, 2015-16. Babson Park, MA: Babson Survey Research Group.
Atkins, D. E., Brown, J. S., & Hammond, A. L. (2007). A review of the open educational resources (OER) movement: Achievements, challenges, and new opportunities. Menlo Park: The William and Flora Hewlett Foundation.
Baum, S., & Ma, J. (2013). Trends in college pricing 2013. New York, NY: The College Board.
Bliss, T., Robinson, J., Hilton, J., & Wiley, D. (2013). An OER coup: College teacher and student perceptions of open educational resources. Journal of Interactive Media in Education.
Farrow, R., Pitt, R., de los Arcos, B., Perryman, L., Weller, M., & McAndrew, P. (2015). Impact of OER use on teaching and learning: Data from OER research hub (2013-2014). British Journal of Educational Technology 46(5), 972-976.
GAO (2013). College Textbooks: Students have greater access to textbook information. A report to congressional committees. Washington, DC: United States Government Accountability Office.
Hilton, J., Robinson, T., Wiley, D., & Ackerman, J. (2014). Cost-savings achieved in two semesters through the adoption of open educational resources. The International Review of Research on Open and Distance Learning 15(2).
National Association of College Stores (2014). Student watch™: Attitudes and behaviors toward course materials 2015-2016. Oberlin, OH: The National Association of College Stores.
Pawlyshyn, N., Bradlee, Casper L., & Miller, H. (2013). Adopting OER: A case study of cross-institutional collaboration and innovation. EDUCAUSE Review, November-December 2013.
Piña, A. A. (2015). Open content licensing. In J. M. Spector (Ed.) The SAGE Encyclopedia of Educational Technology. Thousand Oaks, CA: Sage Publications, Inc.
Piña, A. A. & Moran, K. A. (2017, December). Changing course design culture with free and open educational resources. Presented at the annual meeting of the Southern Association of Colleges & Schools-Commission on Colleges, Dallas, TX.
Píña, A. A., & Sanford, B. K. (2017). . K. (2017). The ID database: Managing the instructional development process. Tech Trends 61(4), 331-340.
Robinson, T., Fisher, L., Wiley, D., & Hilton, J. (2014). The impact of open textbooks on secondary science learning outcomes Educational Researcher 43(7), 341-351.
Schaffhauser, D. (2016). How to go textbook free. Campus Technology. Retrieved from https://campustechnology.com/articles/2016/03/09/textbook-free.aspx
Seaman, J. E., & Seaman, J. (2017). Opening the textbook: Educational resources in U.S. higher education, 2017. Babson Park, MA: Babson Survey Research Group.
Senack, E., & Donoghue, R. (2016). Covering the cost: Why we can no longer afford to ignore high textbook prices. Washington, DC: U.S. Public Interest Research Group/Student PIRGs. Retrieved from www.studentpirgs.org/textbooks.
UMUC (2016). Fiscal year 2016 fact book. Upper Marlboro, MD: Office of Accreditation, Compliance and Reporting, University of Maryland University College.
USGAO (2013). College textbooks: Students have greater access to textbook Information (GAO-13-368). Washington, DC: United States Government Accountability Office.
Wiley, D. (2014). The access compromise and the 5th R. Iterating Toward Openness. Retrieved from https://opencontent.org/blog/archives/3221
Wiley, D., Williams, L., DeMarte, D., & Hilton J. (2016).The Tidewater Z-Degree and the INTRO model for sustaining OER adoption. Education Policy Analysis Archives 24(1).
Online Journal of Distance Learning Administration, Volume XXI, Number 2, Summer 2018
University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Contents