Development and Evaluation of a Distance Learning Master's Degree in Family Studies


Mary Bold, Ph.D., CFLE
Department of Family Sciences
Texas Woman's University
mbold@twu.edu

 

Abstract

This article describes an online Master's degree in Family Studies and reports on student readiness for distance learning, student satisfaction, and program review. Evaluations indicate that graduate students in Family Studies are sufficiently prepared to succeed in online course work and that they are satisfied with their program, identifying convenience as a primary advantage of distance learning. Program evaluations, conducted after course conversion to online delivery, focused on Principles of Good Practice for distance education. This review includes implications and future directions for administrators of distance learning programs.


Development and Evaluation of a Distance Learning Master's Degree in Family Studies

Distance learning (DL) has evolved to span a broad spectrum of technology uses, known variously as distributed education, distance education, and, more recently, e-learning to refer to online distance learning (Baker, 2003). At Texas Woman's Uni versity (TWU), a decision was made in 2001 to mount a fully online Master's degree in Family Studies to extend the mission of the University to provide educational programs to meet the needs of adult students, especially women, who seek graduate study in preparation for career or to enhance career advancement. T he Family Studies program is one of eight graduate online degree programs at TWU (the University also has two certificate programs and two undergraduate degree programs online). Adoption of DL programs can be understood in light of TWU's student composition: 46% of students are enrolled in g raduate programs. Most are working professionals who attend school part-time. The national profile of the distance learner—older, working in paid employment, and enrolling for 3 or 6 credit hours per semester—is reflected at TWU. Students in the online Family Studies program follow the pattern (statistics from Fall 2004 semester): mean age is 33 (range is 22 years to 62 years) and 86% enroll for 6 or fewer hours at a time. At present, all students enrolled in the Master's program are female.

The management of online programs such as the Family Studies M.S. challenges administrators and faculty as traditional program models are adjusted or replaced to meet new needs of students and new pressures in terms of technology. At TWU, a review of the five largest growing online degree programs (which included the program reported here) generated a list of five different management models. Evaluation of all programs operating on a campus must contribute to a campus goal of identifying best models and best practices if online programs are to progress in terms of quality, effectiveness, and accountability. As this article reflects, initial efforts in evaluation are a starting point but small sample size and developing instruments necessarily restrict interpretation and cannot produce summative evaluation.

Description of Program

The online M.S. in Family Studies is a 33-hour degree program. Students can comple te all courses and graduation requirements through asynchronous distance learning. Optional face-to-face (F2F) seminars are offered at the start of every semester to preview online courses. In these “Seminar Days,” Family Studies faculty members also offer advising sessions and conduct additional workshops in scholarly writing, APA format, Statistics, and portfolio presentation. Technology training is also made available to new students during Seminar Days as well as through technology support throughout the semester by the program's technology graduate assistants (called Tech GAs) and the University's HelpDesk.

The online program was officially approved in 2002 at the request of the Department of Family Sciences and the College of Professional Education . The M.S. in Family Studies is one of 12 degree programs in Family Sciences at TWU. It was selected for conversion to distance learning based on its generalist orientation and absence of clinical certification. The 11 courses comprising the degree are also available to students in the Department's other graduate programs (Child Development, Early Childhood Education, Family Therapy, School Counseling, and Community Counseling). Students can elect to write a thesis as their final project or create an electronic professional portfolio; the large majority of students elect the portfolio option. An approved academic program by the National Council on Family Relations (NCFR), this degree earns the provisional credential of Certified Family Life Educator (CFLE) for graduates. The number of students in the M.S. program grew from 27 in Fall 2002 to 169 in Fall 2004.

All of the courses in the Family Studies Master's degree were in the University course in ventory but few had incorporated instructional technology beyond use of e-mail for communication and the Internet for research assignments. Thus, launching of the program necessarily included development of online course activities and assignments to meet the existing course objectives. A phase-in period of seven semesters allowed this development as courses advanced from partially online to 100% online. Only two courses moved to 100% online delivery without gradual phase-in. Faculty members developing these course elements were simultaneously learning to use new technology tools; their development work was not funded and no workload credit was awarded.

Course delivery is primarily through Blackboard, a course management system ( CMS ) utilized by many universities in the U.S. However, an early decision was made to avoid complete reliance on a single platform. Thus, the program uses multiple technologies: Blackboard, e-mail, student-produced video to report on field experiences, wikis and blogs on non-Blackboard servers, and CD courseware created locally and published by the Department. To date, nine faculty members have contributed materials for inclusion on the CDs; faculty members retain copyright to their works. Thus, for a distance student who may take classes with only a few professors, the CDs provide exposure to more faculty members in the Department.

Files on the CDs include short video lectures, written lectures, conceptualizations of theories, animations, and self-quizzes. The CD courseware provides students with study materials that can be utilized without Internet access, a “safety net” especially appreciated by students beginning the program.

Additional pragmatic measures have developed since the launch of the program: maintenance of a back-up website on a separate file server, creation of an emergency contact list every semester, and creation of an orientation CD for new students. Such additions underscore the need for flexible goal-setting and managing of online programs.

Student Readiness

Most institutions of higher education in the 1990s adopted technology tools to utilize e-mail communication and the Internet. Even small universities like TWU set about building the infrastructure necessary to incorporate the new technology in campus operations and instructional curricula. Larger schools moved quickly to establish DL programs as part of their technology adoptions (Charp, 2001; Hons, 2002). Not all of the DL efforts were successful, especially those that attempted commercial models within universities. But DL was not abandoned, by any means, as statistics indicate: the expected average growth rate for online students for 2004 is 24.8%, an increase of 5% since 2003 (“Sloan Survey,” 2004). S tudent enrollment has steadily increased ( National Center for Education Statistics, 2002) as has women's enrollment in graduate study (Burke, 2000; Kramarae, 2001; National Center for Education Statistics, 2003). When course work can be achieved on one's own schedule, working professionals are more likely to enroll in graduate degree programs (Card & Horton, 2000; Hersh, Junium, Mailhot, & Tidmarsh, 2001; Levin, Levin, & Waddoups, 1999). For the woman seeking a postsecondary degree in family sciences, online course work may be especially attractive because she may place high importance on preserving time for family. She may enroll in online courses unaware of how her participation may impact her life in terms of time and resources. Thus, an early evaluation in the online Master's program was assessment of student readiness, both in terms of academic readiness and personal expectations.

Self-Directed Learning Readiness Scale

In the first semester of the phase-in, 51 female students were recruited from graduate courses in which online content ranged from 25% to 65%. They were administered the SDLRS: S elf-Directed Learning Readiness Scale (Guglielmino,1978), a widely-used and accepted instrument for assessment of readiness for self-directed learning (Long & Ageykum, 1988; Martin, 1996; Merriam & Brockett, 1997). Readiness scores ranged from 187 to 283 (compare to national norms ranging from 137 to 290), with mean score of 240 and SD of 24.79 (compare to national mean of 214 and SD of 25.59). The mean score for students in the local sample was above average, at almost precisely +1 SD. Eight students (approximately 16% of the local sample) scored above +2 SD. Thirty-six students (76% of the sample) could be described as more likely than most to be successful in an independent learning environment.

Online Survey Responses

Twenty-six of the volunteers (51% of the sample) also responded to an online survey consisting of open-ended questions and a short demographic section. The mean age of respondents was 39; the range was 23 years to 56 years. When asked about the social aspect of their distance learning experience, most respondents acknowledged feeling connected but some also expressed preference for a mix of on-campus and online settings. Some of their comments were:

In another survey item, respondents identified students "most suited to succeeding in distance education courses" as independent, disciplined, and self-motivated. A few responses mentioned that DL is good for those who prefer writing to talking and for those needing flexibility. Representative comments were:

Discussion of Readiness

On the SDLRS the mean score for students in the local sample was above average, and 76% of the sample could be described as more likely than most to be successful in an independent learning environment. This result, coupled with the students' descriptions of what kind of student is well suited to DL, indicate a not surprising conclusion: graduate students are capable of independent learning, the self-directed sort that suggests the self-discipline needed for success in an online program.

Student Satisfaction

Student satisfaction has served as an indicator of effective DL (“Effective practices,” 2005). While other measures are needed to assess student outcomes, regular checks on student satisfaction are especially informative during the conversion phase of on-campus course work to the online environment. Graduate students can provide immediate feedback to instructors and course designers and, as working professionals, they can make assured statements of whether the content in an online course has met their career-related needs. For the online Master's in Family Studies, student surveys have been conducted in all courses. Results from some of those evaluations are presented below to reflect student satisfaction in the first semester of course conversion to online delivery, at the mid-point of course conversion, and after course conversion was completed.

Evaluation during First Semester of Course Conversion

Twenty-eight participants answered an online survey at the end of the first semester of course conversion. Respondents were asked to rate their satisfaction with distance learning, to comment on what they liked most and least about their courses, and to identify the components of their courses that were most helpful.

Satisfaction with course

Of 28 respondents, more than half ( n = 15) said they were very satisfied with their course. About a third ( n = 9) said they were satisfied. Thus, 86% ( n = 24) reported a positive satisfaction rating in their first distance learning experience.

Liked best about taking a DL course

Relief from driving to and from campus was specified by 14 respondents. Representative comments were:

Liked least about taking a DL course

Twenty-one respondents reported what they liked least, with loss of interaction or face-to-face contact cited by 12 respondents. Representative comments were:

Additional comments

A final open-ended question permitted respondents to add comments on any subject. Only a few respondents wrote additional comments and most of those pertained to an individual professor. A general comment was:

Course components

Likert scale items were marked to indicate whether a course component helped tremendously, helped somewhat, did not help, or hindered the student in course work. Students identified online editing by the professor, on-campus meetings, and the locally produced CD courseware as the most helpful elements of their courses.

Evaluation of Statistics Course at 50% Online

The same survey was used in the Master's level Statistics course at the stage of 50% online delivery. Fifteen students responded at the end of semester. These students echoed the earlier semester's results, with emphasis on the convenience and flexibility of distance learning.

Satisfaction with course

Of 15 respondents, about 73% ( n = 11) said they were very satisfied with their course. A fifth ( n = 3) said they were satisfied and one said somewhat satisfied. Thus, 93% ( n = 14) reported positive satisfaction with distance learning.

Liked best about taking a DL course

This open-ended question generated responses from all students. Convenience, flexibility, or time savings was cited by all but one respondent. Some comments were:

Liked least about taking a DL course

All students responded with at least one comment. Common topics are represented by these comments:

Additional comments

A final open-ended question permitted respondents to add comments on any subject. All but one student wrote comments; most were positive, although two students expressed desire for more F2F class time. Comments reflected previous experience with distance learning, which may explain some of their positive reaction to this class:

Course components.

Likert scale items were marked to indicate how helpful the student found a course component. All respondents marked the highest rating for the online quizzes (non-credit tests that provided immediate feedback) in Blackboard. Students also identified the Blackboard discussions and the locally produced CD courseware as most helpful elements of the courses.

Evaluation after Course Conversion

In the final semester of the program's phase-in period, TWU commissioned a survey on student satisfaction with all of the online degree programs offered by the University. The Priorities Survey for Online Learners (PSOL) by Noel-Levitz, Inc., was conducted online and the randomly selected respondents from the Family Studies program numbered 21. The PSOL reported means on student satisfaction for responses on a 7 point scale. Additional data were reported on the gap between students' satisfaction with an aspect of their online studies and the importance they place on that aspect. This is referred to as the performance gap; the lower the number, the better the school or program's performance.

Three summary items were reported, reflecting overall satisfaction. As detailed in Table 1, mean scores for these items were higher for the local sample when compared to the national mean. (Higher mean scores reflect higher satisfaction.)

Note. Means reflect responses on a 7 point scale

Table 2 displays the survey's five scale means (based on 26 aggregated items) with comparison to the national means for online students. The trend of the summary items is repeated, with the local sample means higher than the national means. Performance gaps were small, most being within a half-point. The small gaps indicate that students' level of satisfaction is close to the importance they placed on items.

Note. Means reflect responses on a 7 point scale. Performance gap refers to the difference between students' rating of importance and students' rating of satisfaction.

Twenty-six individual items (on a 7 point scale) addressed a variety of student concerns, from financial aid to bookstore services. On some items concerning institutional services, the Family Studies means fell below the national means. A typical comparison: on the timeliness of financial aid information, the Family Studies mean was 4.86 and the national mean was 4.99. Similar items where the Family Studies mean was lower were: online career services, tutoring services, registration processes. Other institutional items, where the Family Studies mean was higher than the national mean, were: billing procedures, institution reputation, online library resources, adequacy of financial aid, bookstore services, speed of institution's response to questions.

On all items specific to the program and directly controlled by faculty, the Family Studies means were higher than national means. Table 3 details a number of these items.

Note. Means reflect responses on a 7 point scale

More general items about the program also produced Family Studies means higher than the national means. As Table 4 indicates, most of these means were 6.0 or higher (on a 7-point scale).

Note. Means reflect responses on a 7 point scale

As an initial evaluation, the PSOL indicates a number of strengths in the program. The limited size of sample must be taken into account, however.

Discussion of Student Satisfaction

Course evaluations from the first 3 years of the online program indicate overall student satisfaction with the courses and appreciation for the ability to pursue a graduate degree from a distance. Family Studies faculty benefited from graduate students' ratings of the courses as well as their frank comments about assignments, activities, and processes. The graduate students were generous with their praise when the course met their needs; they were also articulate about what they did not like.

Dominant themes of responses were convenience and flexibility, both of which were deemed benefits of online course work. Social connection also emerged as a theme and was described in terms of a trade-off for the convenience of DL.

Students' most frequent description of DL concerned its convenience and the word “convenient” appeared in answers across multiple survey items, regardless of whether a question sought information about convenience. The greatest single convenience cited was not having to drive to campus for classes.

Flexibility was represented in answers about working “at my own pace” and doing course work “late at night when my son is in bed." Respondents' comments about time management also spoke to this theme.

Social connection emerged as a problematic aspect of DL. Although most students reported feeling connected in the online setting, loss of interaction was cited frequently as a disadvantage with comments such as "I miss the continued face-to-face interaction." While a trade-off between connection and convenience was acknowledged in responses, the importance of connection was evidenced. This speaks to the need for a sense of community in online settings (Card & Horton, 2000; Moller, Harvey, Downs , & Godshalk, 2000) and the need to foster student interactions online (Card & Horton).

For the online Master's program, the need for interaction is answered with optional on-campus seminars three times a year as well as course activities such as small group work within Blackboard, virtual seat mates (whereby students have one-on-one discussion with the students who post directly above and below them on discussion boards), live chat for 5 to 8 students at a time, use of wikis for online groups, and a dedicated Blackboard course in which peer tutors support students' intensive work on their electronic portfolios.

In the earliest evaluation, students identified online editing by the professor, on-campus meetings, and the course CD as the most helpful elements of their courses. A later evaluation of a Statistics course identified online quizzes, Blackboard discussions, and the course CD as the most helpful elements. Published literature suggests that DL students prefer a variety of teaching methods when receiving online instruction (Andrusyszyn, Cragg, & Humbert, 2001). This program's findings lend credence to this supposition and support the local program's decision to provide multiple instructional methods in the online courses.

Levin, Levin, and Waddoups (1999) suggested that online technology provides professors with the opportunity to consciously use a variety of instructional methods. Using multiple online approaches permits students to develop expertise and professors to become comfortable with online technology (Andrusyszyn, Cragg, & Humbert 2001; Levin, Levin, & Waddoups). Faculty members developing online assignments for this program found that as their own comfort with the technology increased, they also served as better mentors to students in assignments that require using new technology. Although the number of students who have been graduated is still fewer than three dozen, this increasing comfort level with technology has been evident in graduates' electronic portfolios.

At the end of the program's phase-in period, the evaluation commissioned by the University, the PSOL, indicated that student satisfaction was high across virtually all survey items. On a scale of 1 to 7, program mean scores were routinely over 6.00 compared to national means that were mainly between 5.00 and 6.00. At present, PSOL data are a beginning point. Trend data over 4 or 5 years will serve evaluation purposes well, as will even a second year's report to compare means and performance gaps with the initial report. Over time, student ratings of importance of items may vary; as students entering the online Master's have more technology skills or prior experience in distance learning, they may place different value on aspects of the program.

Program Review

While studies have shown that distance learning courses produce comparable or superior results in learning (Card & Horton, 2000; English, Rojeski, & Branham, 2000; Hersh, Junium, Mailhot, & Tidmarsh, 2001), many universities have found accrediting bodies suspicious of online courses and degrees. Thus, online programs seem to be undergoing greater scrutiny both within and outside of their institutions.

In the two most recent semesters of the online Master's, since conversion to 100% online delivery, our emphasis has shifted to program evaluation, driven largely by the increased expectation for documentation from the Southern Association of Colleges and Schools (SACS) and the Texas Higher Education Coordinating Board (THECB). Three of the evaluations are described briefly here.

First Program Evaluation

The first evaluation at this level was a blind review of electronic professional portfolios created by the program's first graduates, most of whom took a mix of online and on-campus courses. A faculty panel developed a rubric for assessing the electronic portfolios on measures such as demonstration of competencies (matching certification standards for family life educators), scholarly writing, and presentation technique. Limited to only 11 portfolios, the review nevertheless provided program faculty with valuable insights into how course products can be utilized in the portfolio process. In the next year, the review is expected to generate findings on outcomes for students whose entire course of study has been in 100% online courses.

Second Program Evaluation

The second evaluation at the program level was an assessment of comparability with on-campus classes. The program faculty members were confident that their conversion of the Family Studies Master's courses to online delivery had not negatively altered the existing course requirements or objectives; however, a formal assessment was needed to document comparability. A line-by-line comparison of the syllabi of the online versions of courses with their on-campus counterparts demonstrated that objectives had not been diluted. In some cases, objectives had been increased to include use of technology, making technical proficiency an expected outcome of the program. In most courses, textbook assignments remained the same In the course Teaching Family Sciences, the project assignment in the online version is more rigorous than the traditional assignment in the on-campus version. In the applied Statistics course, online delivery with a highly structured curriculum (supported both by Blackboard and the faculty-authored CD courseware) provides students with increased assessments of their learning and more feedback on their performance than was possible in the on-campus class. Thus, the comparability review provided evidence of equal or greater rigor and content, down to the assignment level.

Third Program Evaluation

The third program evaluation was a review of each online course's application of the Principles of Good Practice (PGP) for distance education courses as outlined by the Texas Higher Education Coordinating Board (THECB). These principles mirror SACS guidelines for distance learning. A number of universities use some form of checklist to address PGP. The TWU checklist condenses the particulars of PGP to 7 pages of items such as provision for student interaction in the course, use of audio and video, and elements of syllabus and course content. The value of the checklist will come as faculty members can track improvements to individual courses over time. A few courses already have two checklists completed (because they have now been taught 100% online twice) and differences are evident in those courses. Simply by completing the checklist review, faculty members become more aware of course elements that are lacking and begin to think about how an addition or revision can be made before the course is offered again. As the inventory of checklists builds, a summary chart can be constructed to track improvements in the courses.

Implications and Future Directions

The growth of the online Master's in Family Studies at this University is indicative of the interest in distance learning in the family field at the graduate level. The high level of satisfaction by students suggests that graduation rates will be high, making the whole-degree approach attractive to higher education administrators. The faculty members of this program comment regularly that much of the local success can be attributed to usual characteristics of the school's graduate students: older students who are working professionals and who are highly motivated to complete the degree. (The graduation rates for whole degrees online may be different at the undergraduate level.) Graduate students are well suited to the demands of online course work; they are capable of independent and self-directed learning.

Social Connection

In planning for and designing online programs, instructors and administrators should provide for students' desire for a sense of connection with peers and professors. Especially in the family field, some students may express a preference for face to face contact, which should be accommodated with optional class meetings or other gatherings. While acknowledging this preference, programs can, nevertheless, promote the use of technology to meet social needs. Thus, students in an online program benefit from what is called a dual education (Haythornthwaite, Kazmer, Robins, & Shoemaker, 2000), mastering distanced interaction as well as the program curricula.

Learning Objects

As this program demonstrates, an online degree is enhanced by tangible products that support student learning, such as CD courseware. Authored by faculty members and published by the academic Department, the online Master's course CDs provide learning objects (LOs) that can be used across course sections and even across courses. LOs are reusable digital resources that can be stored and located efficiently, serving an online course, an online program, or an entire discipline. Typically providing a small chunk of information or instruction, the LO can be a stand-alone unit or a building block in a curriculum. LOs are seen as a long-term solution for creating content for distance learning; standards for their technical attributes are being proposed by national technology agencies and companies (Acker, Pearl , & Rissing, 2003; “Learning objects,” 2003; Polsani, 2003). How, and whether, a program's faculty adopts LOs may be an indicator of the level of collaboration possible in a program. Research into both questions will assist in the development of distance learning.

Assessment

In the space of this program's short history, the shift in evaluation focus is stark: from initial concerns over the individual course components (such as CD courseware and discussion boards) to full-program review to demonstrate comparability with traditional on-campus degree programs. In a shorter span than might have been expected, the need for full program review arose due to accrediting bodies' scrutiny of distance learning. Development of normed instruments has emerged in the same time span, providing opportunity to compare local results with national results but with the caveat that instruments will probably be revised often as standards are set and more programs are created. For example, in the year that the program used the SDLRS to assess readiness, the publisher was simultaneously testing a readiness assessment specific to distance learning. In the year that the University commissioned the PSOL, the publisher of that tool was in the process of adapting it from an earlier paper-and-pencil assessment of on-campus student satisfaction. Managing assessments in this changing environment is likely to be a challenge for most programs and institutions. A flexible management style that tolerates learning over time is needed to manage technological change (Hofman & Orlikowski, 1997). In terms of graduate education, research is needed on emerging management methods that direct assessment of both student achievement and institutional effectiveness.

Trends in student assessment include e-portfolios, embedded assessments in courses, and a general shift in faculty's attention from course evaluation to individual student progress or mastery across a program curriculum. The technology of distance learning can support such shifts but faculty may not have resources in place to utilize the technology. Faculty development, including concentrated skill-building in new technology, is needed if distance learning programs are to maximize the potential of their own tools.

References
Acker, S., Pearl , D., & Rissing, S. (2003, July). Is the academy ready for learning objects? Syllabus. Retrieved January 15, 2005 , from http://www.campus-technology.com/article.asp?id=7886

Andrusyszyn, M. A., Cragg, B., & Humbert, J. (2001). Nurse practitioner preferences for distance education methods related to learning style, course content, and achievement. Journal of Nursing Education, 40 (4), 163-170.

Baker, R. K. (2003). A framework for design and evaluation of Internet-based distance learning courses: Phase one – framework justification, design and evaluation. Online Journal of Distance Learning Administration, 6 (2). Retrieved January 15, 2005, from http://www.westga.edu/%7Edistance/ojdla/summer62/baker62.html

Burke, C. (2000). Time and space for women: Distance learning and domestic constraints. Retrieved October 17, 2001 , from http://www.com.unisa.edu.au/cccc/papers/refereed/paper6/paper6-1.htm

Card, K. A., & Horton, L. (2000). Providing access to graduate education using computer-mediated communication. International Journal of Instructional Media, 27 (3), 235-245.

Charp, S. (2001). E-Learning. T H E Journal, 28 (9), 10. Effective practices. (2005). Retrieved January 15, 2005 , from http://www.sloan-c.org/effective/studentSatisfaction.asp

English, K., Rojeski, T., & Branham, K. (2000). Acquiring counseling skills in mid-career: Outcomes of a distance course for practicing audiologists. Journal of the American Academy of Audiology, 11 (2), 84-90.

Guglielmino, L. M. (1978). Development of the self-directed learning readiness scale. (Doctoral Dissertation, University of Georgia, 1978). Dissertation Abstracts International, 38, 6467-A.

Haythornthwaite, C., Kazmer, M. M., Robins, J., & Shoemaker, S. (2000). Community development among distance learners: Temporal and technological dimensions. Journal of Computer-Mediated Communication, 6 (1). Retrieved January 15, 2005 , from http://www.ascusc.org/jcmc/vol6/issue1/haythornthwaite.html

Hersh, W. R., Junium, K., Mailhot, M., & Tidmarsh, P. (2001). Implementation and evaluation of a medical informatics distance education program . Journal of the American Medical Informatics Association, 8 (6), 570-584.

Hofman, J. D., & Orlikowski, W. J. (1997). An improvisational model of change management: The case of groupware technologies. Sloan Management Review, (38) 2, 11-22.

Hons, C. (2002). Big ten school in cyberspace: A brief history of Penn State 's World Campus. T H E Journal, 29 (6), 27.

Kramarae, C. (2001). The third shift: Women learning online . Washington , DC : American Association of University Women Educational Foundation. Learning objects portal. (2003). Retrieved January 15, 2005 , from http://ilearn.senecac.on.ca/lop/issues/issues.htm

Levin, J. A., Levin, S. R., & Waddoups, G. L. (1999). Multiplicity in learning and teaching: A framework for developing innovative online education. Journal of Research on Computing in Education, 32 (2), 256-269.

Long, H. B., & Ageykum, S. (1988). Self-directed learning: Assessment and validation. In H. B. Long & Associates (Ed.), Self-directed learning: Application and theory (pp. 253-266). Athens , GA : Adult E ducation Department, University of Georgia .

Martin, R. (1996). The role of self-directed learning in career development . Retrieved October 17, 2001 , from http://www.inspiredinside.com/learning/article006.htm

Merriam, S., & Brockett, R. (1997). The profession and practice of adult education . San Francisco: Jossey-Bass.

Moller, L. A., Harvey, D., Downs , M., & Godshalk.,V. (2000). Identifying factors that effect learning community development and performance in asynchronous
distance education. The Quarterly Review of Distance Education, 1 (4), 293-305.

National Center for Education Statistics. (2002). Student participation in distance education. Retrieved February 8, 2004 , from
http://nces.ed.gov/programs/coe/2002/section5/indicator38.asp

National Center for Education Statistics. (2003). Trends in graduate/first professional enrollments . Retrieved February 8, 2004 , from http://nces.ed.gov/programs/coe/2003/section1/indicator07.asp

Polsani, P. R. (2003). Use and abuse of resuable learning objects. Journal of Digital Information, 3 (4), Article 2003-02-19. Retrieved January 15, 2005 , from http://jodi.ecs.soton.ac.uk/Articles/v03/i04/Polsani/

Sloan Survey of Online Learning: Entering the Mainstream: The Quality and Extent of Online Education in the United States , 2003 and 2004. (2004). Retrieved January 15, 2005 , from http://www.sloan-c.org/resources/survey.asp


Online Journal of Distance Learning Administration, Volume VIII, Number III, Fall 2005
University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Content