Research and Evaluation Needs for Distance Education: A Delphi Study


Kay Rockwell, Professor, Agricultural Leadership, Education & Communication
300 Ag Hall, University of Nebraska Lincoln, NE 68583-0709
Phone number: 402-472-3913
krockwell1@unl.edu
 
Jack Furgason, Program Evaluation Assistant,
300 Ag Hall, University of Nebraska Lincoln, NE 68583-0709
jfurgason1@unl.edu
 
David B. Marx, Professor, Biometry
103 Miller Hall, University of Nebraska,Lincoln, NE 68583-0712
Dmarx1@unl.edu

This manuscript has been assigned Journal Series No. 13083, Agricultural Research Division, University of Nebraska.

Conclusions About Research and Evaluation Needs As Seen by Distance Educators

Distance education established its roots as a form of instruction at least 150 years ago as correspondence study (Holmberg, 1986). With the advancements in telecommunications technologies, distance learning programs rapidly expanded so distance education is now defined as "the acquisition of knowledge and skills through mediated information and instruction, encompassing all technologies and other forms of learning at a distance" (United States Distance Learning Association,1998).

As policy makers need to make wise philosophical and policy decisions about using distance education, they are asking for research and evaluation about the distance learning process. Therefore, distance educators were asked to participate in a Delphi study to identify and rank future research and evaluation needs/issues. Four themes emerged:

1. Cooperation and collaboration among institutions. To create a cooperative and collaborative environment among institutions across the state, it will be important to create a long-term vision about the educational system so distance education can be integrated into strategic plans. The research emphasis needs to focus on working relationships among higher education institutions to promote inter-campus cooperation, including connectivity among the institutions and coordination of the technology. It will be important to determine factors impeding or enhancing the development of a distance education programming structure that identifies funding formulas fairly rewarding all collaborative participants.

This research needs to focus on the interface between postsecondary institutions and secondary schools with later research on the interface between secondary and elementary schools.

2. Designing the educational experience for the distance learner. Understanding the characteristics of successful distance learners is necessary to help identify strategies that are highly effective for a successful distance learning experience. It will be important to determine if learning occurs differently via distance. It is necessary to identify processes teachers use to customize the learning experience and assess "innovative" instructional processes to identify which best help students learn.

It’s important to diagnose problems that learners encounter in terms of time and place, along with how the required use of technology affects their motivation.

It’s also important to determine student expectations for credit courses and why they do not take advantage of distance learning opportunities. It is more important to determine expectations for professional improvement than it is to determine expectations for personal enrichment.

3. Teacher preparation. Emphasis needs to be placed on identifying effective teacher competencies, along with the training needed to support faculty development. Identifying the amount and type of support or assistance teachers need, as well as resources required for various distance teaching approaches and course development, is important.

It will be important to identify barriers and incentives for instructors to teach via distance, how instructors can be encouraged to work together, how instructors can most effectively use multiple technologies, how rigor is maintained and how teaching quality is judged by different audiences. It will be important to identify effective and fair teacher evaluations.

4. Educational outcomes. It will be important to determine participation and completion rates and assess outcomes of both formal higher education and K-12. It is less important to assess outcomes of non-formal professional and personal growth workshops. However, it will be important to determine if perceived educational needs are met and if distance education facilitates lifelong learning.

The Study’s Background

In 1987, Holmberg suggested the structure or categories of distance education research include (a) philosophy and theory of distance education, (b) distance students, their milieu, conditions and study motivations, (c) subject-matter presentation, (d) communication and interaction between students and their supporting organization [tutors, counselors, administrators, other students], (e) administration and organization, (f) economics, (g) systems [comparative distance education, typologies, evaluation, etc.], and (h) history of distance education. With this as background information, the research agenda on distance learning greatly expanded over the past decade with research emerging in a number of areas. Some of the researchers have been interested in the learner – their attributes and perceptions, interaction patterns and how these contribute to the overall learning environment. An example of this includes research on a learner-centered approach (Hanson et al., 1996). Researchers such as Verduin and Clark (1991) and Collis, Veen and DeVries (1993) addressed administrative and organizational issues. Others such as Thach and Murphy (1994), Schlosser and Anderson (1994), Roger and Whetten (1982), and Moore (1994) have addressed operational issues relating to networking, cooperation, coordination and collaboration. Still others have addressed faculty issues. Carl, (1991), Clark (1993), Olcott, Jr. & Wright (1995) all document faculty concerns about teaching via distance. Rockwell et al. (1999) looked at faculty incentives and obstacles to teaching via distance. Miller and Carr (1997) and Rockwell et al. (2000) addressed information and training needs in the 1862 Land-Grant universities.

In 1993, Miller stated that distance education entered a particularly important stage in its development. He noted four long-term trends, including:

Research on distance teaching/learning has greatly expanded over the last decade. National and international conferences are now being held to share some of this research along with other experiences related to distance teaching/learning across the county. Examples include the Distance Learning 99 (1999) sponsored by the University of Wisconsin-Madison, the International Conference on Learning With Technology (2000) sponsored by Pennsylvania Department of Education and the nonprofit Global Learning Consortium, and the Distance Learning Administration 2000 Conference (2000) sponsored by the State University of West Georgia.

Although research on distance teaching/learning is rapidly emerging, decision makers in one mid-west state continue to probe a number of questions such as:

While the distance education environment is changing, many questions remain unanswered and research needs to focus on the context within which distance education is developing along with the inputs needed to effectively implement distance education. As steps are identified and actions are taken to implement distance opportunities, these processes need to be evaluated so adjustments can be made, and outcomes need to be assessed to find out if educational needs across the state are being met. Because there are still numerous research and evaluation needs in distance education, priorities need to be established to provide some guidelines for researchers/evaluators. Current research can be directed toward these priorities and proposed research can focus on state-wide needs. Likewise, evaluations can be directed toward these priorities and evaluations that cut across institutional boundaries can be developed.

Therefore, this study identified research and evaluation priorities for distance education in one mid-west state. Specifically, it focused on (a) the planning that is occurring as distance education develops, (b) structuring decisions required for distance education, (c) the implementation process, and (d) evaluation needs in documenting outcomes.

Methodology

Research design

CIPP Model. Stufflebeam’s (1971) CIPP Model (context, input, process and product) provided a structure for looking at distance teaching/learning research/evaluation needs. "Context" focused on planning decisions, "input" on structuring decisions, "process" on implementation processes, and "product" on outcome attainment.

Modified Delphi process. To focus in a future-oriented mode, a modified Delphi technique was selected for identifying research and evaluation priorities for distance education. The Delphi technique was originally used to target future problems and foresee solutions. Part of its success lies in its use of experts in the field in question. By utilizing the knowledge of experts, combining it and redistributing it, the study opens up doors and forces new thought processes to emerge. It also allows for respondents to see how closely they responded to the rest of the field of experts and to justify their train of thought (McKillip, 1987).

Study participants. Three groups of distance educators participated. They identified research and evaluation issues that are relevant at the present time, and/or ranked the importance of the ideas. The three groups included (a) a five-member Steering Committee with interests in distance education locally, nationally and internationally, (b) a 43-member Delphi panel representing state colleges, the State University, elementary/secondary education, the University Medical Center, special interests related to distance education, community colleges, and state government, and (c) 14 participants in a state-wide Distance Learning Conference: Communities of Learning that included, but were not limited to, K-12 classroom teachers, technologists, community leaders, extension educators, instructional designers, media specialists and librarians, college/university professors and school board members.

Steering committee. After the Steering Committee identified potential members for the Delphi panel, they brain stormed relevant topics/issues for distance education research and evaluation needs. The researchers then categorized the topics/issues according to the CIPP framework and specific items were organized into a draft survey instrument. The Steering Committee reviewed and critiqued the items on the instrument to confirm that the 39 items, along with their sub-topics, reflected the committee’s thoughts and ideas about potential research and evaluation needs.

Delphi panel – First round. For the first round, items were rated from very important to very unimportant. The first round instrument was posted on a World Wide Web page. Members of the panel were mailed a letter explaining the study and its purpose, a hard copy of the questionnaire and instructions on accessing and answering the instrument electronically. Twenty panel members participated in the first round.

Delphi panel – Second round. Mean scores were calculated for each item from the first Delphi Panel response using a 5-point scale where very important =1; important = 2; neither important or unimportant = 3; unimportant = 4; and very unimportant = 5. For the Delphi panel’s second instrument, the mean score was marked on an importance scale for each of the original items; panel members were then asked to rate the accuracy of the mean scores using a three-point scale comprised of (a) should reflect More Importance, (b) is an Accurate representation of importance and (c) should reflect Less Importance.

From the comments written in on the first round, 10 new items were added to the second round questionnaire. Respondents were asked to rate the importance of these items using the same 5-point scale employed in the first round instrument. Twenty-eight panel members completed the second round instrument.

Distance education conference participants – Third round. Frequency distributions were calculated for the accuracy ratings given to each of the original items. Round two mean scores were adjusted up or down based on the net difference between the proportions of responses indicating the item was judged either more important or less important (see formula in Appendix A). The adjusted means, together with the raw mean scores calculated for the new items added in round two, were added to an instrument for a third round. The third round instrument again asked for a rating of the accuracy of the mean scores using a three-point scale: (a) should reflect More Importance, (b) is an Accurate representation of importance and (c) should reflect Less Importance.

Results from round two were presented at a state-wide Distance Learning Conference: Communities of Learning in September, 1998. After the presentation, conference participants completed the third round instrument. Scores were again re-adjusted up or down to produce a final score (see formula in Appendix A).

Data interpretation

Data were grouped according to the CIPP categories – context (planning decisions), input (structuring decisions), process (implementation decisions) and product (outcome decisions). One additional category included general education items.

An independent distance education consultant first identified the priorities in each of the categories as he saw them emerging from the data. The principal investigator then independently verified the priorities and presented the findings at the state-wide Distance Learning Conference: Communities of Learning. Conference participants discussed the findings before they completed the third round of the Delphi data collection process. The principal investigator then adjusted the initial findings to reflect how the conference participants confirmed or changed the importance ratings of the different items. And finally, the Steering Committee critiqued the written report to confirm the findings and their interpretations.

Findings Related to Planning Decisions

Research and evaluation needs categorized in the "planning decisions" category were those that help define objectives for distance education across the state and judge whether proposed objectives are sufficiently responsive to the assessed needs. Distance educators were concerned about two areas in the planning category: collaboration and coordination, and distance learners.

Collaboration and coordination. In collaborative efforts, it is important to identify funding formulas that fairly reward all collaborative participants. The highest interest is on collaboration among postsecondary institutions. There is slightly less interest in collaboration between postsecondary and secondary institutions and even less interest in collaboration between secondary and elementary schools (see Table 1).

Technology coordination among higher education is important, as is connectivity among these institutions. Technology coordination and connectivity among the grouping of schools is also of interest. There is much less interest in looking at the role of a state-wide television network in this coordination. There appears to be more interest in research on how the technology is used, rather than on the processes needed to coordinate and connect it.

Distance learners. Understanding the characteristics of successful distance learners ranks high, along with potential problems they may have with required equipment. There is less interest in exploring problems learners have using the technology (see Table 1).

Diagnosing learners’ problems with access in terms of time and place is very important, along with how the required use of technology affects motivation. Less concern was noted for diagnosing problems with overall student costs.

It is very important to determine learners’ expectations for credit courses and professional improvement and less important to identify expectations for personal enrichment. There is more interest in identifying institutions’ opportunities to meet learner expectations for obtaining credit courses than for professional improvement or personal enrichment needs.

It appears very important to determine why potential learners fail to take advantage of distance learning opportunities.

Table 1. Types of assessments needed to serve planning decisions for implementing distance education; How important is it to...
VERY IMPORTANT (_= 1.000 to 1.499a)
1.029 ...diagnose problems learners have with various distance delivery strategies (i.e., integrative TV, audio-conference, web-based, multi-media, etc.) in regard to equipment requirements?
1.130 ...assess how technology is organized for connectivity among higher education institutions?
1.145 ...assess factors that facilitate or inhibit coordination/cooperation for programming (i.e., course development, mutual course use, credit transfer, etc.) among higher education institutions?
1.152 ...identify characteristics of successful distance learners (i.e., self-regulation, independent inquiry, collaborative tendency, familiarity with computer tools, etc.)?
1.154 ...assess how technology is organized for connectivity between K-12 and higher education institutions?
1.168 ...diagnose problems learners have with various distance delivery strategies (i.e., integrative TV, audio-conference, web-based, multi-media, etc.) in regard to access to the learning opportunity at a convenient time?
1.229 ...assess client expectations for access to various educational opportunities for obtaining credit courses and/or degrees?
1.245 ...assess how the technology is coordinated between K-12 and higher education institutions?
1.277 ...assess how the technology is coordinated among higher education institutions?
1.339 ...assess client expectations for access to various educational opportunities for professional improvement?
1.340 ...diagnose problems learners have with various distance delivery strategies (i.e., integrative TV, audio-conference, web-based, multi-media, etc.) in regard to access to the learning opportunity at a convenient place?
1.344 ...diagnose problems learners have with various distance delivery strategies (i.e., integrative TV, audio-conference, web-based, multi-media, etc.) in regard to motivation or desire to participate?
1.373 ...ascertain opportunities that different educational institutions have to meet the client expectations for obtaining credit courses and/or degrees in-state, regionally, and nationally?
1.378 ...assess how technology is organized for connectivity among the pods across the state?
1.391 ...ascertain why potential learners fail to take advantage of distance education offerings?
QUITE IMPORTANT (_= 1.500 to 1.999a)
1.504 ...ascertain opportunities that different educational institutions have to meet the client expectations for professional improvement in-state, regionally, and nationally?
1.517 ...assess factors that facilitate or inhibit coordination/cooperation for programming (i.e., course development, mutual course use, credit transfer, etc.) between higher education institutions and secondary schools?
1.628 ...diagnose problems learners have with various distance delivery strategies (i.e., integrative TV, audio-conference, web-based, multi-media, etc.) in regard to using the technology required in the delivery process?
1.666 ...assess client expectations for access to various educational opportunities for personal enrichment?
1.735 ...assess how the technology is coordinated among the pods across the state?
1.856 ...assess how funding formulas can be changed to fairly reward all institutions in a collaborative distance education project?
1.888 ...ascertain opportunities that different educational institutions have to meet the client expectations for personal enrichment in-state, regionally, and nationally?
1.897 ...identify how the different educational institutions are relating their distance education programming efforts to their institutional mission?
1.984 ...diagnose problems learners have with various distance delivery strategies (i.e., integrative TV, audio-conference, web-based, multi-media, etc.) in regard too overall student costs?
SOMEWHAT IMPORTANT (_ = 2.000 to 2.499a)
2.118 ...assess factors that facilitate or inhibit coordination/cooperation for programming (i.e., course development, mutual course use, credit transfer, etc.) among secondary/elementary units?
2.204 ...assess how technology is organized for connectivity between the television network and educational organizations?
2.305 ...assess how the technology is coordinated between the television network and educational organizations?
a Scale: 1 = Very important 2 = Important 3 = Neither important nor unimportant 4 = Unimportant 5 = Very unimportant

Findings Related to Structuring Decisions

Research and evaluation needs categorized in the "structuring decisions" category were those that help facilitate designing distance education programs and procedures.

Distance educators said it is most important to identify highly effective strategies for successful distance learning including assessing "innovative" instructional processes to identify what best helps distance students learn. Determining factors that impede or enhance the development of a structure supporting distance education programming is also ranked high. Of concern are assessing the financial resources needed and available for course development, along with the time faculty need to develop and teach via distance. Determining a cost/benefit ratio ranks slightly lower in importance (see Table 2).

Distance education teacher competencies are a great concern. Research that identifies effective teacher competencies ranks very high, as do teacher training needs. Also important is identifying the types of support/assistance instructors need.

Ranking less important are an assessment of the pros and cons of different learning models and instructor qualifications for in-state distance learning courses.

Preference is given to developing models designed primarily for in-state use. Assessing models designed for regional, national or international use is viewed as less important. Instructor qualifications for such regional, national and international models are also viewed as less important.

Results indicate less interest in assessing past courses to identify successes and failures as part of a continuing education effort for teachers. Assessing past marketing strategies to identify how to create a market is also ranked as less important. There is little interest in defining and identifying terms relating to distance education.

Table 2. Types of assessments needed to serve structuring decisions for distance education; How important is it to...
VERY IMPORTANT (_ = 1.000 to 1.499a)
1.000 ...identify which strategies are the most effective in making a distance education learning experience successful?
1.000 ...determine the factors that impede or enhance the development of a structure that will make distance education work programmatically (i.e., competing bureaucracies, etc.)?
1.068 ...identify the kinds of support/assistance that is necessary for individual instructors to develop courses?
1.324 ...identify factors that contribute to effective teacher competencies in distance education?
1.328 ...assess resources needed as well as those available in regard to financial costs to an institution?
1.350 ...assess resources needed as well as those available in regard to resource systems that support faculty development?
1.351 ...identify training needs of distance education teachers?
1.372 ...assess "innovative" instructional processes to identify what best helps distance students learn?
1.402 ...assess resources needed as well as those available in regard to time faculty need to develop and teach distance courses?
QUITE IMPORTANT (_ = 1.500 to 1.999a)
1.608 ...determine how competitive interests can cooperate or collaborate?
1.622 ...identify expectations for instructor qualifications required for distance education courses that are used in-state?
1.626 ...assess resources needed as well as those available in regard to cost benefit?
1.673 ...identify the pros and cons of distance education models that are designed primarily for national use?
1.689 ...assess pros and cons of different distant education models or methods for addressing various learning objectives (pods, television based, computer based, telephone based, combinations of the technologies)?
1.693 ...assess past courses, workshops, or conferences to identify success and failures for continuing education?
1.697 ...identify expectations for instructor qualifications required for distance education courses that are used regionally?
1.705 ...assess past courses, workshops, or conferences to identify success and failures for degree programs?
1.711 ...identify expectations for instructor qualifications required for distance education courses that are used international use?
1.725 ...identify expectations for instructor qualifications required for distance education courses that are used nationally?
SOMEWHAT IMPORTANT (_ = 2.000 to 2.499a)
2.067 ...assess past marketing strategies to identify how to best create a market?
2.124 ...identify pros and cons of distance education models that are designed primarily for regional use?
2.272 ...identify the pros and cons of distance education models that are designed primarily for international use?
NEITHER IMPORTANT NOR UNIMPORTANT (_ = 2.500 – 2.999a)
2.791 ...identify pros and cons of distance education models that are designed primarily for in-state use?
3.137 ...to identify and define terms that relate to distance education (i.e., system, method of delivery, pods, etc.)?
a Scale: 1 = Very important 2 = Important 3 = Neither important nor unimportant 4 = Unimportant 5 = Very unimportant

Findings Related to Implementation Issues

Research and evaluation needs categorized as "implementation issues" are those that relate to how distance education is being set in motion. Once implementation questions are answered, distance education procedures can be monitored, controlled and refined.

Distance educators indicated that the most important issue deals with identifying both the barriers and incentives in using distance delivered education. Considerable emphasis is placed on comparing classroom-based instruction and distance learning instruction in terms of being able to apply and use the content. The survey indicates most interest is on application and use of content, followed by knowledge or skills acquired, learning styles, interaction with other learners and with the instructor and feeling a part of a learning community (see Table 3).

There is a great deal of interest in identifying what makes collaborative distance education offerings successful. Specific concerns relative to instructors and instructional processes include the barriers and incentives for implementing distance delivery by the instructor, the processes used to customize the educational experience, factors that encourage educators to work together and the instructors’ use of multiple technologies.

Another important issue deals with identifying current structures blocking distance delivery. Other important administrative issues include factors encouraging or discouraging educators from working together for program development and delivery. The issue of quality and the maintenance of rigor is important, along with identifying factors that influence how different audiences judge quality.

Addressing how multiple technologies can be used is viewed as important, specifically as they relate to learners perceptions about their advantages and disadvantages, the incentives and barriers for instructors to incorporate multiple technologies, and incorporating multiple technology use within the infrastructure.

Less important is identifying the benefits and drawbacks of various administrative models such as cohort designs, lead instructor with instructors-of-record, and facilitator licensed processes. There is less interest in research into how instructors are transferring traditional classroom skills to distance-based instruction.

Although comparing classroom-based instruction with distance instruction ranked high, there was less interest in comparing traditional face-to-face delivery with any one distance delivery method such as television-based, computer-based and telephone-based.

Table 3. Types of assessments needed to serve decisions relative to how distance education is being implemented; How important is it to...
VERY IMPORTANT (_ = 1.000 to 1.499a)
1.128 ...identify barriers and incentives for using distance delivery by the learner?
1.134 ...identify what makes collaborative distance education offerings successful?
1.142 ...identify structures that are in place that are blocking delivering education via distance?
1.181 ...compare learner reactions of distance delivery to traditional classroom delivery in terms of being able to apply and use the content?
1.220 ...identify barriers and incentives for implementing delivery by the instructor?
1.224 ...identify processes distance education instructors are using to customize the educational experience for students along with the benefits and drawbacks of these process?
1.233 ...assess learners perceptions about the advantages and disadvantages of using various technologies?
1.296 ...compare learner reactions of distance delivery to traditional classroom delivery in terms of knowledge or skills acquired?
1.344 ...identify incentives and barriers for incorporating multiple technology use by the learners?
1.364 ...identify factors that are encouraging or discouraging educators to work together for creating new educational structures (i.e., models, systems, etc.)?
1.389 ...compare learner reactions of distance delivery to traditional classroom delivery in terms of learning styles of the learners?
1.441 ...identify incentives and barriers for incorporating multiple technology use by the instructor?
QUITE IMPORTANT (_ = 1.500 to 1.999a)
1.519 ...identify factors that are encouraging or discouraging educators to work together for program development and delivery?
1.526 ...compare learner reactions of distance delivery to traditional classroom delivery in terms of interaction with other learners in the course/workshop?
1.529 ...compare learner reactions of distance delivery to traditional classroom delivery in terms of interaction with the instructor?
1.611 ...identify factors that influence how quality is judged from the perspective of different audiences such as different institutions, accreditation agencies, faculty, general public, and students?
1.616 ...compare learner reactions of distance delivery to traditional classroom delivery in terms of feeling a part of a learning community?
1.617 ...identify barriers and incentives for implementing delivery by the technical support people?
1.637 ...identify indicators that are useful for controlling quality and maintaining rigor?
1.655 ...identify what is being done to control quality and maintain rigor?
1.677 ...identify incentives and barriers for incorporating multiple technology use within the infrastructure.
1.850 ...identify barriers and incentives for implementing delivery by the administration?
1.931 ...identify how instructors teaching a collaborative offering interact with learners from other institutions?
SOMEWHAT IMPORTANT (_ = 2.000 to 2.499a)
2.042 ...research how instructors are transferring skills from using traditional campus-based instruction to using distance-based instruction?
2.288 ...compare traditional face-to-face delivery with multi-media processes?
2.289 ...compare the benefits and drawbacks of various administrative models such as?
2.367 ...compare the benefits and drawbacks of various administrative models such as lead instructor with instructors-of-record at sites?
2.410 ...compare traditional face-to-face with computer-based (Lotus Notes, etc.)?
2.467 ...compare the benefits and drawbacks of various administrative models such as facilitator licensed processes?
NEITHER IMPORTANT NOR UNIMPORTANT (_ = 2.500 – 2.999a)
2.664 ...compare traditional face-to-face delivery with television-based?
3.120 ...compare traditional face-to-face delivery with telephone-based?
a Scale: 1 = Very important 2 = Important 3 = Neither important nor unimportant 4 = Unimportant 5 = Very unimportant

Findings Related to Outcome Needs

Research and evaluation needs categorized as "outcomes needs" are important in judging program attainments. They relate to the outcomes, or impacts that distance education is having.

Distance educators felt that assessing outcomes in formal higher education is very important, as are outcomes reached in K-12. There is less interest in assessing outcomes of non-formal professional and personal growth workshops (see Table 4).

Documenting participation and completion rates was viewed as important, as was identifying effective and fair teacher evaluation processes.

There is less interest in studying the maturation of distance instruction, and in conducting meta-analysis of the research on different types of distance delivery modes.

Table 4. Types of evaluations needed to serve decisions relative to the outcomes of distance education; How important is it to...
VERY IMPORTANT (_ = 1.000 to 1.499a)
1.000 ...assess how well instructional outcomes are reached in K-12?
1.097 ...assess how well instructional outcomes are reached in formal higher education courses?
QUITE IMPORTANT (_ = 1.500 to 1.999a)
1.599 ...document participation and completion rates?
1.705 ...assess how well instructional outcomes are reached in non-formal professional and personal growth workshops (i.e., seminars, video-conferences, etc.)?
1.791 ...identify effective and fair teacher evaluation processes?
SOMEWHAT IMPORTANT (_ = 2.000 to 2.499a)
2.250 ...study the maturation of distance instruction as instructors design and deliver educational processed for distance delivery?
2.467 ...conduct a meta-analysis of the research on different types of distance delivery modes?
aScale: 1 = Very important 2 = Important 3 = Neither important nor unimportant 4 = Unimportant 5 = Very unimportant

Findings Related to General Education

Distance educators felt it’s very important to assess how to include training on adult education theory and practice so distance education instructors become more action-oriented, to study how the change process is managed by students and to identify how distance education can facilitate lifelong learning (see Table 5).

Creating a long-term vision about educational systems is very important, as is integrating distance education into strategic plans. Studying how the change process is managed by faculty is seen as more important than studying how it is managed by administration. Determining if perceived needs are met is also important.

Table 5. Research needed about education in general; How important is it to...
VERY IMPORTANT (_ = 1.000 to 1.499a)
1.104 ...assess how to include training for faculty to learn about adult education theory and practice so the distance education instructors become more action oriented?
1.267 ...identify if distance education creates changes in the learning process? If so, how?
1.308 ...study how the change process is managed by students?
1.359 ...identify how distance education can facilitate lifelong learning?
1.440 ...create long-term vision about educational systems that will serve Nebraskans in 2020?
UPPER SIDE OF IMPORTANT (_ = 1.500 to 1.999a)
1.773 ...assess how institutions are integrating distance education into their strategic plan?
1.879 ...study how the change process is managed by faculty?
1.939 ...find methodologies that will help determine if the education programs meet felt needs?
LOWER SIDE OF IMPORTANT (_ = 2.000 to 2.499a)
2.043 ...study how the change process is managed by administration?
a Scale: 1 = Very important 2 = Important 3 = Neither important nor unimportant 4 = Unimportant 5 = Very unimportant

Summary

In summary, distance educators feel that very important research and evaluation needs for distance education should focus on (a) cooperation and collaboration among institutions, (b) designing the educational experience for the distance learner, (c) teacher preparation, and (d) educational outcomes.

In the area of planning, major interests focused on identifying ways to promote better cooperation among institutions so both technology utilization and distance education programming can be coordinated more effectively. As the coordination and cooperation for programming and technology use improve, it is important to identify the impact the improved strategy has on learners. In the area of structuring, major interests appear to concentrate on effective strategies for successful distance learning experiences; the support needed from the educational institution; and, training needs for distance education teachers. In the area of implementation, the main themes focused on learner issues, instructional delivery, administration and quality control. In the area of outcome needs, major interests focused on assessing outcomes in formal higher education courses and K-12. There is less interest in assessing outcomes of non-formal professional and personal growth workshops. Documenting participation and completion rates were viewed as important, as was identifying effective and fair teacher evaluation processes. In the area of general education, it is seen as very important to assess how to include training on adult education theory and practice so the distance education instructors become more action-oriented, to identify if distance education creates changes in the learning process, to study how the change process is managed by students, and to identify how distance education can facilitate lifelong learning. Creating a long-term vision about educational systems is very important as is integrating distance education into strategic plans.

As more and more educators teach via distance, research and evaluation requirements will need to be monitored. Institutional assistance can then be adapted as technologies change and faculty share their experiences throughout different educational systems. The most supportive aspects can then be implemented so goals for distance education can be realized.


References

Carl, D. L. (1991). Electronic Distance Learning: Positives Outweigh Negatives. T.H.E. Journal, 18, 67-70.

Clark, T. (1993). Attitudes of Higher Education Faculty Toward Distance Education: a National Survey. The American Journal of Distance Education, 7 (2), 19-33.

Collis, B., Veen, W. & DeVries, P. (1993). Preparing for an Interconnected Future: Policy Options for Telecommunications in Education. Educational Technology, 33 (1), 17-24.

Distance Learning Administration Conference (2000). Conference proceedings. State University of West Georgia, Carrollton, Georgia.

Distance Learning 99 (1999). Proceedings from the 15th Annual Conference inDISTANCE Teaching & Learning. University of Wisconsin-Madison.

Hanson, D., Maushak, N. J., Schlosser, C. A., Anderson, M. L., Sorensen, C. & Simonson, M.. (1996). Distance Education: Review of the Literature (2nd ed.). Research Institute for Studies in Education. Ames: Iowa State University.

Holmberg, B. (1986). Growth and Structure of Distance Education. Croom Helm: New Hampshire.

Holmberg, B. (1987). The Development of Distance Education Research. The American Journal of Distance Education, 1 (3), 16-23.

International Conference on Learning With Technology (2000). Educators examine impact of technology on learning reported in the Lincoln Journal Star, June 12, 2000. Lincoln, NE.

McKillip, J. (1987). Need Analysis: Tools for the Human Services and Education. Newbury Park, CA: Sage Publications.

Miller, G. (1993). American Independent Study, News from the NUCEA Division of Independent Study. Fall 1993.

Miller, G. & Carr, A. (1997). Information and Training Needs of Agricultural Faculty Related to Distance Education. Journal of Applied Communications, 81 (1):1-9.

Moore, M. (1994). Administrative Barriers to Adoption of Distance Education. The American Journal of Distance Education, 8 (3), 1-4.

Olcott, D. Jr. & Wright, S. J. (1995). An Institutional Support Framework for Increasing Faculty Participation in Postsecondary Distance Education. The American Journal of Distance Education, 9 (3): 5-17.

Rockwell, S. K., Schauer, J., Fritz, S. M., & Marx, D. B. (1999). Incentives and Obstacles Influencing Higher Education Faculty and Administrators to Teach Via Distance. Online Journal of Distance Learning Administration, 2 (3). Available: http://www.westga.edu/~distance/rockwell24.html

Rockwell, S. K., Schauer, J., Fritz, S. M., & Marx, D. B. (in press). Faculty Education, Assistance and Support Needed to Deliver Education via Distance. Online Journal of Distance Learning Administration.

Roger, D. L. & Whetten, D.A. (1982). Interorganizational Coordination: Theory, Research, and Implementation. Ames: Iowa State University Press.

Schlosser, G. A. & Anderson, M. L. (1994). Distance Education, Reviews of the Literature. Washington, D.C.: Association for Educational Communications and Technology.

Stufflebeam, D. L. (1971). The Relevance of the CIPP Evaluation Model for Educational Accountability. Journal of Research and Development in Education, 5, 19-25.

Thach, L. & Murphy, L. (1994). Collaboration in Distance Education: from Local to International Perspectives. The American Journal of Distance Education, 8 (3), 5-21.

United States Distance Learning Association. (1998). Distance Learning Definition [online], 13 July. Available: www.usdla.org/Pages/define.html

Verduin, J. R. Jr. & Clark, T. A. (1991). Distance Education: the Foundations of Effective Practice. San Francisco: Jossey-Bass.

APPENDIX A – Formulas for calculating an adjusted mean.

Adjusted score for round 2. The adjustment factor for round two mean scores was calculated according to the following formula:

where
Mean = Mean for item from round one
Probit = the inverse of the standard normal cumulative distribution function
S = Standard Deviation for item from round one
SE = Standard Error = (Square root [nmore + nless]) (S / square root
[nround 1 + nround 2])
and
Q = Absolute Value( PD) + 0.5
PD = (number more important - number less important)/total n from round two
For cases where PD was greater than or equal to 50 percent, the value of Q was set at 1, yielding a Probit value of 1.96 (the point two standard deviations above the mean in a normal distribution).
 

Final scores. The adjusted round two scores were again re-adjusted up or down to produce final scores using a modification of the round one formula.

Final Mean = Adjusted Mean +/- Probit(Q) x SE
where
Adjusted Mean = Mean for item from round two
Probit = the inverse of the standard normal cumulative distribution function
S = Standard Deviation for item from round one
SE = Standard Error = (Square root [nmore + nless]) (S / square root [nround 1 + nround 2 + nround 3])
and
Q = Absolute Value( PD2) + 0.5
PD2 = (number more important - number less important)/total n
from round three
For cases where PD2
was greater than or equal to 50 percent, the value of Q was set at 1, yielding a Probit value of 1.96 (the point two standard deviations above the mean in a normal distribution).
 

Online Journal of Distance Learning Administration, Volume III, Number III, Winter2000
State University of West Georgia, Distance Education Center
Back to Journal of Distance Learning Administration Contents