Effectiveness Evaluation

2004-2005

Distance & Distributed Learning

Mission

The Distance & Distributed Education Center is a university-wide function at UWG which serves to develop and enhance the university's ability to deliver education to students at remote locations, and to meet institutional distance learning goals. Through intercampus sharing of resources, the Distance and Distributed Education Center facilitates collaboration among university colleges and departments to deliver quality distance instruction, faculty and student services, and initiatives.

Goals

Goals and functions of this department mirror the institutional distance learning goals :

Work with faculty to plan and create distance learning environments that encourage and support excellence in a personal environment
In collaboration with other campus and state departments, maintain the human and technical resources and network infrastructure necessary to successfully support and deliver distance and distributed learning.
Ensure that academic and student services are appropriate to meet the needs of distance and distributed learners.
Conduct continuous evaluation of distance and distributed learning and support services to ensure the advancement of the University's mission and adherence to quality standards.
Support research, scholarship, and creative endeavors which promote knowledge of distance learning.

These goals are reviewed annually in March, by the Distance and Distributed Education Steering Committee , and revised as appropriate.

Statement of Outcomes, Processes to Assess These Outcomes, and Assessment Results Where Appropriate

Goal 1. Plan and create distance learning environments that encourage and support excellence in a personal environment . Accomplishing this goal will ensure that:

Student and faculty satisfaction with distance and distributed education courses is high.

Fall 2004 evaluations showed that 89% of distance students agreed that their attitude toward online learning was positive at the end of the term. 86% agreed in Spring 2004. 93% in Summer . Fall and Spring showed a 11% or greater increase from the 2003 semesters in this area.
79% of students (Fall 2004) said they'd like to take more courses in future mostly online. (Spring 2004 - 74%; Summer 2004 – 85%). All semesters showed a 9% or greater increase from 2003 semesters in this area.
66% (Fall 2004) said they'd like to take courses in future completely online. (Spring - 61%; Summer – 67%) Fall and Summer semesters showed an increase of 7% or greater increase in this area from 2003 semesters.
96% (Fall 2004) said that their instructor was positive about the online component of their course. (88% in Spring 2004; 94% in summer). All semesters showed a 7% or greater increase in this area from the 2003 semesters.
The number of faculty using distance technologies continues to increase. FY05 totals were 411 faculty , compared to 376 in FY04, and 319 in FY03.

Student retention in distance and distributed education is comparable to that of traditional courses.

Fall 2004 retention for Distance students was 88.7% for distance students, compared to 89.5% for non-Distance students. Spring 2004 retention for Distance students was 85.9% compared to 88.9% for non-Distance students. Fall 2003 retention for Distance students was 85.2% compared to 90.1%

Students enrolled in distance courses have access to student services.

The UWG Online Connection ( http://www.westga.edu/~online/ ) provides easy web access for distance students to access student services.
February 2005 phone survey indicated that less than 5% of students felt there needed to be improvements for distance students in the areas of advisement, registration process, and orientation. This is an improvement over the 8% in the 2004 phone survey.

Student learning outcomes are comparable to those in traditional courses. (ex. http://coe.westga.edu/mit/outcomes/index.html )

These are generally the same as traditional and are evaluated on a departmental basis.

Interaction among student-faculty, and student-student are at least as high as in a traditional course.

Open-ended responses from written surveys Fall 2004, Summer 2004, and Spring 2004 surveys indicate that students appreciate the ability to interact at flexible times with faculty and other students. For Fall 2005, 90% of respondents agreed that having the flexibility to contribute to class discussions outside the classroom on my own time was valuable to them.
February 2005 phone survey showed that only 29% reported less interaction in DL course with instructor and other students. This is down from 40% in February 2004.

Faculty demonstrate competence in developing distance courses whose academic standards and student learning are the same as those for other courses delivered.

All faculty MUST complete technical and pedagogical training prior to teaching a distance course.
Academic standards and student learning are evaluated on a departmental basis. The DDEC reviews student surveys, class by class, to assure that overall student satisfaction with the faculty and the course are adequate. Since 1998, there have only been two cases in which the faculty competence seemed questionable based on student complaints. These were referred to the appropriate department head and dean. Both faculty members have since left the institution.

The number of courses developed and offered through distance media meets the demand of the region's students.

The phone survey of February 2005 indicates that 35% of distance students believe there are not enough distance courses. This is down from 44% in 2004.However, in 2004 only 14% of students were uncertain, while in 2005 that number climbed to 36%.
Although this is improving, a number of students mentioned in their open-ended responses on the phone survey that they need more online courses.

Assessment methods: Written student surveys at end of each term, annual focus group with distance students, informal discussions with Distance Learning Steering Committee and distance faculty. Student learning outcomes are assessed by academic units offering instruction. See: http://www.westga.edu/~distance/data/eval/

Goal 2. Maintain the human and technical resources and network infrastructure necessary to successfully support and deliver distance and distributed learning. Accomplishing this goal will ensure that:

Faculty are trained and prepared to teach distance and distributed courses.

All faculty MUST complete technical and pedagogical training prior to teaching a distance course.

Students are able to receive immediate technical assistance through telephone or email.

The DDEC staff provide immediate response to technical questions from students weekdays from 8 am until 8 pm weekdays. Students may contact a statewide support line after hours.
Helpline satisfaction surveys indicate that a 98 percent students in July 2004 – June 2005 ranked the amount of time it took them to get help at least an 8 on a scale of 1 to 10.

Students and faculty are able to receive assistance through a central point-of-contact .

The DDEC provides a central point of contact for support for all UWG distance courses.

A variety of delivery methods are available.
- In addition to the primary online mode of WebCT , distance courses may also utilize GSAMS two-way videoconferencing, and Horizon Live for synchronous or archived delivery of lectures featuring voice and visuals, and Impatica for simple audio-visual online presentations.
Distance courses are easily accessible to a growing number of students and potential students.

The total number of online courses offered in FY05 was 292, compared to FY04 total of 243, compared to 132 in 2003, and 144 in 2002.

Downtime for courses residing on UWG servers is non-existent or minimal, with backup plans in place and utilized as needed.

Except for scheduled maintenance, the WebCT system has functioned without interruption since January 2002. Faculty use WebCT and Horizon Live as a backup to GSAMS.
There have been extensive problems with downtimes for WebCT Vista eCore courses residing on a statewide centralized server. Plans to move all UWG courses to this centralized system are presently being reevaluated.

Assessment methods: Written student surveys at end of each term, annual focus group with distance students, informal discussions with Distance Learning Steering Committee and distance faculty, departmental annual self-review. See: http://www.westga.edu/~distance/data/eval/

Goal 3. Ensure that academic and student services are appropriate to meet the needs of distance and distributed learners. Accomplishing this goal will ensure that:

Each distance course or program provides students with clear, complete and timely information on the curriculum, course and degree requirements, nature of faculty/student interaction, prerequisite technology competencies, technical requirements, availability of academic support service, financial aid resources and costs and payment policies. See: http://www.westga.edu/~distance/handbook.html

Information available on web and syllabi for all distance courses and programs.
Information also available in online student handbook, and other online points of access for students.

Students express satisfaction with the level of academic and student services received when taking distance and distributed courses.

Fall 2005 phone survey students expressed general satisfaction. The majority of complaints were course-specific.
90% of distance students surveyed in Feb. 2005 phone survey reported that they received prompt and courteous student support from West Georgia as a distance learner.

Students are aware of and utilize online resources available to them for academic and student support.

According to Fall 2004, Summer 2004, and Spring 2004, most students were either satisfied with support services, or said "did not apply," with the exception of Financial Aid where students mostly disagreed. "Did not apply" usually referred to career services, which are not needed by a large percentage of students.
Sixty-five percent of Fall 2004 distance students (up from 62% in Fall 2003) reported that they used library services, but 31% (Fall 2004) said they were unaware of library services available specifically for distance students (down from 39% in 2003). Sixty-four percent of Feb. 2004 phone survey students also said they used library services. This is up from 48% in Feb. 2003.
Phone survey indicated that as many as 77% of DL students get orientation from the DDEC staff or instructor rather than through online or printed information. DDEC has prepared kit for instructors to use in orientation for those who choose, in order to make sure that all services are explained.

Enrolled students have reasonable and adequate access to the range of student services and resources appropriate to support their learning.

Distance students have access to the range of student services and resources that traditional students do, and also special services such as support from the DDEC, and special services from the library. Information regarding services is available at www.westga.edu/~online

Assessment methods: Written student surveys at end of each term, annual focus group with distance students, informal discussions with Distance Learning Steering Committee and distance faculty, departmental annual self-review. See: http://www.westga.edu/~distance/data/eval/

Goal 4. Conduct continuous evaluation of distance learning and support services to ensure the advancement of the University's mission. Accomplishing this goal will ensure that:

Faculty use results of evaluations to improve courses.

All distance faculty must complete the Distance Evaluation Summary form, documenting what changes they will make in future distance courses based on their student surveys.

Distance learning staff uses results of evaluations to improve programs and services as a whole.

The DDEC staff reviews evaluations and completes an annual effectiveness evaluation each June. A staff retreat is also held each December to assess staff quality standards, issues and plan for the next year.

The technologies selected are appropriate to meet course or program objectives.

The DDEC selects technologies for campus use and support based on student need, recommendations from other institutions, and cost-benefit.
Individual departments and instructors select from combinations of the various institutional technologies ( WebCT , GSAMS, Horizon Live) based on the program and course objectives.

Documentation of evaluations for each course and the overall distance program is available and accessible.

Overall evaluations for student written surveys, focus groups, phone surveys, retention and other data is available at the DDEC website ( www.westga.edu/~distance/data/eval/
Raw survey data is also maintained by the DDEC and each department offering distance courses.

Assessment methods: Faculty summary of evaluations each term, written student surveys at end of each term, annual focus group with distance students, informal discussions with Distance Learning Steering Committee and distance faculty, departmental annual self-review. See: http://www.westga.edu/~distance/data/eval/

Goal 5. Support research, scholarship, and creative endeavors which promote knowledge of distance learning. Accomplishing this goal will ensure that:

Our journal, conference, and certificate programs maintain excellent reputations among distance learning administrators in the United States and worldwide.

The journal is required reading for many institutional programs, including University of Nebraska's doctoral program in Higher Education, is referenced in many papers and books.
The conference attracts a growing international audience of practitioners in the field.
Our online Distance Learning Certificate Program is in its 8th year; it has been featured in The Chronicle of Higher Education and has had 201 participants from 14 countries. Participants are from the private sector, as well as the public sector and the US Armed Forces.

Our Online Journal of Distance Learning Administration continues to increase in readership.

The average edition had 18,500 hits in 2004, up from 15,000 hits in 2003, up from 12,000 in 2002, and 8,000 in 2001.
In its eighth year of production, readers of our Online Journal of Distance Learning Administration span 29 countries. As of June 15, 2005, the journal had been accessed over 250,000 times.

UWG faculty conduct research to enhance distance courses at UWG and to provide scholarly information to their field .

UWG faculty and staff regularly present research at the DLA Conference and other conferences, including SITE, Educause , and other professional meetings. Many UWG faculty articles and books on distance learning are linked from the distance website.

Assessment methods: Certificate program and conference evaluations , readership data of journal , feedback from readers and participants .

Examples of Using of Assessment of Goals and Outcomes to Improve a Process

Delivery methods : Some faculty indicated the need to use multimedia in their courses beyond what WebCT offers, but did not need the complexity of the Horizon Live tool. The DDEC purchased additional Impatica licenses to enable faculty to quickly and easily narrate their Powerpoint slides and upload them to WebCT .
Faculty training: Training housecalls (Dr. D) were implemented in January 2004 in order to allow faculty to get just-in-time training to assist them in preparing for their distance courses. Approximately 80 housecalls were made in FY2005.

Department Condition

Students: Student satisfaction with distance and distributed courses continues to be high. In Fall 2004, retention for distance courses was 89 percent. Eighty-nine percent of students reported that they had a positive attitude about distance learning after taking a course in Fall 2004 . 77% of phone survey respondents reported that they never had problems accessing the course or utilizing the WebCT in any way; this is a significant increase over the 48% who reported never having had problems last year.

Course Offerings: The number of courses using WebCT has grown dramatically over the past five years, with this year's total estimated at an all-time high of 1318, up from 962 last year. The number of courses offered 90-100% via distance was 233, up from 190 one year ago, and 81 two years ago. However, enrollments in distance courses (51% or more of instruction time) dropped slightly to 4781, from 4884 last fiscal year. Total distance and distributed enrollment supported by the DDEC climbed to 36,655 from 27,504 last fiscal year.

Resources: As the demand for distance learning courses and services has increased, so has the need for human resources. Two student assistants, funded by Student Technology Fees, have continued to be vital in providing first-level telephone support for distance students.

Department Achievements and Highlights

Continued to make multiple operational improvements, particularly in the area of evaluation and student services for distance students, based on suggestions and recommendations from SACS.
Participated in BOR's testing, training, and planning for WebCT's new Vista product for online courses.
Conducted 157 one-on-one faculty and staff-training sessions for WebCT . The average training session lasts two hours.
The DDEC began offering “ housecalls ” to faculty who need assistance with distance learning in Spring 2004. Support staff were available to visit faculty in their offices on Monday and Wednesday mornings by appointment. Visits were limited to 45-minutes, and support was limited to providing assistance with WebCT course tools, demos of technologies and software (such as HorizonLive , Impatica , etc.) and basic course evaluation and improvement. Approximately 80 visits were made during FY2005.
Published four quarterly editions of the Online Journal of Distance Learning Administration. Average number of visits to the site each month has climbed to 18,600 ..
Delivered 36 Distance Learning group workshops, serving approximately 100 faculty .
Delivered two sections of the Distance Learning Certificate Program to participants from 18 states and 6 countries, including Chile , Canada , Japan , Germany , Malta , and the United States .
Began delivery of new certificate program, called the Distance Education Certified Trainer Program, at the request of many of the participants in the original certificate program who wanted to continue with another course. Developed by Melanie Clay, this 5-month course was offered twice during FY2005 to approximately 30 participants.
During FY ‘05, UWG continued to be number one among the eCore host institutions, in eCore enrollment. The University of West Georgia accounted for 38% of all students in eCore , averaging 400 students per term for the year.

Two UWG distance faculty developed a first-of-its-kind online core Chemistry courses, complete with lab components, now used in the state-wide online eCore program. The CHEM1211K course is the first of its kind to address the primary concern of how to implement a pedagogically sound and safe off-site lab experience that is truly comparable to that of a college chemistry lab situation, including the use of college-level chemistry lab materials. CHEM1212K has since followed.

The department presented nine distance faculty who had met certain criteria in terms of online instruction, training, and mentoring with Apollo Awards in Spring 2005.
A new staff member, Chris Fearrington was hired, and is developing online training modules for faculty, and continually enhancing the departmental website to respond to the needs of faculty and students. He also provides on-call telephone and email support to online students during weekends.

Effectiveness Evaluation

2004

2003

2002

Past Annual Reports:

SACS Self Study Response to SACS