Maximizing Learning Using Online Student Assessment



Patrice C. Boyles, Ed.D.
Chicago State University
pboyles@csu.edu



Abstract

As the technological age reaches its peak, so does the need to improve assessment for online instruction.  Assessment includes all activities that teachers and students undertake to get information that can be used to improve teaching and learning (Black and William,1998b). Assessment is a critical factor of the learning environment. The popularity of distance education and online assessment has forced educational institutions to revise delivery methods, reshape teaching methods and reevaluate learning environments. The transformation in delivery of instruction has consequently brought on the need to reassess how educational institutions are implementing online assessment. According to Allen and Seaman (2008), over 20% of all students took at least one online course in 2006; consequently this has brought more attention to the quality of online instruction. Today, educators are wavering in new territory and educational institutions are forced to adapt to an online environment and change curriculum to meet the needs of learners. The purpose of the study is to investigate pre-service teachers’ perceptions of online assessment and its impact on student learning.

Online assessment has many benefits. Like ice-cream, it comes in a wide array and can be encrusted in several arrangements. Instructors have the capability to create test-banks that include multiple-choice, true/false, completion, essay writing or matching. Reports are easily generated to allow instructors to preview grades, participation, and assessment results for the entire class or individual students at any time. Since many distance education courses are either asynchronous or synchronous, students benefit by having more flexibility.  

At this juncture, the effects of attitudes for online assessment vary. The complexity of teaching or taking online courses can be overwhelming for some and impossible for others. Successful online instructors incorporate a variety of instructional strategies and implement a variety of formative assessments to maximize learning benefits. Online assessments are relatively new because they have been emphasized in predominately face-to-face environments.               

Literature Review

Advances in technology have forced many educational institutions to move beyond face-to-face traditional assessment to one that expands more online curriculum. The goal is to increase student enrollment.  Distance Education is giving traditional instructional methods a run for their money (American Federation of Teachers, 2009). Pope (2006) stated, “3.2 million people took at least one online course last fall.” The numbers are increasing every year. This was affirmed again by Allen and Seaman (2010) report. Approximately 5.6 million students were enrolled in at least one online course in fall 2009.  Based on this, educational institutions have been required to prepare their students and instructors for online classes. 

Online instructors are utilizing assessment techniques and strategies to determine “what students are learning in the classroom and how they are learning” (Angelo and Cross, 1993: 41). Pecheone and Chung (2006) addressed performance assessment as part of the teaching practice. First, these assessments inform instructors on needed curriculum changes. Second, these assessments can contribute to the improvement of teacher quality. Curriculum embedded assessments are stressed more than ever before. These assessments occur throughout teacher preparation programs. Additionally, Pecheone and Chung reported that these assessments come in the form of case studies of individual students, lessons, unit plans, analysis of student work, and observation of student teaching. It is also believed these assessments can improve instructional practices. Beebee, Vonderwell and Boboc (2009) research studied emerging patterns in transferring assessment practices from face-to-face to online environments.

In addition to acquiring a new technology skill set, instructors using the online medium structure required more time spent streamlining course content, requirements, building test banks, and uploading resources. Instructors need to build a sense of community as they redefine pedagogy to meet the needs of learners in a virtual environment. This was affirmed by Woods and Ebersole’s (2010) research using “communal architect” or building community-building strategies.  A “communal architect” is someone who erects a communal scaffold for the purpose of community building (Woods and Ebersole, p2).

Method

As more educational institutions move towards online instruction, authentic online assessment has become the heart of every learning experience because it helps negate negative student learning outcomes.. Therefore, the purpose of this study was to investigate pre-service teachers’ perceptions of online assessment. The study was conducted at an urban university in the Midwest, where the majority of the students were African-American and all were enrolled in a technology class.

Overview of Design

This study incorporated the survey methodology to gain insight to student perceptions of online assessment. All students enrolled in a technology course were given the opportunity to complete the survey and descriptive and frequency statistics were conducted to analyze the data.

Procedure

The procedure used to conduct this study comprised of several phases: 1) the survey construction 2) sample selection 3) data collection and 4) data analysis. Students were given the option to participate in the study.  Prior to class, students were given the purpose of the study and insight to how more instructors are incorporating online assessment into their instruction because it provides immediate knowledge of results. It also provides instructors an opportunity to improve learning outcomes by maximizing instruction.

To protect the anonymity of the participants, students were asked not to identify themselves and to return the survey to the instructor’s office or mailbox. All students participating in the study completed a self-report questionnaire that gathered information on their perception toward online assessment. Participants were encouraged to reflect on their experience with online assessment.  To ensure validity and reliability of the study, students were reminded of the theoretical orientation of the study and informed they will be provided with periodic updates and interpretations of the data throughout the study to determine if the results are plausible.

Survey Construction

The researcher developed a self-constructed survey to gather information on students’ perceptions of online assessment. The instrument consisted of two-page questionnaire which was divided into three sections: 1) perceptions of online assessment 2) general comments and suggestions and 3) demographic information.

The first section consisted of ten questions rated on a Likert-type scale where the participants selected strongly agree to strongly disagree. These questions focused on participants’ preference and use of online assessment. Participants reflected whether online assessment improved their pre-service teaching performance. Participants were also asked if online assessment enhanced their learning experience and if it should be required for all classes. To ascertain if more training is needed, participants were asked if they perceived instructors used online assessment effectively and if additional training was needed for faculty. To determine technology integration into curriculum planning, participants were asked if they plan to implement online assessment into their instruction after graduation. Participants were also asked if online assessment is an effective teaching method used in education courses.

Section two section of the survey was left for general comments from the participants. The researcher was soliciting qualitative data which could be used to improve the use and effectiveness of online assessment at the university. The last section allowed participants to provide some general demographic information. (e.g. area of concentration, grade level, gender, age range). This information could be used to plan future online courses.

Data Collection

As the surveys were returned, the researcher sought for potential themes from analyzing and sorting the data. The data was entered and analyzed using statistical software to complete descriptive statistics to provide student perceptions of online assessment.  Findings were also shared with students to determine if data was acceptable.

 Participants

The basis for selecting the participants in the study was derived from identifying students who were enrolled in a technology course at a four-year, urban higher education institution in the Midwest. Students were informed about the purpose of the study and procedures as well as the risks and benefits. Students who volunteered for the study were given a survey instrument. The survey was distributed to 38 students. Twenty-three students agreed to participate or 60%. Nineteen students were female and four identified themselves as males. In relations of grade level, fourteen identified themselves as graduate students, three were seniors, two were juniors, one was a sophomore and the remaining identified themselves as freshmen.

Data Analysis

A phenomenological approach was used to analyze the experiences of the students as they reflected on their perception of online assessment. The researcher’s aim was to give students a voice to the online assessment process and gain an understanding of past experiences using online assessment. To ensure every student had the opportunity to participate in the research, students were reminded during a face-to-face session that they could still participate in the survey until the end of the term. 

Results

Of the 38 students selected for the sample, a total of 23 participated. This provided a final response rate of 60%. Of the students who participated in the survey, eighty-two (82%) were female. This coincides with the demographics of the student enrollment of the class and the university population. In 2007, the university reported 72% of its population were female and only 28% were male. This data validated the student population enrolled in the course. The demographics of the participants are shown below in Table 1.

Table 1: Participant Gender Demographics

Gender

No. Students

Percent

Female

19

82.6

Male

4

17.4

Total

23

100.0

Further analysis of the data revealed the majority of the students were over 25 years of age. Out of the 23 students who responded to the question regarding age, 12 or 52 % reported they were between the ages of 31 and 40. This data also is validated in the 2007 institutional report which indicated only 34% of the student population was between the ages of 22 and 29. Since this is a technology course, many students who enroll in this course are interested in learning how to integrate technology in a variety of majors. Subsequently, the majority of the participants did not identify their area of concentration; however 21% identified their majors as elementary; 13% early childhood; 17% as career and technical education; and 8% as special education. The area of concentration is shown below in Table 2.

Table 2: Age and Area of Concentration

Area of Concentration

Age Range

Total

 

21-30

31-40

41-50

 

Early Childhood

1

2

1

4

Elementary Education

1

3

1

5

Special Education

0

2

0

2

Career and Technical Education

2

0

2

4

Other

2

5

1

8

Total

6

12

5

23

Preference Using Online Assessment

Of the students who participated in the survey, an overwhelming 95% agreed they preferred some use of online assessment. Many students prefer the use of online assessment so that they can receive immediate feedback from instructors, and have the opportunity to work in an asynchronous learning environment. The preference of online using online assessment is clearly shown in Table 3.

Table 3: Preference using Online Assessment.

Rating

Frequency

Percent

Disagree

1

4.3

Mildly agree

6

26.1

Agree

12

52.2

Strongly agree

4

17.4

Total

23

100.0

Enhanced Learning Experiences

Because the researcher was interested in learning if online assessment impacts learning, participants were asked if online assessment enhanced their learning experiences. The majority of the students, 91%, agreed online learning enhanced their learning experiences. Students have the capability of using tools such as threaded discussions, forums, e-mail, boards, and chats. This data was reflected in Table 4 shown below. 

Table 4: Online Assessment Enhancing Learning Experience

Rating

Frequency

Percent

Disagree

3

13.0

Mildly agree

5

21.7

Agree

10

43.5

Strongly agree

5

21.7

Total

23

100.0

Instructors’ Usage of Online Assessment

While more instructors are revising their delivery methods to an online course platform, some instructors and students are having difficulty adapting to an online environment.  Of the twenty-three students who participated in the survey, over 95% perceived to some degree instructors who use online assessment, use it effectively. This could be attributed to the implementation of best practices and training being incorporated into online courses. Instructors are encouraged to foster a climate of learning in a window without walls; and adapt instruction to allow every student the opportunity to express them and offer deeper discussion of topics. Students are encouraged to share experiences and suggestions related to issues being discussed. Instructors are also encouraged to develop social bonds online and switch their roles from content transmission to content acquisition while keeping students on task with relative topics and promoting cooperative learning.  Participants responses to usage of online assessment are shown in Table 5 below.

Table 5: Instructors who use online assessment, use it effectively                                                                                                                       

Rating

Frequency

Percent

Disagree

1

4.3

Mildly agree

10

43.5

Agree

7

30.4

Strongly agree

5

21.7

 Total

23

100.0

Need of Online Instruction

While some pre-service teachers, 21% do not agree additional instruction is needed using online assessment; a majority of the students, 79%, perceive additional instruction is needed. The complexity of taking an online course can be overwhelming. Students must be able to identify barriers prior to enrolling in an online course. One of the first barriers is that of technology itself. Many students who enroll in online courses do so without proper preparation. This could confirm the results of additional instruction responses as shown in Table 6.

Table 6: Additional instruction is needed using online assessment

Rating

Frequency

Percent

Strongly disagree

2

8.6

Disagree

3

13.0

Mildly agree

4

17.4

Agree

10

43.5

Strongly agree

4

17.4

Total

23

100.0

Usage of Online Assessment

To examine the usage of online assessment, participants were asked if online assessment should be used in all classes; should all pre-service teachers experience it, and if it is an effective teaching method used in education courses. Over half of the participants, 78%, agreed online assessment should be used in all classes. All respondents, 100% agreed all pre-service teachers should experience online assessment, and it is an effective method used in education courses. A high number of respondents also agreed online assessment is used effectively as illustrated in Table 7.

Table 7: Instructors who use online assessment, use it effectively

Rating

Frequency

Percent

Disagree

1

4.3

Mildly agree

10

43.5

Agree

7

30.4

Strongly agree

5

21.7

Total

23

100.0

Future Implementation Using Online Assessment

This preliminary research suggested that over 95% percent of the respondents agreed to some extent that they plan on implementing online assessment into their instruction after they graduate. These factors suggest that future teachers are becoming aware of the latest technologies and pedagogical methods to maximize instruction in an online environment by using online assessment.  They are also cognizant of the training that is required prior to initiating an online course.  Prior to graduation, pre-service teachers are obtaining a first-hand look at what online assessment entails. The impact of technology integration and online assessment is clearly identified in Table 8 shown below.

Table 8: Future Plans using Online Assessment

Rating

Frequency

Percent

Disagree

1

4.3

Mildly agree

3

13.0

Agree

10

43.5

Strongly agree

9

39.1

Total

23

100.0

Discussion and Conclusion

Insight was gained into pre-service teacher perceptions on how online assessment is used and how it impacts teaching and learning. The findings from this preliminary study can be divided into three discrete yet equally important outcomes associated with teaching and learning.  

This study showed that there is a definite correlation between graduate course work and online instruction. The majority of the students participating in the survey were graduate students who plan to implement online assessment into their instruction upon graduation. This sets a roadmap for educational institutions to aggressively prepare for online instruction and assessment that is both flexible and authentic. Data from this preliminary study could also be used to explore attitudes of other pre-service professionals and help define parameters for further studies.

The first outcome related to pre-service teacher preference using online assessment. Responses clearly showed that the majority of the students prefer online assessment. The findings revealed how online assessment has enhanced their learning experiences and improved their pre-service teaching performance.The study also suggested that the majority of the students planned to implement online assessment into their instruction after graduation.

The second outcome related to instruction. While more educational institutions are developing new strategies to develop online curriculums and assessments, results show that more training is needed for the instructor. There is always need for improvement. Instructors are learning to assess themselves as learners as well as their students. “People who rely merely on a few learning strategies are likely to over-use them and apply them in inappropriate situations” relates Roth, (1997, p. 2).

The third outcome related to allowing the pre-service teachers have a voice in the online process. This section allowed respondents to make comments of online assessment and how it is used in the course. Two students provided narrative feedback on online assessment.

One respondent said, “All teachers at this university need training on how to use Live Text.” A different respondent said, “Chicago Public Schools has started an on-line assessment called (Scantron).” The responses from these students clearly indicated they are aware of movement towards online assessment.

Limitations and Future Considerations

This preliminary study had two major limitations: first, a limited sample. Data was obtained from a small percentage of students enrolled in a technology course in fall 2009. Participation in surveys is contingent on participants’ interest in the subject matter. Participants may refuse to participate in surveys due to past experience with previous surveys. The use of a computer generated survey could have generated more responses and limited the amount of time of the respondents. The second limitation involved setting. This study took place during an evening and weekend technology class. Participants may feel they were overburdened and did not have sufficient time to complete the survey.

When educational institutions elect to implement online learning, it is important for them to recognize the impact of online assessment. This study suggested that graduate students, in particularly prefer online assessment. It also suggested that pre-service teachers who are exposed to online assessment, plan to implement it in their future classrooms. The findings of this study also revealed online assessment has enhanced pre-service teachers learning experiences. A broader version of this study would include colleges and universities who offer online assessment exclusively for the graduate programs.      


References

Allen, I. E. & Seaman, J. (2008) Online nation: Five years of Growth in Online Learning. Needham, MA: Sloan Consortium.

Allen, E. I., and Seaman, J. (2010) ’Class Differences:  Online education in the United States’,2010, Report from the Sloan Consortium, Retrieved December 28, 2010, from [Online], Available: http://sloanconsortium.org/publications/survey/pdf/class_differences.pdf

American Federation of Teachers (2009). Distance Education is here to stay, On Campus, 28(4), 16.

Angelo, T.A. and Cross, K.P. (1993). Classroom assessment techniques: A handbook for college teachers, 2nd edition, San Francisco, CA: Jossey-Bass Publishers.

Beebe, R, Vonderwell, S. and Boboc, M. (2010). Emerging patterns in transferring assessment practices from f2f to online environments, Electronic Journal of e-Learning, 8(1), 1-12.

Black, P. and William, D. (1998b). Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80 (2): 139-148.

Pecheone, R. L. and Chung, R.R. (2006). Evidence in teacher education: the performance assessment for California Teachers (PACT), Journal of Teacher Education, 57(1), 22-32.

Pope, J. (2006, November 9). Number of students taking online courses rises. USA Today. Retrieved December 28, 2010 from http://www.usatoday.com

Roth, G. (1997). Helping adults learn how to learn. The Korean Journal of Lifelong Education,  3(1), 139-153.

Woods, R. and Ebersole, S. (2010). Becoming a communal architect in the online classroom-Integrating cognitive and affective learning for maximum effect in web-based learning, Retrieved on June 28, 2010 from http://www.westga.edu/~distance/ojdla/spring61/woods61.pdf


Online Journal of Distance Learning Administration, Volume XIV, Number III, Fall 2011
University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Contents