Predictors of Engagement and Participation in an On-Line Course


Marc D. Miller, Ph.D. Augusta State University, mmiller@aug.edu

R. Kelly Rainer, Jr., Ph.D. Auburn University

J. Ken Corley, Auburn University

 

Introduction

The use of online learning materials is increasing in both academic and industrial training settings. While the advantages of online learning are exciting and numerous, the problems associated with successful implementation of an online learning program can detract from the educational experience of even the most motivated student. These problems include such factors as poor attendance, procrastination, feelings of isolation, and a general lack of structure in the course (Brown, 2001; Kulik, 1991; Fishman, 1999; Oliver, 1999; Olugbemiro et al., 1999; Young-Ju Joo et al., 2000, Wang and Newlin, 2000). These problems can limit the amount of participation and engagement with the course materials that are offered in the online environment.

In addition, the lack of face-to-face contact can create a lack of control within the teaching environment. This lack of control raises the question: Can instructors predict the amount of time that a student should spend in the learning spaces of the online environment to be successful? Hiltz (1994) points out that students may withdraw from an online course because they do not manage the time required to be successful in the course. That is, they put off performing the online activities until they are not able to complete the class within the allotted time frame.

This problem of time management may be affected by the medium as a whole. That is, if the student is uncomfortable with the delivery mechanism (i.e. the computer), then the student may spend less time engaging in online learning. Two primary research streams have explored the use and non-use of computer technology: the technology acceptance model (TAM) (Davis et al., 1989; Davis, 1989) and various renditions of the computer self-efficacy (CSE) constructs (Compeau et al., 1995, 1999; Gist , 1989; Gist et al., 1989; Murphy et al., 1989)

If these models can predict computer use and satisfaction, they may also be able to predict and explain participation and engagement with an online learning course which uses the computer as its primary access medium. Therefore, the purpose of this research is to empirically explore the constructs of the TAM and CSE as they relate to the use of computers in the delivery of online learning. Understanding these factors can help course designers, educators, and directors of corporate training and development develop and implement more effective online learning and training programs.

Predictors of Computer Use Technology Acceptance Model

Davis et al. (1989) first introduced the TAM as a theoretical extension of the Theory of Reasoned Action (TRA) (Ajzen & Fishbein, 1980). The TAM (see Figure 1) consists of three primary factors that predict computer use: Perceived Usefulness, Perceived Ease of Use, and Intention to Use.

Figure 1: Technology Acceptance Model

Perceived Usefulness (USE) refers to individualsí perceptions that the technology or computer application will help them perform their jobs better (Davis, 1989). That is, the application results in a positive use-performance relationship. In the context of online learning, this definition can be interpreted as whether or not actively participating in the online course would help the student achieve job or school related outcomes.

Perceived Ease of Use (EOU) refers to "the degree to which a person believes that using a particular system would be free of effort" (Davis, 1989 p. 320). That is, can the application be easily used for the intended purpose? Ease of Use, in the context of online learning, can be interpreted as whether or not the learning tools and online learning modules are easy to work with in order for the student to participate and engage in the course materials.

Within the TAM, the constructs of usefulness and ease of use predict the attitudinal component of Intention to Use. Intention to perform a particular behavior has been shown to be an effective predictor of the actual behavior itself (Ajzen & Fishbein, 1980). In the context of online courses, intention to use translates directly into intention to participate and engage in an online course.

The TAM has been tested within a wide variety of computer settings and has been shown to be a robust predictor of computer use (Szajna, 1996; Taylor & Todd, 1995; Thomson, et al., 1991; Venkatesh, 2000; Venkatesh & Davis, 1996, 2000). The TAM should also be a successful predictor of online course use. The TAM suggests that one of the reasons (among many others) that a person takes a computer-based online course, is that he or she perceives that the vehicle (the computer) for the course is both easy to use and useful. Therefore, the TAM should predict the intention to participate in an online course and subsequently the actual behavior.

Theory of Planned Behavior

Another model that has been shown to be useful in predicting computer usage is the Theory of Planned Behavior (TPB). The TPB is shown in figure 2 and includes constructs that do not appear in the TAM. Mathieson (1991) and Taylor & Todd (1995) indicated that the primary differences between the TPB and the TAM are the subjective norm and the perceived behavioral control constructs (Mathieson et al., 2001).

Figure 2: Theory of Planned Behavior

Subjective Norm (SN) refers to individualsí perceptions of othersí opinions of their behavior. SN has been shown to be a predictor of behavior (Bagozzi et al., 1992; Fishbein & Ajzen, 1975; Mathieson, 1991). In the context of online courses, subjective norm would be the amount of influence a personís superiors (i.e. employers, parents, or spouse) would have in influencing a choice to take an online course or training.

Perceived Behavioral Control (PBC) refers to an individualís perception of whether or not he or she can perform a particular behavior. Therefore, PBC would be a similar construct to Computer Self Efficacy (CSE) (Bagozzi et al., 1992). In the context of an online course, PCB would be defined as whether or not an individual could use the online tools to successfully participate and engage in the course. The CSE construct is covered in the next section of this paper.

The two attitudinal models (TAM and TPB) use multiple constructs to predict and explain behavior. Usefulness, Ease of Use, Subjective Norm, and Perceived Behavioral Control should have a positive relationship with the individualís behavior in an online course. The behavior of interest in this research is whether or not an individual participates and engages in an online course. The individualís behavior in the online course is operationalized by the amount of time he or she spends working on the course modules. Therefore, the following propositions are offered:

Proposition 1: Perceived Ease of Use will have a positive impact on the amount of time spent in an online course.

Proposition 2: Perceived Usefulness will have a positive impact on the amount of time spent in an online course.

Proposition 3: Subjective Norm will have a positive impact on the amount of time spent in an online course.

Computer Self Efficacy

Bandura (1986) defined self-efficacy as "Peopleís judgments of their capabilities to organize and execute courses of action required to attain designated types of performances. It is concerned not with the skills one has but with the judgments of what one can do with whatever skills one possesses." (p. 391) Self-efficacy, then, is an individualís belief that he or she can perform a particular task or behavior.

Computer self-efficacy (CSE) is defined as the judgment of oneís capability to use an information technology (Agarwal et al., 2000; Compeau et al., 1995; Gist, 1989; Gist et al., 1989). Marakas et al. (1998) suggested that CSE can be defined with two sub-constructs: General Computer Self-Efficacy (GCSE) and Task-Specific Computer Self-Efficacy (TCSE).

GCSE "refers to an individualís judgment across multiple computer domains" (Marakas, et al., 1998; p. 129). That is, GCSE refers to general feeling of competence with a computer. GCSE refers to an individualís perception of being able to use a computer without regard to a particular task, application, or software. Within the context of an online learning course, GCSE would refer to a general ability to use the computer to engage and participate in the course.

TCSE "refers to an individualís perception of self-efficacy in performing specific computer related tasks within the domain of general computing" (Marakas et al., 1998; p. 128). That is, TCSE refers to an individualís perception of performing specific task related computing behaviors. Therefore, in order for an individual to engage and participate in an online course, he or she must have feelings of self-efficacy toward the primary delivery mechanism, the computer. They must feel confident that they will be able to manipulate and use the mechanisms that control the course (Chau et al., 2001; Hill et al., 1987; Martocchio, 1993, 1994; Thompson et al., 1996; Webster & Martocchio, 1996; Young et al., 2000).

Computer self-efficacy has been shown to be a predictor of computer use and web-based distance education. Marakas et al. (1998) reviewed 40 studies on CSE and found that the majority of the research showed a relationship between CSE and some computer-related behavior. In addition, Agarwal et al. (2000) found that both TCSE and GCSE had a strong relationship with Ease of Use beliefs toward various software packages. These studies suggest that TCSE and GCSE would have a relationship with the overall engagement and participation in an online course. Finally, Lim (2001) found that CSE effectively predicted satisfaction in the web-based on-line course. Therefore, the following propositions are offered:

Proposition 4: General Computer Self-Efficacy will have a positive impact on the amount of time spent in an online course.

Proposition 5: Task Specific Computer Self-Efficacy will have a positive impact on the amount of time spent in an online course.

Study Setting and Subjects

The study setting was an online course on Microcomputer Applications. The course covered the complete Microsoft Office package (Windows, Word, Excel, Powerpoint and Access), in addition to the basic computer concepts of Hardware, Software, Data, Communications, People, and Procedures. The course is delivered completely online through interactive modules that were developed using Authorware. The students have the option of taking the course from home or from on-campus computer labs. The course also has an optional lab where the student can receive help from an instructor or graduate assistant. Each time the student logged into a course module, the system would keep track of the time spent in the course. Therefore, the studentís total time spent working in the interactive modules were logged and compiled. The students were informed at the beginning of the term that their time in the course modules would be measured throughout the semester.

To test the five propositions developed in this study, a survey instrument was given to sixty-six (66) students taking the course. At the end of the semester, the students were administered a survey which included measures for Perceived Ease of Use, Perceived Usefulness (Davis, 1989), Subjective Norm (Mathieson, 1991), Task Specific Computer Self Efficacy (Murphy, Coover, & Owen, 1988), and General Computer Self Efficacy (Compeau and Higgins, 1995). These instruments are presented in the Appendix.

The independent variable for this study is the construct of engagement and participation. For this study a surrogate measure of time spent in the course was used. This is a measure of the actual number of minutes spent working on the online modules of the course. While this measure is not a perfect measure, it is one that indicates intensity and duration of time spent performing the task of utilizing the online modules. This measure was correlated with the grade given in the course and a significant correlation (.91 p=.0001) was found.

Data Analysis

The data were analyzed by first examining the descriptive statistics and scale reliabilities of the measures. Table 1 shows the descriptive statistics for this sample. The data were then analyzed with a multiple regression model, using mean indicators of each the constructs. Table 2 indicates these results. Finally, the results of each proposition are presented in Table 3.

Results

Table 1 indicates that each of the five constructs demonstrated acceptable internal reliability. All Cronbach alphas were above the acceptable value of .70 (Nunnally, 1967). Therefore, the five scales were considered reliable measures of the intended constructs. Table 2 indicates that the general regression model is significant (F = 6.94, p<.001) and an adjusted R2 of .31. Finally, Table 3 indicates that the data analysis provides support for Propositions 1 and 2, while failing to provide support for Propositions 3, 4, and 5.

Table 1. Descriptive Statistics

Covariance Matrix Demonstrating Internal Reliability
  GCSE TSCSE EOU USE SN Chronbach's Alpha

GCSE

General Computer Self Efficacy

1.143         .91

TSCSE

Task Specific Computer Self Efficacy

0.875 1.094       .89

EOU

Ease of Use

1.038 1.197 1.531     .95

USE

Usefulness

0.695 0.915 1.001 1.053   .96

SN

Subjective Norm

0.673 0.892 0.905 0.871 1.910 .78

Table 2. Regression Analysis Results

ANALYSIS OF VARIANCE
  SS DF MEAN S F-RATIO P-VALUE
REGRESSION 5010243.459 5 1002048.6919 6.935 0.000
RESIDUAL 8668921.570 60 144482.0262    
TOTAL
13679165.03
65
     
      
Multiple R 0.6052                             
R-square 0.3663
Adjusted R-square 0.3135
F (5, 60) 6.9355
Prob > F 0.0000
Std. Error of Est. 380.1079
Durbin-Watson Stat 1.9753

Table 3. Does Data Analysis Support the Propositions?

Proposition

Measure

Coefficient (p-Value)

Supported

Proposition 1: Perceived Ease of Use will have a positive impact on the amount of time spent in an online course.

Ease of Use (Davis, 1989)

421.40 (0.0002)

Yes

Proposition 2: Perceived Usefulness will have a positive impact on the amount of time spent in an online course.

Usefulness (Davis, 1989)

333.36 (0.0005)

Yes

Proposition 3: Subjective Norm will have a positive impact on the amount of time spent in an online course.

Subjective Norm (Mathieson, 1991)

-13.51 (0.7642)

No

Proposition 4: General Computer Self-Efficacy will have a positive impact on the amount of time spent in an online course.

GCSE (Compeau & Higgins, 1995)

91.90 (0.2172)

No

Proposition 5: Task Specific Computer Self-Efficacy will have a positive impact on the amount of time spent in an online course.

TSCSE (Murphy, Coover & Owen, 1987)

15.031 (0.9197)

No


Conclusions

The data supported the propositions centered on the TAM. Perceived Ease of Use and Usefulness both had a significant, positive relationship on the amount of time spent in the course. That is, if a student perceived the delivery mechanism (the personal computer) to be Easy to Use and Useful then they were more likely to become engaged in the course as measured by time spent in the online modules. Prior research has found that Perceived Ease of Use and Usefulness can be affected by training. Therefore, online learning instructors may find that training on the PC before the start of an online course might help the student become more engaged in the online course. In addition, if a student were to rate the perceived usefulness of the PC relatively low, then the student may be advised that they should rethink taking this course.

The propositions centered on the TPB and TRA were not supported. The constructs of Subjective Norm, General Computer Self-Efficacy, and Task Specific Computer Self-Efficacy did not have a significant impact on engagement in the online course. One possible explanation for these results is that the students had no other choice but to have some level of computer activity within the course. Therefore, their feelings of whether or not otherís opinions of their engagement in the online course were irrelevant to their overall time spent on the course. In addition, whether or not students believed their level of self-efficacy to be high or low, they had to use the personal computer to complete the course

One limitation to this study is that the measure of engagement used in this course was limited to time spent on the computer. While this is an important measure, it is not the only measure of engagement. Other measures such as quantity and quality of bulletin board writings, feedback, and general satisfaction with the online course may be important avenues for future research. In addition, the final outcome of the grade in the course should be factored into the engagement quotation. Finding the relationship to other measures of engagement may provide support for the propositions that were not supported in this study. That is, more specific measures of engagement may help to more clearly define the factors that lead to more active participation in the virtual classroom. This is definitely an area for follow-up and future inquiry.


References

Agarwal, R., Sambamurthy, V., & Stair, R. (2000). Research report: The evolving relationship between general and specific computer self-efficacy - An empirical assessment. Information Systems Research, 11(4), 418-430.

Ajzen, I. & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: PrenticeHall.

Bagozzi, R.P., Davis, F.D., & Warshaw, P.R. (1992). Development and test of a theory of technological learning and usage. Human Relations, 45, 659-686.

Bandura, A. (1977). Social learning theory. Englewood, NJ: Prentice-Hall.

Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37, 122-147.

Brown, K.G. (2001). Using computers to deliver training: Which employees learn and why? Personnel Psychology, 54(5), 271-296.

Chau, P. (2001). Influence of computer attitudes and self-efficacy on IT usage behavior. Journal of End User Computing, 13(1), 26-34.

Compeau, D. R. & Higgins, C. A. (1995). Computer self-efficacy: Development of a measure and initial test. MIS Quarterly, 19(2) 189-211.

Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3) 319-340.

Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management Science, 35(8), 982-1003.

Fishbein, M. & Ajzen, I. (1975). Belief, attitude, intention and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley.

Fishman, B.J. (1999). Characteristics of students related to computer-mediated communications activity. Journal of Research on Computing in Education, 32(1), 73-77.

Gist M. (1989). The influence of training method on self-efficacy and idea generation among managers. Personnel Psychology, 42, 787-805.

Gist, M.E., Schwoerer, C.E., & Rosen, B. (1989). Effects of alternative training methods on self-efficacy and performance in computer software training. Journal of Applied Psychology, 74, 884-891.

Hill, T., Smith, N. D., & Mann, M. F. (1987). Role of efficacy expectations in predicting the decision to use advanced technologies: The case of computers. Journal of Applied Psychology, 72(2), 307-313.

Hiltz, S. R. (1994). The virtual classroom: Learning without limits via computer networks. Norwood NJ: Ablex Publishing Corp., Human-computer Interaction Series.

Kulik CC, Kulik JA. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 7, 75-94.

Lim, C.K. (2001). Computer self-efficacy, academic self-concept, and other predictors of satisfaction and future participation of adult distance learners. The American Journal of Distance Education, 15,2, 41-51.

Marakas, G., Yi, M., & Johnson, R. (1998). The multilevel and multifaceted character of computer self-efficacy: Toward clarification of the construct and an integrative framework for research. Information Systems Research, 9(2), 126-163.

Martocchio J.J. (1993). Employee decisions to enroll in microcomputer training. Human Resource Development Quarterly, 4, 51-69.

Martocchio J.J. (1994). Effects of conceptions of ability on anxiety, self-efficacy, and learning in training. Journal ofApplied Psychology, 79, 819-825.

Mathieson, K, Peacock, E., & Chin, W. (2001). Extending the technology acceptance model: The influence of perceived user resources. Database for Advances in Information Systems, 32(3), 86-112.

Mathieson, K. (1991). Predicting user intentions: Comparing the technology acceptance model with the theory of planned behavior. Information Systems Research, 2(3) 173-191.

Murphy, C.A., Coover, D., & Owen, S.V. (1989). Development and validation of the computer self-efficacy scale. Educational and Pyschological Measurement, 49, 893-899.

Nunnally, J. (1967). Psychometric Theory. New York: McGraw-Hill.

Oliver, R. (1999). Exploring strategies for online teaching and learning. Distance Education, 19(1), 81-88.

Olugbemiro J., Taplin, M., Fan, R., Chan, M. & Yum, J. (1999). Differences between low and high achieving distance learners in locus of control and metacognition. Distance Education, 20(2), 255-269.

Szajna, B. (1996). Empirical evaluation of the revised technology acceptance model. Management Science, 42, 85-92.

Taylor, S. & Todd, P. A. (1995). Understanding information technology usage: A test of competing models. Information Systems Research, 6(2), 144-176.

Thompson, R. L., Higgins, C. A., & Howell, J. M. (1991). Personal computing: Toward a conceptual model of utilization. MIS Quarterly, 14, 125-143.

Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research, 11(4), 342-365.

Venkatesh, V. & Davis, F. D. (1996). A model of the antecedents of perceived ease of use: Development and test. Decision Sciences, 27, 451-481.

Venkatesh, V. & Davis, F. D. (2000). A theoretical extension of the technology acceptance model: Four longitudinal field studies. Management Science, 46, 186-204.

Wang, A. and Newlin, M. (2002). Characteristics of students who enroll and succeed in psychology Web-based classes. Journal of Educational Psychology, 92,1,137-143.

Webster, J., and Martocchio, J. J. (1993). Turning Work into Play: Implications for Microcomputer Software Training. Journal of Management, 19(1), 127-147.

Young-Ju Joo, Mimi Bong, & Ha-Jeen Choi. (2000). Self-efficacy for self-regulated learning, academic self-efficacy, and Internet self-efficacy in Web-based instruction. Educational Technology, Research and Development, 48(2), 5-13.


Appendix A: Instruments

Task Specific Computer Self-Efficacy (Murphy, Coover, & Owen, 1988)

I feel confident entering and saving data (numbers or words) into a file.

I feel confident calling up a data file to view on the monitor screen.

I feel confident storing software correctly.

I feel confident handling a floppy disk correctly.

I feel confident escaping/exiting from a program or software.

I feel confident making selections from an on screen menu.

I feel confident copying an individual file.

I feel confident using the computer to write a letter or essay.

I feel confident moving the cursor around the monitor screen.

I feel confident working on a personal computer (microcomputer).

I feel confident using a printer to make a "hardcopy" of my work.

I feel confident getting rid of files when they are no longer needed.

I feel confident copying a disk.

I feel confident adding and deleting information from a data file.

I feel confident getting software up and running.

I feel confident organizing and managing files.

I feel confident understanding terms/words relating to computer software.

I feel confident understanding terms/words relating to computer hardware.

I feel confident describing the function of computer hardware (keyboard, monitor, disk drives, computer processing unit).

I feel confident troubleshooting computer problems.

I feel confident explaining why a program (software) will or will not run on a given computer.

I feel confident understanding the three stages of data processing: input, processing, output.

I feel confident learning to use a variety of programs (software).

I feel confident using the computer to analyze number data.

I feel confident learning advanced skills within a specific program (software).

I feel confident using the computer to organize information.

I feel confident writing simple programs for the computer.

I feel confident using the user's guide when help is needed.

I feel confident getting help for problems in the computer system.

General Computer Self-Efficacy (Compeau and Higgins, 1995)

I could complete the job using the software package...

... if I had seen someone else the using it before trying it myself.

... if I could call someone for help if I got stuck.

... if someone else had helped me get started.

... if I had a lot of time to complete the job for which the software was provided.

... if someone showed me how to do it first.

... if I had used similar packages before this one to do the same job.

Perceived Ease of Use (adapted from Davis. 1989)

I find it easy to get a personal computer to do what I want it to do.

My interaction with the personal computer is clear and understandable.

I find the personal computer to be flexible to interact with.

It is easy for me to become skillful at using the personal computer.

Perceived Usefulness (adapted from Davis. 1989)

Using a personal computer can improve my learning performance.

Using a personal computer can make it easier to learn.

Using a personal computer in my job can increase my learning performance.

I find a personal computer useful learning.


Online Journal of Distance Learning Administration, Volume VI, NumberI, Spring 2003
State University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Contents