Planning the Development and Maintenance of Online Distance Learning Courses


Alison Clapp
FMS Graduate School
Alison.clapp@newcastle.ac.uk


Ashley Reynolds
FMS Graduate School
ashley.reynolds@ncl.ac.uk

Barry Bell
FMS Graduate School
barry.bell@ncl.ac.uk

Eleanor Lockhart
AGE Research Group
eleanor.lockhart@ncl.ac.uk

Gemma Todd
Academic Health Science Network
gemma.todd@ahsn-nenc.org.uk

Tracy Connell
FMS Graduate School
tracy.connell@ncl.ac.uk


Abstract

Development and maintenance of online distance learning Masters courses in a higher education institute Medical School Graduate School involves the interaction of an e-learning team with subject specialists, all of whom are time poor. To allow course development to proceed smoothly it must be a managed process. Challenges were revealed during an ethnography of the team; the ethnography narrative was used to study these further to answer the question ‘how can online course development be managed?’. Team interactions with subject specialists consist of relational working, leading to the challenges of course development being analysed using Activity Theory. This involved finding activity systems in course development and detailing where ‘çontradictions’ (difficulties) in the process occurred. A new model of working was developed containing a checklist to be followed prior to course development, providing a means of managing online course development and maintenance, to enable working with best use of time.

Introduction

In view of the increasing requirement for online distance learning (ODL), a sizeable amount of time and effort from faculty staff is required for development and maintenance of courses, be they programmes or single modules within programmes. It has long been known that there needs to be management of these processes (Care and Scanlon (2001)). However, there is no definitive method of doing so. Ideally, learning materials will have been designed using pedagogical theories according to the institutional quality assurance standards; technologies used are supportive of users, and timely development can be within budget. The use of project management software to streamline the process has been used (Abdous and He (2008)). These rely on an ideal world; ideal worlds are not the routine experience of this study’s authors. In the real world it is the authors’ experience that there is little strategic management of online distance learning (ODL) development. This paper’s authors are members of team developing ODL within a medical school Graduate School in a UK higher education (HE) institution.

A case study exemplifying the use of relational working between the e-learning team and subject specialists developing ODL master courses is described in this paper. The team took part in a collaborative self-ethnography, originally to ascertain requirements for faculty development for online learning and teaching, covered elsewhere (Clapp 2017). Information on the management of course development, including challenges, emerged during the ethnography.

The ethnographic information from the case study was subsequently analysed using Activity Theory to show the complexities of the process of developing online courses. The results were used to create a management plan as a more joined-up process for the development of new Masters level online modules as part of the expansive learning cycle in Activity Theory (Sannino, Engestrom and Lemos 2016). It could equally be applied to the development of whole programmes. This was in answer to the research question ‘how can online Masters course development be managed?’.

Background

Previously, there has been a range of research into challenges in the development of ODL. Maguire (2005) reviewed barriers to ODL in HE institutions and found lack of time, training and faculty workload was repeatedly a problem. These were still a problem twelve years later (Clapp 2017). Amongst other factors involved in the management of ODL development are the presence of organisational infrastructure and market research; provision of all these factors could be accounted for in an ODL strategy but it needs to be applied in practice. This was examined in the past in the development of an ODL strategy and an ODL media unit, finding patchy implementation of strategy with evidence of missed targets in development, concluding that thorough planning is required for ODL courses (Sharpe et al. (2006)). Although this research is now dated, the ethnography conducted by this papers’ authors shows a similar picture with the same barriers. None of these previous studies used ethnography as a research methodology.

In a UK HE Faculty of Medical Sciences (FMS), an e-learning team was established to develop postgraduate ODL courses. The team works with the subject specialists to produce useable materials for course content, hence ODL development is a process involving multidisciplinary working. Teams and units within institutions have been employed to facilitate online course development in many cases (Davis et al (2008, p126)). The e-learning team is in effect a community of practice as knowledge and skills are practised by a group. Communities of practice were first mooted by Lave and Wenger in the 1990s and are found in ‘stable’ workplaces where the community of individuals does not change very much and others enter by participating, thereby gaining the identity of the community (Wenger 1998 p89). However, relational agency is at the heart of the team’s work outside their own community of practice, as working with subject specialists involves training individuals for ‘joint action’ (the development of ODL) when required (Edwards, 2017, p6). Contrasting this, most of the subject specialists are not dependent on the team in any way during the rest of their own working practice.

The study exemplified in this paper uses a collaborative self-ethnography of the e-learning team. Self- or auto-ethnography has been used to study personal experiences within a particular culture (Ellis, Adams and Bochner 2011). Where there is more than one researcher, it is considered a collaborative self-ethnography (Lapadat (2017)). Whilst several ethnographies have been published relating to the process of learning online (e.g. Saadatmand (2017)), as far as the authors are aware there have previously been no ethnographies of ODL development teams, making this a unique contribution. Whilst barriers to the development of ODL have previously been researched, ethnography has not been a method. Ethnography was the ideal research methodology for several reasons. Accessibility: Bryman (2012 p433) questions ability to access study populations without problems but due to the collaborative nature of this self-ethnography, rich data could be obtained without difficulty. The interests of the researchers do not have tensions with the reflection of what is occurring due to any assumptions they might make as they are also the participants. Richer data can be collected by ethnography compared to interviews as a data collection method; the problem of positionality of the interviewer is avoided (Mikecz 2012) and the social and cultural context of daily life is emphasised. Hence the findings in this study are driven by the rich data produced by ethnography.

The working practices involved in the development of online courses result in several ‘activity systems’ being created. Activity systems are defined by Engeström (2008) as ‘a collective formation that has complex mediational structure’. They contain a subject who interacts with an object through mediating tools, resulting in an outcome or a product (Engestrom (2000)). In the case described, as an example, the e-learning team interacts with subject specialists to produce ODL. As these are in effect separate communities, the use of Activity Theory is of utility to show the complexities of ODL development, further adding to conceptual thinking about the use of Activity Theory in education. It details the process, and where there are flaws in the process, it shows which points need changing.

Originally developed by Leont’ev and Vygotsky, Engestrom describes Cultural Historical Activity Theory (CHAT) as a theoretical tool to examine activity systems, to enable ‘frictions and conflict’ or ‘contradictions’ within the interactions to be found. Where these occur, there is instability in the activity system. In trying to resolve the frictions, staff within the activity system question it, creating a new model of the system. The cycle of ‘expansive learning’ is completed by collaborative development of a new model which is then tested and reflected upon (Sannino, Engestrom and Lemos 2016). Implementing it will be followed by reflection. If successful, the new practice will be consolidated and permanently implemented. Edwards and Daniels (2012) state that ‘the use of Activity Theory provides a depth of description enabling analysis’, leading to development of practice in education.
Analysis of activity systems encouraged by activity theory results in change (‘the new model’) (Engestrom (2000)). This new model is developed as multiagency working, with each set of players coming together when necessary to provide smooth delivery of ODL courses. This approach has been used in primary school education (Sannino and Engestrom (2017)), and to provide ‘tools’ in multi-professional service networks (Seppanen and Toiviainen (2017)) as examples. Examining the activity systems involved in ODL development is likely to identify areas where contradictions can be rectified on a local level, transforming practice.

Methods and data analysis

Steps in this study were: a collaborative self-ethnography of e-learning team, followed by thematic analysis of the narrative. The narrative was interrogated for ‘activity systems’ and for negative comments applied as contradictions within the activity systems, and from this a new model was developed.

Ethnography

Following ethical approval from Faculty Preliminary Ethics, a collaborative self-ethnography was carried out by the e-learning team over five months, extending beyond the full cycle of course development, maintenance, running and evaluation. The final fieldnotes were available for all the team to triangulate to ensure this was an honest account.

The team consisted of seven members shown below:



The team works on Masters level ODL, some continuing professional development, and resources for blended learning for both under- and post-graduates.

Discussions at a monthly team meeting led to team agreement with the study design: each member recorded their interactions within and outside the team, either electronically or in paper-based records. Anonymity was maintained for those interactions outside the team and if comments were recorded for publication, permission was obtained. Notes were collated weekly by the e-learning academic who discussed the interactions recorded by members and also observed the team at work. The team leader’s records were supplied for analysis separately, with overly personal material not being recorded for ethical reasons.

Thematic analysis

The fieldnotes forming the ethnographic narrative were analysed by the lead author thematically after transfer to qualitative research software tool NVivo (QSR 2016). After familiarisation, the initial codes were generated, which were then divided into themes. These were agreed by the team for subsequent analysis.

Analysis based on CHAT

Analysis of the ethnography themes used an approach based on CHAT to find ‘activity systems’ , described in detail below. For the purposes of this paper, ‘activity system’ refers to a collective of separate activity units which make up the work actions of the e-learning team and subject specialists as ‘subjects’ to produce the online courses as ‘products or outcomes’.

Once the main activity systems were elucidated, analysis of the themes pertaining to management was interrogated to find areas where there were contradictions or frictions, preventing smooth running of the activity system, according to comments in the narrative. The contradictions show which areas in the system need changing.

Subsequent action

The team met to discuss the contradictory areas to develop a new model of course development, as part of the expansive learning cycle.

Findings

Ethnography narrative

The day-to-day working of the team involved interacting with faculty staff (clinicians and scientists as subject specialists, who usually have lecturing, but not online, teaching experience), to develop Masters level courses for ODL students. Meetings take place on and offline; mostly offline for long content discussions with much shorter email comments and occasional Skype meetings when participants are not on campus. During these meetings development and content is considered. Informal training of faculty staff takes place. Other meetings include full team monthly meetings where we discuss what we are doing, the challenges and solutions. There are also meetings between individual team members and the team leader to decide the future areas of developments such as increasing the dissemination of reusable learning objects (for example illustrations and animations) and widening page types to increase functionality of the content management system. The team also talk to each other informally, solving problems and developing new software and expertise. These informal discussions were observed by the lead author and recorded in the field notes along with summaries of monthly meetings and the collations of team-recorded interactions.

Analysis of the narrative found the management codes and the themes within. For the purposes of this paper only the areas where comments were negative are reported in Table 1 which shows examples from the narrative which were problematical in the management of development of ODL.

Table 1.



Activity systems

From the ethnography it became obvious that more than one activity system or unit was at work in the overarching activity system of course development when considering socio-cultural activities undertaken by the e-learning team. It showed that the team was both subject and object of activity systems depending upon the point reached in the development of courses. Also, that there were different objects within these activity units.

The first activity system defined consisted of the clinicians and scientists as subjects providing pre-course clinical and scientific knowledge, along with e-learning team knowledge, mediating the transformation of various online materials (presentations, personal capture recordings, audio) to the pre-course materials as the outcome. Pre-course materials are subsequently worked upon in a separate second activity unit with the e-learning team as the subject using their technical and pedagogical skills to transform online pre-course materials into the outcome, an online course. A third activity system was found where the e-learning team as subjects use their pedagogical and technical knowledge as mediating tools training the objects, the clinicians and scientists. The outcome is competence in online teaching as course leaders and online lecturers.

The negative themes in table 1 from the ethnographic analysis showed the ‘contradictions’, where the activity process does not flow smoothly. These are applied to the three activity systems, described in figures 1-3 with the contradictions shown as wavy red arrows.

Figure 1: Activity unit with subject specialists as subjects





Figure 2: Activity unit with the e-learning team as subjects

Figure 3: Activity unit with the e-learning team as subjects and subject specialists as objects

In the first activity system, subject specialists have little knowledge of types of materials to usefully provide pre-course for collaboration with the e-learning team. They are unused to collaborating in education development and have little time set aside. They fail to engage well with the team, missing deadlines. Subject specialists sometimes do not take ownership of the course due to a lack of confidence in technical matters. The time issues create problems with the division of labour when editing and updating. Market research is often absent or sketchy before ODL course development starts.

In the second activity system there was a problem with the ‘tools’: the e-learning team has issues with timeliness of obtaining materials from subject specialists so the tools for activity were lacking. Rules: quality assurance processes for approvals of courses should be in place before commencing development requiring scheduling on time; this was a ‘time’ problem when not all information was available for administrative activities due to subject specialist time limitations. A contradiction was seen in the division of labour for finding further resources and developing activities: team members expended time searching for materials when subject specialists did not deliver them. Further contradictions in the system were that the budget needed discussion and negotiation with the Graduate School community for new software and licences for images required Knowledge of enrolment numbers was required in good time to create activities. Communication with the wider community: information technology services communication was problematical for the bespoke CMS.

In the third activity system with the e-learning team as subjects, contradictions included specialists having little idea of online pedagogy and technology, requiring training by the team ‘on-the-job’, causing delays. The timing of engagement between the team and subject specialists for training combined with a lack of enthusiasm amongst the clinicians and scientists for training was contradictory as was the lack of management of approval process.

Developing the new model

The contradictions within the activity units outlined above was discussed within the team. These contradictions in the management of course development provided an initial list to concentrate on for a new model. The points in the list are discussed next.

Training subject specialists

Knowledge of online pedagogy is required for online teaching (Salmon 2014). It was decided within the team that there should be pedagogy and technology training for subject specialists with some being mandatory before course development starts, so there is some knowledge of what is required as input.  Situated learning meant that some training is ‘on-the-job’, relationally working with the team. This professional development can occur in an informal way similar to an apprenticeship where learning also takes place ‘on-the-job’ (Billett (2016)); this is also the experience of the e-learning team. Training will help define the roles of the subject specialist.

Time as a resource

Lack of enthusiasm shown by subject specialists for taking charge of courses is due in part to time constraints; time is a major contradiction. University staff require time to engage in staff development for online teaching, as well as actually developing courses (Barcyzk et al (2011)). Provision of time will also help the e-learning team meet its’ deadlines in a timely fashion, rather than trying to complete tasks at the last minute due to lack of time of subject specialists. Timing of quality assurance processes needs to be managed with approvals obtained in good time before course development commences.

Financial and technical support for course development resources

Although new learning programmes and updating older ones may require experimentation with different technologies so there is ‘trial and error’, all technology needs support from developers or university-wide information technology staff, planned for in advance (Davis et al. 2008, p100). Similarly, library resources need to be available along with time for library staff to aid reading list availability. These need to be part of the development management plan.

Market research

Where ODL is a requirement of grant-funding bodies in this institution, course development goes ahead without market research as development is mandatory. However, were market research to take place, a course more specifically aligned to potential student requirements could be designed. All courses need to be economic so market research needs to be carried out before too much investment in the course takes place (Chipere (2017)).

The new model

The e-learning team used a checklist to define the new model. Checklists have been initiated in the aircraft engineering industry to reduce human error and in surgical procedures where mortality rates were subsequently halved (Clay-Williams and Colligan (2015)). The checklists are in place before procedures start and without their completion the procedure does not go ahead. Whilst course development is hardly a life-threatening procedure, it is time and resource-consuming so the process should be streamlined and without disturbances, hence the advantage of having a checklist prior to commencement. However, as Clay-Williams and Colligan state, it is how checklists are used which means they will be successful or not. In this case it is a simple work flow to be achieved in steps, resulting in yes/no answers, making the checklist a useful tool for planning course development.

From the discussions of the e-learning team on the contradictions, it was decided that the development of ODL should proceed once the elements of a checklist had been completed. The checklist has the following nine points:

  1. Has market research, and marketing, taken place? If so, is there an idea of student numbers?
  2. Are university course approvals for quality assurance in place? If not, how long will they take to be arranged?
  3. What is the budget (software, buying time for clinicians’ availability, attending or running training courses)?
  4. Do the e-learning team and subject specialists have the time and other resources required?
  5. Is there a subject specialist willing to lead the course?
  6. Subject specialist staff need training in pedagogy and technology: is this in the timeline of course development prior to course running?
  7. Are institutional support systems suitable and available?
  8. Are the roles of subject specialist staff and team developers well defined?
  9. Has consideration been given for course updating including time?

The implementation of the checklist could be managed using project management software, similarly to Abdous and He (2008). It will be used as and when future courses are planned, and the effectiveness subsequently evaluated.

Conclusions

In answering the question ‘how can online Masters course development be managed?’ this study used an ethnographically-inspired approach and activity theory as the methodology and in the analysis of data to show where contradictions lie in the process of ODL course development. These areas were particularly in time available to staff, but also in training, resources, quality assurance approval, supporting systems from wider faculty and in marketing. The lack of time and the unknown quantity of online pedagogy and technology were the main constraints for subject specialists, reducing their buy-in to ODL development.

Collaboration within the e-learning team resulted in a checklist to be used in the management of course development, streamlining the process which should save time for all concerned. Use of the checklist as part of the new model of activity before the online development process begins will be tested using action research with reflection and further editing. This completes Engestrom’s cycle of expansive learning (Sannino, Engestrom and Lemos (2016)).

This research was original in its subject and methodology (the ethnography of the e-learning team and those who interact with it; the use of Activity Theory to elucidate areas where professional working was challenging). It has provided insights into collaborative working across disciplines towards online education, increasing conceptual knowledge of relational working. The resulting checklist is likely to prove useful in managing the process of course development if as successful as their use in the aero-engineering industry and in surgery. Benefits from streamlining in a time-poor environment using this management may be generalizable to similar situations of ODL module and programme development in other HEIs.

Funding

No funding was provided for the study which was conducted in the course of daily work with protected time provided by the EquATE (Equal Acclaim for Teaching Excellence) programme.

Acknowledgements

The authors would like to thank the Newcastle University EquATE programme without which the research would not have taken place, and Lynne Rawles for her help in data gathering.


References

Abdous, M. & He, W. (2008). Streamlining the online course development process by using project management tools. The Quarterly Review of Distance Education, 9, 2, 181–188 retrieved from https://www.researchgate.net/profile/Mhammed_Abdous/publication/234631992_Streamlining_the_Online_Course_Development_Process_by_Using_Project_Management_Tools/links/573ddd7908aea45ee842da43/Streamlining-the-Online-Course-Development-Process-by-Using-Project-Management-Tools.pdf

Barczyk, C, Buckenmeyer, J, Feldman, L and Hixon, E (2011) Assessment of a University‐Based Distance Education Mentoring Program from a Quality Management Perspective, Mentoring and Tutoring: Partnership in Learning, 19:1, 5-24, DOI: 10.1080/13611267.2011.543567 retrieved from http://dx.doi.org/10.1080/13611267.2011.543567

Billett, S. (2016) Apprenticeship as a mode of learning and model of education Education and Training; 58, 6, 613-628.  DOI:10.1108/ET-01-2016-0001 retrieved from
https://search.proquest.com/docview/2084371088/fulltextPDF/7998CAF7CE354F1BPQ/1?accountid=12753

Bryman, A (2012) Social Research Methods 4th Ed. UK, Oxford, Oxford University Press

Care, W.D and Scanlon, J.M (2001) Planning and Managing the Development of Courses for Distance Delivery: Results From a Qualitative Study; Online Journal of Distance Learning Administration, Volume IV, Number II retrieved from https://www.westga.edu/~distance/ojdla/summer42/care42.html

Chipere, N (2017) A framework for developing sustainable e-learning programmes, Open Learning: The Journal of Open, Distance and e-Learning, 32:1, 36-55, DOI: 10.1080/02680513.2016.1270198 retrieved from https://www.tandfonline.com/doi/pdf/10.1080/02680513.2016.1270198?needAccess=true

Clapp A, (2017) An e-Learning Team’s Life On and Offline: A Collaborative Self-Ethnography in Postgraduate Education Development The Electronic Journal of e-Learning Volume 15 Issue 1 , (pp33-45) retrieved from http://www.ejel.org/volume15/issue1/p33

Clay-Williams, R and Colligan, L (2015) Back to basics: checklists in aviation and healthcare; BMJ 24, 7, 428-432 retrieved from http://dx.doi.org/10.1136/bmjqs-2015-003957

Davis, A. (2008). Developing an infrastructure for online learning. In T. Anderson & F. Elloumi (Eds.), Theory and Practice of Online Learning, 97-114 retrieved from http://biblioteca.ucv.cl/site/colecciones/manuales_u/99Z_Anderson_2008-Theory_and_Practice_of_Online_Learning.pdf

Edwards, A (2017) Revealing Relational Work; in Edwards, A (Ed) Working Relationally in and Across Practices: a Cultural-Historical Approach to Collaboration; USA, New York, Cambridge University Press

Edwards, A and Daniels, H (2012) The knowledge that matters in professional practices, Journal of Education and Work, 25:1, 39-58, DOI: 10.1080/13639080.2012.644904 retrieved from https://doi.org/10.1080/13639080.2012.644904

Ellis, C, Adams, T.E and Bochner, A.P (2011) Autoethnography: an overview; Forum: Qualitative Social Research 12, 1, retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/1589/3095 

Engeström, Y., 2000. Activity theory as a framework for analyzing and redesigning work. Ergonomics, 43(7), pp.960–974 retrieved from https://doi.org/10.1080/001401300409143

Engeström, Y., 2008. From Teams to Knots: Activity-Theoretical Studies of Collaboration and Learning at Work (Learning in Doing: Social, Cognitive and Computational Perspectives) UK, Cambridge, CUP.

Lapadat, J.C (2017) Ethics in autoethnography and collaborative ethnography; Qualitative Inquiry 23, 8, 589-603 retrieved from https://journals.sagepub.com/doi/pdf/10.1177/1077800417704462

Maguire, L.L (2005) Faculty Participation in Online Distance Education: Barriers and Motivators;  Online Journal of Distance Learning Administration, Volume VIII, Number I, retrieved from https://www.westga.edu/~distance/ojdla/spring81/maguire81.htm

Mikecz, R., (2012). Interviewing Elites : Addressing Methodological Issues. Qualitative Inquiry18, 6, 482-493 retrieved from https://journals.sagepub.com/doi/pdf/10.1177/1077800412442818

QSR 2016 at: http://www.qsrinternational.com/

Saadatmand, M (2017) A new ecology for learning: an online ethnographic study of learners' participation and experience in connectivist MOOCs; Helsinki Studies in Education 9, retrieved from https://oerknowledgecloud.org/sites/oerknowledgecloud.org/files/saadatmand.pdf

Salmon, G. (2014). Transforming future teaching through 'Carpe Diem' learning design. Educ. Sci., 4, 52-63; doi: 10.3390/educsci4010052 retrieved from https://files.eric.ed.gov/fulltext/EJ1117871.pdf

Sannino, A and Engeström, Y (2017) Relational agency, double stimulation and the object of activity: an intervention study in a primary school; in Edwards, A (Ed) Working Relationally in and Across Practices: a Cultural-Historical Approach to Collaboration; USA, New York, Cambridge University Press
 
Sannino, A, Engeström, Y and Lemos, M (2016) Formative Interventions for Expansive Learning and Transformative Agency, Journal of the Learning Sciences, 25:4, 599-633, DOI: 10.1080/10508406.2016.1204547 retrieved from https://doi.org/10.1080/10508406.2016.1204547

Seppanen, L and Toiviainen (2017) Relational agency and the development of tools in service networks; in Edwards, A (Ed) Working Relationally in and Across Practices: a Cultural-Historical Approach to Collaboration; USA, New York, Cambridge University Press

Sharpe, R, Benfield, G and Francis, R (2006) Implementing a university e-learning strategy: levers for change within academic schools; ALT-J, Research in Learning Technology Vol. 14, No. 2, pp.135–151 retrieved from https://doi.org/10.1080/09687760600668503

Wenger, E (1998) Communities of Practice: Learning, Meaning and Identity; UK, Cambridge, Cambridge University Press


Online Journal of Distance Learning Administration, Volume XXII, Number 1, Spring 2019
University of West Georgia, Distance Education Center
Back to the Online Journal of Distance Learning Administration Contents