A Conceptual Framework For Mitigating Cheating Behaviours In Online Assessments

Reflections on Practice

Recommended Citation:
Ho H. K. (2021). A conceptual framework for mitigating cheating behaviours in online assessments. Asian Journal of the Scholarship of Teaching and Learning, 11(1). 56-63.

View as PDF      Current Issue

Abstract

The coronavirus outbreak in 2020 has expedited the proliferation of online assessments globally in lieu of face-to-face assessments. However, this has led to concerns regarding the compromise of academic integrity in the absence of supervision. Many strategies have been proposed to circumvent cheating behaviours, although each approach may differ in its effectiveness and contextual applicability. By adapting the well-established framework for hazard management, I attempt to categorise these strategies into different levels of control measures that are synonymous to the measures of elimination, substitution, engineering, administrative, and personal protection. It is hoped that such stratification can motivate a self-directed profiling of different proposed measures to help the assessor appraise and arrive at a strategy that is effective in upholding assessment integrity. To add granularity for the assessor’s consideration, the advantages, and disadvantages of each of these strategies are outlined. Ultimately, it is envisaged that this conceptual framework can help examiners make objective decisions that are relevant to their assessment tasks, and to inspire approaches to be used in combination wherever appropriate.

Keywords: Online assessments, academic integrity, assessment integrity, cheating behaviour, hazard management

The Advent of Online Assessment

Information technology and wireless communication are the enablers of online platforms in education. To date, they have revolutionised learning management systems for knowledge curation and repository, while transforming blended learning to facilitate continuous and self-directed learning. It is expected that this advancement will also drive assessment reforms, specifically with the use of online assessments (Fontanillas et al., 2016).

Online assessments, in combination with other classroom-based assessments, expand the repertoire for the instructors to utilise different evaluation tools to bring out the best in students. Moreover, these approaches reduce dependence on dwindling physical resources, such as space, manpower constraints, and the excessive use of printed materials.

In 2020, the coronavirus pandemic (COVID-19) thrust online assessments into the limelight. In many countries and cities, lockdowns and social-distancing measures prohibit mass congregation, including within classrooms. With such restrictions becoming a protracted reality, a replacement of face-to-face assessment became an imminent consideration.

Problem with Cheating Behaviour

Yet, instead of widening the options for instructors to administer assessments, online assessment has become the obligatory substitute during COVID-19, where face-to-face activities are limited. Therefore, online assessment is belaboured with an additional situation of being conducted offsite. Inadvertently, this creates concerns over assessment integrity where the lack of an effective proctoring mechanism creates a loophole for students to cheat. In fact, assessment integrity is purportedly one of the biggest limitations curtailing effective implementation of online assessment (Gibson & Blackwell, 2007; Berkey & Halfond, 2015).

There are two principal ways where assessment integrity can be compromised: 

• Lack of controls to restrict students’ access to illicit materials. If assessment problems include recall or derivation of information obtainable from different resources, then online assessment would avail unrestricted information to the students, rendering it difficult to unbiasedly evaluate students’ knowledge and understanding. Even with the use of examination software that blocks internet access, one could still cheat by surreptitiously using a second device to garner information. 
  
• Online assessment cannot prevent students from soliciting external help during assessment. Online exam proctoring remains difficult to accomplish, and further exploits of such proctoring tools are constrained by privacy protection measures. There are also expressed concerns that close surveillance may increase anxiety and in turn compromise students’ performance (White, 2020).
  
  

As a result, students’ cheating behaviour in online assessments is fast becoming a global phenomenon (Dyer et al., 2020). Locally, institutions are muddled with reported cases of cheating in assessments, including high profile mass cheating cases that have received national attention (Wong, 2020). While sanctions are instituted to address such incidences, penalties alone cannot deter or prevent all cases of cheating. It also creates a moral dilemma whereby many students who are not caught in the act are left unpunished, and in turn may motivate repeat offences in the future.

The Need For A Stratified Appraisal Of Control Measures Against Cheating 

To date, several control measures have been curated to preserve academic integrity during online assessments. Some have averted the ease of cheating by randomising questions, shortening the assessment duration, or administering online proctoring. Others have redesigned the assessment by turning them into open-book tasks, group work, and home assignments. There are also attempts of avoidance whereby assessments are eliminated or simply delayed till a later time. While each measure may circumvent cheating behaviour to some extent, they are inadvertently accompanied with unique challenges. For instance, one approach is to redesign an assessment task into one whereby the student is required to produce original work (e.g. a project-based assessment). However, this may not be feasible where the timeframe for grading is limited and where the class size is huge. Ultimately, implementation of control measures is currently reliant on personal experience, or defaulted to prescribed institutional practice. What is lacking right now is an overarching framework to systematically appraise the different control measures and to help teachers on the ground crystallise out a sound response for themselves. 

In this Reflection, I will attempt to stratify possible control measures against online assessment cheating using the conceptual framework that was first developed by the National Institute of Occupational Safety and Health (NIOSH) for hazard controls, and has since been popularised amongst risk management think tanks around the world (Centers for Disease Control and Prevention, n.d.). Currently, it is not known whether this framework has been adopted in the evaluation of control measures used to mitigate cheating in assessments. There is also no reported studies of other risk assessment approaches to manage academic integrity. The motivation for considering this framework is evoked from a closer examination whereby cheating behaviour presents a harm to assessment integrity, much like how physical hazards exert ill-effects on the safety of an activity. This resemblance provides reasonable grounds to consider the contextual extension of the principles of risk assessment towards managing and circumventing cheating risks. The central idea for risk assessment is to provide a systematic way of thinking in order to appraise and prioritise the response measures, on the basis of their effectiveness. In the same way, it is my conjecture that this framework can facilitate an objective process of decision-making to select the most efficacious response measure among the many options available to safeguard against cheating.  At the very least, this exercise offers a new perspective to profile and categorise each strategy, which in turn can help us compare and contrast between them, to even consider the merits of applying some of these measures in combination.

Fundamentally, risk mitigation measures can be stratified into five hierarchical levels of intervention: elimination, substitution, engineering, administrative, and personal protection. The effectiveness of the approach follows the same order. However, the applicability of individual strategy depends on situational constraints. Therefore, combining this stratification with a clear description of the strengths and limitations of each measure would help end-users make informed decisions.

Elimination measures

“Elimination” is to remove the exposure to the risk in its entirety. In the context of cheating prevention, this is to remove the use of online assessment. While effective, a critical consideration is whether alternative face-to-face assessments are available. With COVID-19, such measures may cause a delay in academic candidature, or else the instructor may have to settle for assessment-free options. The feasibility of such drastic actions will have to be exercised as an institutional or professional decision.

Substitution measures

“Substitution” is to replace an online assessment with a different type of online activity. This is useful where complete elimination is not feasible, but we can choose an alternative online activity that carries a lower risk for cheating. For instance, a time-limited online assessment can be replaced by a take-home assignment. Another option is to conduct an oral examination or presentation. Take-home assignment simplifies the governance of the assessment. Even if students are discussing among themselves, they can still be assessed individually and be subjected to plagiarism checks. That said, the drawback of this approach is the extended timeframe required for students to complete the task, as well as the complexity of the grading process. This will not be possible all the time because the manpower cost behind this measure is extremely high. Another caveat is that not all questions are amendable for a take-home assignment. The questions must be open-ended or otherwise, students will all converge onto the same response, and plagiarism checks will not be practicable. Even so, there is no way of proving that the student attempted the assignment individually unless there is an oral examination that follows the submission.

An alternative substitution for online written assessment is an oral examination (i.e. viva voce). This can be in the form of a presentation on a given topic, a verbal question and answer, or a combination of the two. An oral examination evaluates the ability for the students to construct their thoughts logically through understanding. A challenge with this approach is that some students may stumble under stress. The assessor may also need to prepare multiple sets of questions as they can only assess students singularly. The logistical cost to such assessment could be prohibitive, especially in large classes.

Engineering measures

“Engineering” measures are to mitigate risk by reworking the environment that circumscribes the online assessment task. This can include redesigning or limiting some of the conditions for the online assessment (see Appendix for the different levels of of controls to mitigate cheating behaviour among students). A simple re-engineering effort is to shorten the duration of the assessment. By limiting time, it reduces the risk of cheating as students will not have enough time to look up additional resources or to communicate with one another. However, a big concern with such an approach is that students’ performance will be differentiated by how fast they complete the task, rather than how well they know the content.

In some online assessment tools such as ExamSoft, there is an option for an internet lockdown. This engineering option blocks access to online resources during the online assessment. However, this feature cuts both ways: On one hand, it reduces access to illicit online information; however, it also restricts legitimate online searches. This measure also does not prevent discussion between individuals nor prevent access to information using a second device.

Other engineering tools include examination software that impose special features such as a forward-only function and randomisation of questions etc. Such features mitigate risk by removing the incentives for illegitimate discussion among students. However, they do not prevent the most egregious behaviours from students who are bent on cheating. Also, the forward-only function removes self-checking as a discipline and may propagate unintentional errors.

Administrative measures

Administrative measures are aimed at providing advisories and guidelines that promote assessment integrity. To achieve sustainability, academic integrity must be inculcated through education. To reinforce this, a just-in-time self-declaration can be administered at the commencement of the assessment to remind students of the need to observe assessment integrity. As a deterrent, sanctions could be meted out when violations are detected.

Administrative measures can also be supported by a robust whistleblowing policy. However, the effectiveness of such a measure is nestled in the cultural predispositions of the population. Furthermore, this will not be helpful in violations that involve a single individual referring to illicit resources in the privacy of his/her own home during an online assessment. Short of an actual surveillance, these activities are unlikely to be discovered by anyone.

Protective measures

In the context of risk management against physical hazards, the final level of control measure is known as personal protective measures. In another word, if the exposure to the hazard cannot be avoided, then what remains is to proceed with caution by protecting oneself against possible harm. While there is no exact equivalent of “personal protection” in the context of mitigating student cheating, the spirit of this measure can be addressed through proctoring of the assessment. Categorically, physical checks serve as a final layer of safeguard that we can apply.

However, proctoring an online assessment is technically challenging. Firstly, there must be continuous visual monitoring to be effective in detecting illicit activities. Closed-circuit TVs are options for remote monitoring if students are taking it in common spaces. If students are taking the assessments from home, other video-capturing devices such as web cameras and smart phones will be required. The placement of the cameras would be critical to achieve effective monitoring, but this is difficult to administer because students are the ones controlling their devices. Micromanaging individual students to do this the right way would be laborious and time-consuming. Moreover, the whole process also provokes some concerns about privacy protection. Therefore, remote proctoring remains imperfect and should be considered as a last resort and not to be used as a standalone measure.

Discussion 

Overall, this stratification of control measures offers a hierarchical consideration that mirrors the risk management of environmental hazards. Macroscopically, it allows us to compartmentalise different control measures against cheating in online assessment, and a mental model to assess their effectiveness and limitations. The value that this insight can serve is to provide greater clarity on the platform by which each measure is operating within. For instance, implementing two strategies from engineering approaches may create duplication of efforts; whereas complementing it with a measure from a different category can elicit orthogonal coverage that reinforces overall effectiveness. This awareness is crucial because it helps us (as educators) readily see what is missing, and prompt appropriate add-ons to existing strategies. Implicitly, this also means that we should actively explore opportunities for multi-prong controls to improve overall effectiveness. For instance, a prevailing re-engineering built into an online assessment can always be supplemented by administrative controls.

The conceptual framework also unveils the principal characteristic of each category of measure to shape the prioritisation of our options. Intuitively, strategies that minimise risk of exposure in the first place tend to be more effective than those that simply subvert the risk of damage. Hence, “preventive measures” such as elimination and substitution methods which remove the temptation for cheating behaviour should take precedence over “mitigative measures” of re-engineering, administrative and protective measures where the risks are managed. 

That said, the most effective strategy may not always be the most feasible option. Eliminating all forms of online assessments in a prolonged pandemic will adversely affect overall academic progression and workforce readiness. Substituting the online assessment with other forms of assessment may also run into problems with cost and manpower constraints. Moreover, one must also be mindful that a different mode of assessment must still measure the intended competency the tool was set up for in the first place. Taking all these considerations, the benefits of the chosen mitigating measures must outweigh the cost for implementation, and any compromises of achieving the learning outcomes. Otherwise, alternatives strategies placed lower down on the hierarchy should be considered.

Finally, this framework illuminates the inconvenient truth that remote proctoring resides at the bottom of the hierarcy of controls when filtered through the lens of risk assessment. Evidently, online monitoring alone does not remove the risk of cheating, especially when technology has not been able to fully recapitulate the prowess of a face-to-face proctoring. Therefore, online proctoring should only be implemented if we are mindful of the need for additional safeguards by applying higher levels of control. It remains our hope that this technological limitation can be surmounted in future, as artificial intelligence and other innovations continue to transform online surveillance and assessment software iteratively.

Conclusion 

In conclusion, this reflection promulgates a conceptual framework to guide the thought process in managing cheating behaviours during online assessment. It brings together a hierarchical consideration as well as a careful exposition of the strengths and limitations of each strategy. Ultimately, there is no one-size-fits-all strategy. Therefore, it is the hope that this framework can be a starting point where new information can fill in with time to enrich the protocols for more robust management.

References

Berkey, D., & Halfond, J. (2015). Cheating, student authentication and proctoring in online programs. New England Journal of Higher Education. Retrieved from https://nebhe.org/journal/cheating-student-authentication-and-proctoring-in-online-programs/.

Centers for Disease Control and Prevention (n.d.). Hierarchy of controls. U.S. National Institute for Occupational Safety and Health. https://www.cdc.gov/niosh/topics/hierarchy/ 

Dyer, J. M., Pettyjohn, H. C., Saladin, S. (2020). Academic dishonesty and testing: How student beliefs and test settings impact decisions to cheat. J Nat College Testing Assoc, 4(1), 1-30. https://www.ncta-testing.org/assets/docs/JNCTA/2020%20-%20JNCTA%20-%20Academic%20Dishonesty%20and%20Testing.pdf

Fontanillas, R. T., Carbonell, R.M., & Catasús, G. M. (2016). E-assessment process: giving a voice to online learners. International Journal of Educational Technology in Higher Education, 13(1), 1-14. https://doi.org/10.1186/s41239-016-0019-9 

Gibson, J. W., & Blackwell, C. W. (2007). The challenge of student assessment in the online classroom. College Teaching Methods & Styles Journal, 3(4), 1-8. https://doi.org/10.19030/ctms.v3i4.5580

White, N. (2020, April 22). 'Creepy' universities are recording students and monitoring their eye movements to stop them cheating as they're forced to take tests in their bedrooms because of Covid-19. Mail Online. https://www.dailymail.co.uk/news/article-8243637/Creepy-software-used-stop-university-students-cheating-online-exams-amid-coronavirus.html

Wong Y. (2020, June 15). Coronavirus: NUS students get zero marks for cheating in take-home exam. The Straits Times. https://www.straitstimes.com/singapore/education/coronavirus-nus-students-get-zero-marks-for-cheating-on-take-home-exam

Appendix. Different Levels of Controls to Mitigate Cheating Behaviour Among Students 

HO Han Kiat, Ph.D., DABT, is currently the Deputy Head (Education) at the Department of Pharmacy, and Vice Dean of Students. In 2009, he started his own research programme, exploring various aspects of liver diseases including the of drug-induced liver toxicity, liver fibrosis and liver cancer. He has published more than 80 papers in internationally recognised journals, won multiple university-level teaching excellence awards, and is an elected fellow of NUS Teaching Academy.

Han Kiat can be reached at phahohk@nus.edu.sg.