AutoCheat

Automatic programming assignment assessment systems, or autograders, are used in educational institutions to ease the work-workload of human markers by automatically grading large batches of submissions using unit testing suites. However, they have limited ability to flag possible academic dishonesty. Therefore, students may exploit vulnerabilities in these systems. Our proposed solution, AutoCheat, is a working prototype that flags such suspicious submissions for review, with a focus on detecting plagiarism between students (peer-to-peer plagiarism), and hard coding (where code is tailored to pass unit tests using conditional statements, without fulfilling the requirements of the assignment). It consists of three modules: a supervised machine learning module trained on labelled pairs of Java programs to detect peer-to-peer plagiarism, an unsupervised machine learning module that compares the underlying structure of Python programs and clusters similar programs together to detect peer-to-peer plagiarism, and a Bayesian network that extracts evidence of hard-coding from the code structure of Python programs. AutoCheat brings these methods together in a single, reviewer-facing interface, highlighting submissions for further investigation and presenting evidence in an interpretable way, aiding judgement. Our model demonstrates efficient performance, and classification results that are competitive with existing systems such as MOSS and ARI. Evaluation showed that no Java similarity approach performed best across its benchmarks, while Python clustering showed promise, and hard coding detection achieved 85.2% accuracy on a small, constructed dataset. Future work should evaluate the system on authentic coursework to assess its usefulness to human reviewers.