Automated Ranking Assessment based on Completeness and Correctness of a Computer Program Solution - Suhailan, S.; K. Yusof, M.; F.A. Abidin, A.; A. Fadzli, S.; S. Mat Deris, M.; Abdul Samad, S. (International Journal of Engineering and Technology, 2018)
Many automated programming assessment methods requires program to be represented into certain calculated features. In order to assess the difficulty of a program in answering a computational programming question, two main factors need to be considered in extracting the features; program incompleteness and solution correctness. Common features were based on solution's template matching to assess a program correctness. However, incomplete program that usually occurs among novice learners may rise difficulty for the technique in parsing the program's structure. This research proposes program's scoring features based on instruction template's sequence and ratio to represent the programs into a solution ranking list in solving a programming question. The features were evaluated against manual rubric's assessment of 67 incomplete Java programs. The result shows that the proposed features were highly correlated with the manual rubric's assessment (rho = 0.9142086, S = 4299.5, p-value < 2.2e-16). Thus, the proposed features can be used to automatically rank computer programs based on expected instruction-based of solution templates. The ranking result can be used to identify most struggled user especially in assisting students in a programming lab exercise session.Â Â