Use of Artificial Intelligence in Open and Distance Learning: Opportunities and Challenges

Abstract

Artificial intelligence (AI) can transform digital traces from open and distance learning (ODL) into early-warning information, but practical value derives from leakage control, robust validation, interpretability, and responsible use rather than merely discrimination. This study evaluates a leakage-controlled early-window pipeline on a public learner-level extract from the Open University Learning Analytics Dataset (OULAD), course CCC. The analytic sample comprised 1,043 unique learners with weekly assessment-score sums and virtual learning environment (VLE) click aggregates plus demographic/context variables. Risk was operationalized as final_score<40; 379 learners (36.3%) met this score-defined criterion, 296 of whom had final_score=0. Hence, the endpoint is interpreted as low/zero final performance rather than a reconstructed withdrawal label. Only weeks 1-4 of the eight observed weekly aggregates were used as predictors. Non-parametric inference, effect sizes, Spearman correlations, six machine-learning models, five-fold stratified cross-validation, bootstrap confidence intervals, feature-family ablation, SHAP explanations, and subgroup/error analyses were assembled. At-risk learners recorded substantially fewer early clicks (median 246 vs. 703.5; rank-biserial r=-0.614), lower positive-score occurrence (0.50 vs. 1.00; r=-0.773), and lower early mean score (17.5 vs. 77.75; r=-0.880), all Holm-adjusted p<0.001. Regularized logistic regression achieved held-out ROC-AUC 0.936 (95% CI 0.901-0.968), precision-recall performance 0.915 (0.869-0.955), F1 0.789, and recall 0.789. Random Forest produced comparable discrimination and was used for TreeSHAP analysis, which ranked early mean score, positive-score occurrence, non-zero assessment performance, score trend, and recent-click share as the most informative predictors. Assessment-only features nearly matched the full feature set, whereas engagement alone remained informative and demographic/context variables were comparatively weak. The findings support human-in-the-loop risk ranking and targeted support, but formal probability calibration, external validation, sensitivity analysis of the zero-score subgroup, and comparison with very simple assessment rules remain necessary before deployment. AI in ODL is most defensible as transparent decision support that augments rather than replaces educational judgment.