A forensic linguistic dataset for offensive content detection in a low-resource language: Setswana

This paper presents a monolingual Setswana dataset for offensive language detection, developed for digital forensic applications. The dataset comprises texts drawn from social media, civic discourse, and lexicographic sources. It also includes span-level semantic trigger annotations. Double coding yielded strong inter-annotator agreement, with a Cohen’s kappa of 0.86. The corpus was divided into an 80:20 ratio of a training set and an untouched, semantic-trigger-free (tag-free) holdout set. Model selection was conducted using five-fold stratified cross-validation on the training split, followed by final evaluation on the holdout set. Classical machine learning baselines, such as TF-IDF with Naïve Bayes, Logistic Regression, and a Linear Support Vector Machine, were benchmarked against transformer models under tag-free training. On the tag-free holdout set, the classical models achieved macro-F1 scores of 0.80–0.87, while the transformers performed only marginally better. However, the 95% bootstrap confidence intervals for macro-F1 overlapped, and the paired Wilcoxon signed-rank test was non-significant with (Formula presented), indicating no statistically reliable difference at this corpus scale.In contrast, training with semantic trigger tags and testing on tag-free data reduced transformer performance, which suggests supervision leakage. For forensic traceability, Local Interpretable Model-agnostic Explanations (LIME) and Stabilized LIME (S-LIME) were applied. Their attributions aligned with annotated trigger spans and remained stable across perturbations. Counterfactual edits identified the conditions under which predictions changed. Limitations include dataset size and dialectal coverage. The dataset and reproducible code have been released to support low-resource forensic natural language processing.