CacaoBench: A Factorial Comparison of CNN Architectures and Attention Modules with Visual-Similarity-Grouped Cross-Validation for Classifying Cacao Pod Diseases and Pests

Diseases and pests of cacao (Theobroma cacao L.) pods motivate visual classification tools, but random per-image evaluation can overestimate performance when nearly identical photographs exist. Ca-caoBench is a factorial comparison of nine configurations combining ResNet50, EfficientNetV2B0, and MobileNetV3Large with three variants: no attention, CBAM, and a Transformer block. From 4390 pub-lic images in three classes, 4388 were retained after duplicate auditing; similar images were grouped so they never crossed partitions, and 3786 were analyzed with 10-fold nested grouped cross-validation. The primary metric was macro F1, with 95% bootstrap intervals on out-of-fold predictions and six planned contrasts with Bonferroni adjustment. MobileNetV3Large with a Transformer block obtained the highest macro F1 (0.901; 95% CI: 0.882–0.920; accuracy 0.945) at 9.4 ms per image on a T4 GPU. The Transformer block raised macro F1 in EfficientNetV2B0 (+0.120) and MobileNetV3Large (+0.136), whereas in ResNet50 differences were small and inconclusive. In a single exploratory evaluation on a historical reserve of 602 images, the selected model obtained a macro F1 of 0.934 (95% CI: 0.895–0.968), consistent with the out-of-fold estimate. The results are internal evidence from one dataset, not external field validation.