Beyond Reported Accuracy: A Survey and Comparability Audit of Machine-Learning Android Malware Detection, 2009–2025
Description
Machine-learning detectors of Android malware are routinely compared by the accuracies their papers report, although those figures come from different corpora, labels, metrics and evaluation protocols. This survey asks what such comparisons can support. We tabulate 113 studies published 2009–2025, separate 93 core detector studies from supporting and context work, and trace every headline figure to its primary source, releasing the source records; every headline figure was confirmed. For the 58 core studies read in full we code the evaluation protocol: 37 evaluate only on partitions not stated to be ordered by time, 13 order a partition by time, one also withholds malware families, 8 state a duplicate control and 28 report a false-positive rate. The 29 confirmed single-valued accuracies cluster at a median of 98.0% (interquartile range 2.2 points), a spread smaller than the effect of protocol choice: on one real-device corpus, ordering the split by time lowers one detector’s accuracy by 4.73 points, five years of drift cut its recall at a fixed low false-positive rate by 26–38%, and withholding whole families separates four implementations by up to 51 points of recall. Reported accuracies therefore cannot rank detectors. We close with open problems and reporting recommendations.
Files
Institutions
- DAV UniversityPunjab, Jalandhar