Memorisation-Aware Evaluation of a Leakage-Free Feature-Selection Framework for Android Malware Detection
Description
Android malware detectors are usually evaluated with one feature-selection method, one classifier and a random split, leaving open what drives a reported figure and whether it holds for unseen applications. This paper designs and implements a robust detection framework from feature-selection methods and classification techniques, and evaluates it against signature-based and behaviourbased baselines. The framework removes duplicate and conflicting records, fits every reduction and scaling step inside the training folds, and classifies with a gradient-boosted tree ensemble on all attributes; correlation-based selection of 16 attributes is a cost-control mode. Its components are chosen by a factorial study of ten reduction methods and ten classifiers on five public datasets: at default settings the classifier explains 33.3% of the variation in the Matthews correlation coefficient (MCC) against 9.6% for the reduction method, and the configuration chosen from four datasets is within 0.014 MCC of the best on the fifth. A closedform diagnostic predicts, before training, the share of test applications already seen in training under a random split; on four datasets it is within 0.6 percentage points of the observed share (44.55% against 44.57% on Droidware), and it predicts the recall of an exact-match signature. On every dataset the framework’s MCC is lower on unseen than on seen applications (0.531 against 0.986 on TUANDROMD). On new applications an exact-match signature detects nothing and a tolerant signature reaches 0.50–0.70 MCC, against 0.86–0.97 for the framework; on behavioural attributes the framework exceeds the learner families of behaviour-based detectors by at least 0.060 MCC