HMF-PC-MTP: Data and Reproducibility Materials for Correctness-Oracle-Free Testing of Vision-Based DNNs
Description
This dataset provides the data, analysis outputs, source data, and reproducibility materials supporting the study “Hierarchical Multi-Fidelity Pareto-Optimized Coreset Prioritization for Correctness-Oracle-Free Testing of Vision-Based Deep Neural Networks.” The repository supports the HMF-PC-MTP evaluation on four image-classification dataset–backbone settings: CIFAR-10 with ResNet-50, CIFAR-10 with ConvNeXt-Base, ISIC2019 with ResNet-50, and ISIC2019 with ConvNeXt-Base. Experiments use six nested per-class Top-K budgets (50, 100, 150, 200, 300, and 400) and optimizer seeds 40–59. The archive contains run-level and derived analysis tables, budget-matched baseline results, statistical summaries, mechanism-oriented analyses, runtime measurements, figure source data, the r6 manuscript figure-generation code, reproducibility and provenance manifests, RQ–contribution–evidence mappings, and SHA-256 checksums. These materials support the reported analyses of confirmed-failure revelation (SCFW), selected-set representation quality, diversity, MR balance, redundancy, multi-fidelity Pareto policy behaviour, selective Grad-CAM evidence usage, and runtime characteristics. The prioritization protocol is correctness-oracle-free rather than label-free: ground-truth class labels are used only to define fixed evaluation strata, while source correctness, follow-up correctness, and confirmed metamorphic-violation outcomes are excluded from optimization and ranking and are revealed only for post-selection evaluation. The original CIFAR-10 and ISIC2019 image datasets are third-party public benchmark data and are not redistributed in this archive. They should be obtained from their original providers. The deposited archive contains the generated experimental records and analysis materials necessary to reproduce the findings reported in the associated manuscript.