SARMF-Bench: Reproducible Smart Contract Vulnerability Benchmark Dataset
Description
SARMF-Bench is a compact, reproducible benchmark dataset for smart contract vulnerability analysis. It contains five Solidity smart contracts aligned with canonical vulnerability classes: • Reentrancy • Arithmetic Overflow Behavior • Access Control Weakness • Unchecked External Call • Denial-of-Service Pattern Each contract is paired with machine-readable static analysis outputs generated using Slither v0.11.5. The dataset is designed to support reproducible benchmarking experiments for static analyzers, symbolic execution engines, fuzzers, and AI-based smart contract security tools. Primary development repository: https://github.com/profmohit-edu/sarmf-framework Software archive DOI: https://doi.org/10.5281/zenodo.18754015 Reproducibility protocol: https://doi.org/10.17504/protocols.io.bp216eyxdgqe/v1
Files
Steps to reproduce
Data Generation and Methodology SARMF-Bench was constructed through controlled manual design and static analysis validation workflows. 1. Contract Design Five minimal Solidity smart contracts were intentionally written to isolate canonical vulnerability classes aligned with the Smart Contract Weakness Classification (SWC) taxonomy: - Reentrancy - Arithmetic Overflow Behavior - Access Control Weakness - Unchecked External Call - Denial-of-Service Pattern Each contract was designed to contain a single dominant structural vulnerability to avoid confounding interactions. 2. Static Analysis Workflow All contracts were analyzed using Slither v0.11.5, executed in a controlled Linux environment (Ubuntu 24.04 LTS). Slither JSON output format was used to preserve: - Detector identifiers - Impact classification - Confidence levels - Source code mapping metadata 3. Data Extraction Machine-readable JSON outputs were exported directly from Slither and paired with their corresponding Solidity source files. 4. Reproducibility Protocol To reproduce: - Install Slither v0.11.5 - Run static analysis on provided Solidity contracts - Compare generated JSON outputs with benchmark JSON files included in the dataset - Validate detector consistency (impact/confidence fields) 5. Toolchain Dependencies - Solidity compiler (version compatible with Slither 0.11.5) - Python 3.x environment - Slither static analysis framework No synthetic generation tools, fuzzing engines, or AI-based mutation systems were used in dataset construction. All contracts were intentionally engineered for pedagogical clarity and benchmarking reproducibility. This dataset is intended as a controlled experimental baseline for evaluating static analyzers, symbolic execution tools, fuzzers, and AI-assisted smart contract security systems.
Institutions
- Bharati Vidyapeeth's College of Engineering, DelhiDelhi, New Delhi