Reproducibility package for "Where 5G Throughput Prediction Actually Fails: A Transition-Aware, Leakage-Audited Evaluation on Real Drive-Test Traces"

Description

Reproducibility package for the paper "Where 5G Throughput Prediction Actually Fails: A Transition-Aware, Leakage-Audited Evaluation on Real Drive-Test Traces" (submitted to Future Internet, MDPI). It contains the analysis code and all derived artifacts (processed 1 Hz features, out-of-fold predictions, metric tables, and publication figures) needed to reproduce every number, table, and figure in the paper. The study audits how machine-learning models for short-horizon 5G downlink-throughput prediction are evaluated: the optimism of random-split versus leave-sessions-out evaluation, the dominance of autoregressive self-lag features, where error concentrates under absolute versus scale-invariant metrics, and whether lightweight transition-context features help (a null result). All headline numbers carry per-session bootstrap confidence intervals. This record does NOT redistribute the raw data; the pipeline regenerates the processed features from the public Raca et al. (2020) 5G production dataset (https://github.com/uccmisl/5Gdataset). Requirements: Python 3.12, CPU-only (numpy, pandas, scikit-learn, scipy, matplotlib, pyarrow); all randomness is seeded. See README for run order. Code is MIT-licensed.

Steps to reproduce

STEPS TO REPRODUCE ================================================================================ Reproducibility package for the paper: N. N. Sirhan, "Where 5G Throughput Prediction Actually Fails: A Transition-Aware, Leakage-Audited Evaluation on Real Drive-Test Traces," submitted to Future Internet (MDPI). 1. ENVIRONMENT Python 3.12, CPU-only. Create an environment and install dependencies: pip install -r requirements.txt (numpy, pandas, scikit-learn, scipy, matplotlib, pyarrow). All randomness is seeded (random_state=0), so results are deterministic for a given scikit-learn version. Minor last-digit differences may occur across library versions. 2. OBTAIN THE RAW DATA (not included here) Download the public Raca et al. (2020) 5G production dataset from https://github.com/uccmisl/5Gdataset and unzip 5G-production-dataset.zip. Set the RAW path at the top of the scripts to the resulting folder (it should contain the Download/, Netflix/, and Amazon_Prime/ session subfolders). 3. RUN THE PIPELINE (from the package root, in this order) python code/01_explore.py dataset summary, session index, missingness. python code/02_probe.py zero-inflation by app, throughput by RAT, raw transition check. python code/03_build_features.py builds the 1 Hz feature table (artifacts/processed/download_1hz.parquet). python code/04_experiments.py RQ1 (random/temporal/LOSO across horizons), RQ2 (self-lag ablation), RQ4 (transition-feature remedy); writes metric tables and out-of-fold predictions. python code/05_rq3_analysis.py RQ3 diagnostics: error concentration, decile bias, Lorenz/Gini, event-aligned bias. python code/07_robustness.py streaming-session robustness check. python code/08_reviewer_response.py per-session bootstrap confidence intervals, scale-invariant error, naive/smearing/linear retransformation comparison, Random-Forest replication. python code/09_linear_event_aligned.py linear-space event-aligned curve used in the upgrade-bias figure. python code/06b_figures_pub.py regenerates all publication figures (fig1-fig7). 4. VERIFY Steps 1-3 (feature build) must run before the rest. Headline values can be checked against: artifacts/tables/key_numbers.json artifacts/tables/reviewer_response.json which are the machine-readable sources of every number reported in the paper. The provided artifacts/ folder already contains all outputs, so the figures can be regenerated (final step) without re-running the models. NOTES - Throughput is modelled in log1p(kbps); errors are reported in Mbps. - Splits: random 5-fold; per-session temporal (70/30); leave-sessions-out (5-fold GroupKFold by session). - No number in the paper is hand-entered; each traces to a file under artifacts/.

Institutions

• Petra University

  Amman, Amman

Categories

Licence