Simulated wind-solar benchmark for cold-start hydropower forecasting and dispatch: hourly meteorological and generation data on a 36 N, 100 E grid, 1981-2024
Description
Input data of the cold-start wind-solar benchmark, published so that the accompanying software record can be reproduced from the data alone. It is read directly by Cold-start wind-solar benchmark: reproducible pipeline for probabilistic forecasting, scenario generation and scenario-value assessment, DOI 10.5281/zenodo.23132623. It covers a 0.5 degree grid of eight points between 36.00 and 36.50 N and between 99.50 and 100.50 E, an area of the upper Yellow River basin in Qinghai, China. The series is hourly from 22 December 1980 to 1 January 2025, 385,944 steps; 1980 is the spin-up head, so the study period begins on 1 January 1981. data_full.h5 holds 247 datasets of 385,944 points: 208 native ERA5 variables (26 parameters at each of the eight grid points, including wind speed at 10 m and 100 m, pressure, temperature, humidity, cloud cover, snow depth and surface radiation), 34 derived series (solar geometry, time encodings, air density at hub height, plant power), four simulated plant output series, three photovoltaic and one wind, and the time axis. Values are normalised; normalization_params.npz returns them to physical units. The ERA5 variables are reanalysis, a model product rather than a measurement, and the four plant output series are simulations derived from them, not metered generation of any real plant. The package is representative of the region and the technology and must not be read as a record of a particular facility. The output series were derived with Renewables.ninja and are licensed CC BY-NC 4.0; the remaining data are provided under the same terms. The package serves the cold-start setting, in which a model must be trained and evaluated with little or no local history: feature screening, point and probabilistic forecasting, scenario generation, and day-ahead stochastic dispatch. Because the generation series are simulated, the expected values are known, so the dataset works as a benchmark with a reference answer. data_full.h5 is the full package. normalization_params.npz holds the normalisation parameters. preprocess.py converts between the normalised and the physical form, derives the 34 additional series and verifies the package. query.py reads one series over a time window. params.local.csv configures the interpreter. run_data_check.bat runs the self-check by double-click. The scripts need only Python with numpy, pandas and h5py.
Files
Steps to reproduce
1. Download the files of this deposit and put them in one folder. 2. Install one Python 3.10 environment with the three packages the scripts use: <python> -m pip install numpy==1.26.4 pandas==2.2.2 h5py==3.16.0 3. Write the path of that interpreter into params.local.csv, in the python row: environment,python,<path to python.exe>,Python 3.10 interpreter 4. Double-click run_data_check.bat. It runs the whole check without further input and keeps the window open when it is done. From a console, the same check in four steps, about ten minutes in total: <python> preprocess.py info <python> preprocess.py inverse --src data_full.h5 --out _check\data_native.h5 --params normalization_params.npz <python> preprocess.py preprocess --src _check\data_native.h5 --out _check\data_features_raw.h5 <python> preprocess.py normalize --src _check\data_features_raw.h5 --out _check\data_full_rebuilt.h5 --params normalization_params.npz <python> preprocess.py verify --a data_full.h5 --b _check\data_full_rebuilt.h5 --params normalization_params.npz 5. Read the verdict of the last command. A correct copy prints: exactly equal: 246/246 every value agrees point by point. The comparison is made dataset by dataset, 246 of the 247 datasets numerically; the time axis is checked separately. The tolerance is a relative or absolute difference of 1e-5, the rounding margin of float32 storage and the log1p round trip. Any other verdict is reported with the affected datasets and the time points where they differ. 6. Inspect what was computed. The check leaves the package in physical units at _check\data_native.h5, so the normalised values in data_full.h5 can be read in m/s, W/m2 and degrees. The two pass-through files of the round trip are removed once the comparison is done. data_full.h5 itself is only read and is never modified. 7. Read individual series of the normalised package, for example the hourly wind output of plant qj over two days: <python> query.py --file data_full.h5 --var qj_wind_output --start "1990-05-21 22" --end "1990-05-22 03" To see the physical values instead, read the same series from _check\data_native.h5.
Institutions
- China Institute of Water Resources and Hydropower ResearchBeijing, Beijing