Hybrid Residual Reinforcement Learning Control of a Fuel Cell Converter

Description

The MATLAB/Simulink models, controller scripts, multi-seed metric workbooks, and figure-generation scripts that support the findings of this study are available here.

Steps to reproduce

This document is a “how to reproduce” guide for the MATLAB package behind this paper’s PI-versus-SAC comparison. Its central idea is evaluation first: rather than retraining agents (slow and stochastic), to re-evaluate the saved agents on the same common plant and regenerate the metrics and figures from the current code. What is in the package: five files, agent_seed_1.mat to agent_seed_5.mat, each holding a trained SAC agent, its seed, the training configuration, training statistics and an embedded metric structure (M). Only seeds 1–5 are supplied, whereas the manuscript reports ten-seed statistics, so the ten-seed results cannot yet be fully reproduced from what is included. How to run it: unzip to a writable folder, keep the helper files together, open MATLAB with the Reinforcement Learning and Deep Learning Toolboxes, set the current folder to publishable_package, confirm the helper functions and agent files are found, then run EVALUATE_SAVED_AGENT_SEEDS. The script checks that key configuration fields stored in the agents agree with the plant code, chooses a median-ranked “representative” agent using the composite score, compares that agent with PI on the common averaged plant, and re-evaluates every agent file present. No training happens. Outputs: a wrapper .mat containing the representative agent, a results workbook (time series, comparison, fairness audit, re-evaluation metrics), a “FIXED” workbook with recalculated metrics and publication figures, an audit workbook of the metrics that were already embedded in the MAT files, and a folder of PNG plots. The critical warning (section 4): the metrics embedded in the five MAT files do not match the ten-seed values quoted in the manuscript. For example, RMSE is 0.6436 V in the files versus 0.6513 V in the paper, overshoot 1.99% versus 1.79%, and reference settling 3.20 ms versus 3.35 ms. The guide says to treat the embedded values, the archived workbook and the newly computed values as three separate sources, and to report only numbers generated by the final declared code and configuration. Completing the protocol: either add compatible agents for seeds 6–10 and rerun the evaluator (checking they use the same observation/action definitions, governor, plant, event profile and metric definitions), or run RUN_REPRODUCTION_WORKFLOW to train ten fresh agents, which takes hours and will not reproduce the archived numbers exactly. Protocol to preserve: identical plant and limits for PI and SAC; the fixed event sequence (0.5 s horizon, 100 µs step, 70 V reference, load steps at 0.08 s and 0.22 s, 75 V step at 0.30 s, sinusoidal load from 0.34 s); the residual-plus-governor control path; and honest reporting of the ripple trade-off. Audit trail: archive MATLAB and toolbox versions, source, all ten agent files, configuration, raw time series, scripts and figures; keep the original workbook separate; investigate and document any discrepancy rather than editing metrics.

Institutions

• Universiti Tenaga Nasional

  Selangor, Kuala Selangor

Categories

Licence