Documentation Gap Analysis: Independent Audit of TGA COVID-19 Vaccine Safety Monitoring Plan
Description
Research context: The Therapeutic Goods Administration (TGA) COVID-19 Vaccine Safety Monitoring Plan (February 2021) was developed as part of Australia’s provisional approval framework for COVID-19 vaccines. The Plan described enhanced monitoring strategies intended to support ongoing safety surveillance during vaccine rollout. This dataset presents an independent documentary audit examining whether implementation of those commitments could be verified through available records. Research question: Whether the TGA COVID-19 Vaccine Safety Monitoring Plan could be independently verified as implemented through documentary evidence. The Plan specified 17 enhanced-pharmacovigilance outputs across five objectives defining enhanced monitoring for provisional approval, plus two governance outputs, resulting in 19 assessed outputs. Data and methods: Documentary evidence collected from 2022–2026 through FOI requests, OAIC decisions, Senate testimony, TGA publications and peer-reviewed literature. Evidence was assessed against ISO 19011:2018 and ANAO audit principles using the Plan’s 19 outputs as assessment criteria. The dataset includes the evidence matrix, classification criteria, codebook, variables, evidence tiers and rating scales used for assessment. The dataset contains an audit evidence matrix, including output-level classifications, evidence references, evidence tiers and severity ratings. The classification framework distinguishes between inability to verify implementation and evidence of non-occurrence. All assessed outputs, supporting evidence references and analytical classifications are documented to enable independent review of the assessment process. Findings: 3 of 19 outputs (16%) were fully implemented; 10 were partially implemented and 6 were not documented. The findings indicate that implementation of enhanced monitoring could not be independently verified from the available documentary record. They do not establish that no monitoring occurred, but identify documentation and traceability gaps that may warrant further examination. Scope and limitations: The dataset assesses documentary verifiability of implementation and does not evaluate the clinical effectiveness or safety outcomes of COVID-19 vaccines, nor does it infer operational absence solely from documentary gaps. Reproducibility: Summary statistics, including status distribution, objective classifications and severity aggregation, are reproducible using the underlying evidence matrix and standard analytical tools including spreadsheet software, R or Python. The assessment records the evidence and classification basis for each finding so that other researchers can review, challenge or reassess the conclusions if additional evidence becomes available. Related outputs: Data paper: https://doi.org/10.2139/ssrn.6610438 Complete audit repository (primary sources, methodology and version history): https://github.com/paulrekaris/TGA-COVID19-Vaccine-Safety-Monitoring-Audit
Files
Steps to reproduce
1. Identify assessment framework: Select a public policy, regulatory plan or government commitment with defined outputs. This audit examined the TGA COVID-19 Vaccine Safety Monitoring Plan (February 2021), comprising 19 assessed outputs (17 Plan strategies and 2 governance outputs). 2. Establish assessment criteria: Extract the stated outputs from the target framework and use them as assessment criteria. Develop an evidence matrix containing defined outputs, evidence requirements, classification criteria and rating scales. The assessment framework was informed by recognised standards and guidance, including ISO 19011:2018 auditing principles, ANAO and GAO audit principles, ISO 15489-1:2016 records management principles, relevant pharmacovigilance guidance including ICH E2E, WHO/EMA GVP and CIOMS, and FAIR data principles for research data management and reproducibility. 3. Collect documentary evidence: Systematically collect and record evidence from FOI requests and decisions, OAIC decisions and review materials, parliamentary records and Senate testimony, agency publications, partner surveillance sources, and relevant international guidance and literature. Maintain a source log recording document origin, date, relevance and evidence tier. 4. Apply assessment protocol: Assess evidence using the documented evidence hierarchy. Tier 1: OAIC decisions, FOI decisions and direct agency communications. Tier 2: Senate testimony, Questions on Notice and agency reports. Tier 3: Partner organisation and publicly accessible surveillance data. Tier 4: International guidance and contextual material. Assign implementation status using the documented criteria: Fully Implemented / Partially Implemented / Not Documented. Ratings are assigned based on available documentary evidence. Findings are cross-referenced across independent sources. 5. Reproduce analytical outputs: Use the evidence matrix and codebook to reproduce status distribution, status-by-objective cross-tabulation and severity rating aggregation. All headline statistics are directly derived from the underlying matrix and can be reproduced using spreadsheet software, Python, R or Stata. 6. Falsifiability and replication: The assessment records the evidence and classification basis for each finding so other researchers can review, challenge or reassess conclusions. Findings can be updated if additional documentary evidence becomes available. The dataset structure, codebook, source references and methodology documentation are preserved through version-controlled records to support independent verification and future reassessment. Methodological limitations, including reliance on available documentary evidence and the absence of statutory audit powers, are documented. 7. Verify and archive outputs: Review primary source references, methodology records and version history. Repository materials support independent verification, replication and any future reassessment.