LLM NetOps/AIOps Six-Pillar Evidence Audit Dataset

Description

This dataset accompanies the survey “Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety” (arXiv:2605.12729, https://arxiv.org/abs/2605.12729). It contains a record-level evidence audit of 185 claim-linked sources used to study LLM-enabled NetOps and AIOps across operational capability, architecture and tool grounding, assurance and control, evaluation, security, and human or organisational constraints. The data distinguish a three-layer scope hierarchy from four evidential statuses and six non-overlapping primary analytical pillars. Separate files identify 28 direct LLM-facing records, 13 operational systems used for autonomy analysis, 18 final-search additions, 14 transparent exclusions, and 6 contextual comparison surveys outside the quantitative denominator. The release includes machine-readable CSV and JSON files, a formatted workbook, a data dictionary, a matching 191-entry bibliography, citation metadata, a CC BY 4.0 licence, and integrity checksums. It contains no source full text, reviewer correspondence, private contact details, or internal compilation material.

Steps to reproduce

This dataset documents a retrospective, record-level claim-source audit conducted for a structured integrative review. Each of the 185 retained sources is assigned one non-overlapping primary analytical pillar for quantitative reporting and may also receive descriptive secondary tags for cross-cutting relevance. Evidential status defines the types of claims that each record may support. To reproduce the reported pillar distributions, count each record in `Evidence_Base_185_Public.csv` once according to its `Primary analytical pillar` and divide the resulting counts by 185. Secondary tags must be excluded from these percentages. Evidential-status totals can be reproduced by grouping the `Evidential status` field. Direct LLM-facing evidence can be checked against `Direct_LLM_28.csv`. Autonomy results must be calculated only from the 13 direct operational systems listed in `Operational_Systems_13.csv`. All resulting totals can be compared with `Corpus_Summary.json` and `Pillar_Distributions.csv`. The final literature update is documented in `Search_Update_18.csv` and `Update_Search_Log.csv`. The release supports reproduction of the reported corpus classifications and quantitative summaries. It does not reconstruct the complete historical search process because source-level retrieval totals, independent duplicate screening, inter-rater statistics, and a registered protocol were unavailable.

Institutions

• Lancaster University

  England, Lancaster
• University of Cambridge

  England, Cambridge
• Nanjing University of Information Science and Technology

  Jiangsu, Nanjing
• TU Wien

  Vienna, Vienna

Categories

Related Links

Licence