SplitRAG-KGQA: An Anonymized Knowledge Graph and Query Benchmark for Vertical-Domain Question Answering

Description

This dataset contains the anonymized research data used in the SplitRAG KGQA study. It includes an anonymized knowledge graph represented by numeric triples, generated query-answer pairs for five query topologies, prompts for five reasoning and retrieval strategies, and processed experimental results including F1 scores, ablation data, token-efficiency analysis, and Bonferroni-corrected statistical test results. Raw source pages, source URLs, and entity/relation name mappings are not included.

Steps to reproduce

1. Download and unzip the dataset package. 2. Use result/KGs.csv as the anonymized knowledge graph. Each row represents a triple in the form of head, relation, and tail, where all identifiers are anonymized numeric IDs. 3. Use generate_query/ to obtain the benchmark queries and ground-truth answers. The queries are organized by topology, including 1p, 2p, 2i, 2u, and 3i. 4. Use prompt/ to obtain the model inputs for each strategy, including RAG, Decomp, SplitRAG, CoT, and FS-CoT. Each strategy folder contains prompts organized by query type. 5. Run the corresponding prompts on the evaluated models, or directly inspect the processed results in result/f1_score/ and result/analysis/. 6. To reproduce the main tables and figures, use the files in result/f1_score/ for F1-score comparisons and the files in result/analysis/ for ablation, error recovery, token-efficiency, fine-tuning, and statistical-test analyses.

Institutions

• University of Chinese Academy of Sciences

  Beijing, Beijing

Categories

Licence