Research Gap Discovery Dataset: LLM-Extracted Limitations and Research Gaps from 3,326 Scientific Papers
Description
This dataset contains 3,326 recent arXiv computer science papers (first versions dated 16 March to 17 July 2026). For each paper, a large language model read the title and abstract and wrote one sentence each on the paper's main limitation, an open research gap for future work, and why that gap matters. It was built to support research on automated research gap discovery. How the data was gathered: papers were retrieved from the arXiv API with 40 keyword queries in cs.CL, cs.AI, cs.LG, cs.CV, cs.IR, cs.HC, cs.RO and cs.NE, and de-duplicated by Paper ID. Each title and abstract was sent to mistral-small-latest (Mistral AI, 3,117 papers) or llama-3.3-70b-versatile (Groq, 209 papers) with the same prompt (max_tokens 200). The Generator Model column records the model for each row. Fields: Serial, Paper ID, Version, Title, Published, Primary Category, Categories, Abstract, Limitation, Research Gap, Importance, Gap Refers To Own Work (heuristic flag), Generator Model and Split (train/validation/test, 80/10/10, stratified by category and model, seed 42). The JSON file also contains the raw model output. How to use it: the data can be used to study research gaps across fields, to cluster or rank gaps, and to train or evaluate models that write limitations and research gaps from abstracts. Use the Split column for comparable results, and fit text vectorizers on the training split only. Interpretation: the text fields are inferred by an LLM from abstracts and have not been validated by human experts. See README.md for the exact prompt, column definitions, usage notes and known limitations. Changes in version 2: - Complete arXiv categories (30 missing values filled, 279 truncated lists restored, official case) and new Primary Category column - Generator Model column replaces API Used - Abstract included in the CSV and Excel files (previously JSON only) - Text cleanup of LaTeX, markdown and punctuation in 565 cells - New columns: Version, Gap Refers To Own Work, Split - Rows sorted by date; use Paper ID to match rows with version 1 - Same 3,326 papers; text content unchanged apart from formatting
Files
Steps to reproduce
1. Scientific papers were collected from the arXiv API with 40 keyword queries (e.g. "cat:cs.CL AND (limitation OR future work)") across cs.CL, cs.AI, cs.LG, cs.CV, cs.IR, cs.HC, cs.RO and cs.NE, sorted by submission date (newest first, up to 300 results per query), and de-duplicated by Paper ID. 2. The title and abstract of each paper were given to a Large Language Model through structured prompting: Mistral-Small (mistral-small-latest, Mistral AI API; 3,117 papers) or LLaMA-3.3 (llama-3.3-70b-versatile, Groq API; 209 papers), with max_tokens 200. The exact prompt is in README.md. 3. The LLM was asked to write one sentence for each of three components, which were then split into separate columns: - LIMITATION: the main weakness of the paper - RESEARCH GAP: the unsolved problem for future researchers - IMPORTANCE: why the gap is important to solve 4. Version 1 results were stored in JSON, CSV and Excel formats. 5. Version 2 was built with build_v2.py from the version 1 files and the paper categories re-fetched from the arXiv API (arxiv_categories.csv): text cleanup, complete categories, new columns and a stratified train/validation/test split (seed 42). This step is fully reproducible. Note: Steps 1 to 3 cannot be repeated exactly, because arXiv search results change over time, LLM outputs are not deterministic, and the model names may now point to newer versions or be unavailable. Total papers: 3,326 Collection date: July 2026 Version 2 prepared: September 2026