Research Gap Discovery Dataset: LLM-Extracted Limitations and Research Gaps from 3,326 Scientific Papers
Description
This dataset contains 3,326 scientific papers collected from arXiv across NLP, AI, Computer Vision, and Machine Learning domains. For each paper, research gaps were automatically extracted using Large Language Models (LLaMA-3.3 via Groq and Mistral-Small via Mistral AI) through structured prompting. Each entry includes: - Paper ID (arXiv) - Title - Published date - Categories - Abstract - Limitation (main weakness of the paper) - Research Gap (unsolved problem for future researchers) - Importance (why the gap matters) - API used for extraction This is the first large-scale dataset specifically designed for automated research gap discovery from scientific literature. The dataset supports research in NLP, information extraction, and AI-assisted literature review. Collected: July 2026 Total papers: 3,326 Sources: arXiv (cs.CL, cs.AI, cs.LG, cs.CV, cs.IR, cs.HC, cs.RO, cs.NE)
Files
Steps to reproduce
1. Scientific papers were collected from arXiv using the arXiv API across categories: cs.CL, cs.AI, cs.LG, cs.CV, cs.IR, cs.HC, cs.RO, cs.NE. 2. For each paper, the abstract was extracted and processed using Large Language Models (LLaMA-3.3 via Groq API and Mistral-Small via Mistral AI API) through structured prompting. 3. The LLM was prompted to extract three components from each abstract: - LIMITATION: The main weakness of the paper - RESEARCH GAP: The unsolved problem for future researchers - IMPORTANCE: Why the gap is important to solve 4. Results were stored in JSON, CSV, and Excel formats. Total papers: 3,326 Collection date: July 2026