Generative AI-Assisted Title and Abstract Screening
Description
This dataset supports the study titled "Do it Faster with PICOS: Generative AI-Assisted Systematic Review Screening." The research explores the impact of using an open-source Large Language Model (LLM), Mistral-Nemo-Instruct-2407, to generate structured PICOS (Population, Intervention/Exposure, Comparison, Outcome, Study Design) summaries for title and abstract screening in systematic reviews. The dataset includes article metadata, manually labeled inclusion/exclusion decisions by human reviewers, and LLM-generated structured PICOS summaries. This dataset can be useful for researchers working on AI-assisted evidence synthesis, systematic review automation, and human-machine collaboration in biomedical literature screening. The dataset consists of the following columns, capturing key metadata and screening decisions for systematic review articles: Title – The title of the research article. Authors – List of authors for the article. Journal – Name of the journal where the article was published. Year – Year of publication. Volume – Journal volume number. Issue – Journal issue number. DOI – Digital Object Identifier (DOI) for the article. Abstract – The abstract of the research article. PICOS Summary – Structured summary generated by the LLM Gold_std dataset – Ground truth classification (Include/Exclude) based on expert review. A1_reviewer – Screening decision by Reviewer A1 (LLM-assisted, less experienced). A2_reviewer – Screening decision by Reviewer A2 (Traditional, less experienced). B1_reviewer – Screening decision by Reviewer B1 (LLM-assisted, more experienced). B2_reviewer – Screening decision by Reviewer B2 (Traditional, more experienced).