LLM Peculiarity
Description
This dataset accompanies the study “BART: Bipolar Argumentation Reasoning Topologies for Analyzing LLM Peculiarities.” It contains approximately 7,000 argumentative documents generated by ten open-weight and proprietary large language models. The experiments use 710 debate topics that are publicly accessible through the original DEBATunE repository and were introduced by Li et al. (2024). Each generated document is represented as a Bipolar Argumentation Framework graph. Nodes represent claims, premises, evidence, objections, and rebuttals, while directed edges represent support and attack relations. The deposited materials include the generated LLM outputs, parsed argument graphs, source-model labels, validation results, rejection information, topic-wise cross-validation splits, experimental configurations, and associated code. These materials support reproduction of the graph-classification experiments, pairwise and multi-class source-attribution results, and topological measurements, including hierarchy depth, branching factor, and attack ratio. The original DEBATunE topic collection is publicly accessible at https://github.com/tianyi-lab/DEBATunE and is not covered by the license applied to this deposit.
Files
Steps to reproduce
1. Prepare a plain-text list of debate topics. Users may provide any arbitrary topic list compatible with the input format expected by the script. To reproduce the deposited dataset specifically, obtain the 710 topics from the DEBATunE repository. 2. Clone the repository https://github.com/taghizad3h/peculiarity (will be available after paper being published) 3. Configure the required models and credentials. Open-weight models can be executed on a GPU server, while proprietary models are accessed through OpenRouter. API keys are not included in this deposit. 4. Run data_generator.py with the selected topic list. The script applies the argument-generation prompt to each topic and model and stores the generated structured argumentative documents. 5. Retain the topic identifier, model identifier, generated response, and generation status for each output. Record structurally invalid outputs as rejected rather than silently removing them. 6. Combine the accepted outputs and export them using the organization and format provided in this deposit. This procedure reproduces the dataset-generation process only, not the classification experiments or reported experimental results. Users can generate a new dataset from any suitable topic collection; however, reproducing the deposited dataset requires the original DEBATunE topics and model configurations. Because hosted models and inference providers may change, regenerated responses may not be identical to the deposited records.
Institutions
- Iran University of Science and TechnologyTehran, Tehran