A dataset of paired source-supported and source-contradicted Arabic claims generated from Iraqi Telegram posts
Description
Version 2 contains 536 pairs of Arabic claims (1,072 claims) generated from 536 unique public Iraqi Telegram posts across 17 channels. Each pair contains a claim annotated as supported by its referenced source post and a synthetic counterpart annotated as contradicted by that source. These labels express relation to the referenced post and do not establish independent real-world truth. Version 2 reassesses all 635 pairs in Version 1. It excludes 99 unsuitable or duplicate-source records, retains stable pair identifiers, canonicalizes public source links, and separates historical requested alteration types from human-reviewed alteration types. The retained data comprise 121 negation, 120 place, 98 organization, 80 date, 69 person-name and 37 quantity pairs, together with 6 multi-edit and 5 unclear cases under explicit labels. The deposit provides pair-level CSV and JSON data, a long claim-level CSV file, annotation and adjudication records, a complete changelog from Version 1, a rehydration manifest, channel metadata, a codebook, generation prompts and available code provenance, file-integrity hashes, and an offline validation script. Full Telegram source bodies are not redistributed. Source recovery depends on continued public availability and authorized access. Synthetic contradicted claims must not be interpreted or circulated as verified allegations about named individuals.
Files
Steps to reproduce
1. Source collection. Public posts were fetched from Iraqi Telegram channels with the Telethon client. The first line of each message was taken as its headline and the rest as material for generation; timestamps were converted to Asia/Baghdad (UTC+3). Post bodies are not part of this deposit. Seventeen channels are represented here; every pair written from an eighteenth channel was removed, because the title it carries on Telegram does not match what its username suggests. 2. Pair generation. One call asked one model for both claims of a pair: one faithful to the post and one differing in a single factual element of a named type. Three open-weight models were used through the Groq API. allam-2-7b wrote 254 of the retained pairs, gpt-oss-120b 118 and gpt-oss-20b 64, while 100 pairs carry no recorded model. Both claims of a pair always come from the same call, so a within-pair difference cannot be attributed to model style. 3. Automated screening. Screening applied to the later batches only: 353 of the retained pairs come from the first batch, which went straight to human review. The gate extracted the declared element from each claim and kept the pair only if the two differed, and it rejected a faithful claim naming a year the source did not support. A later audit of that code found the date extractor matching substrings, so that the Arabic word for tomorrow is found inside the Arabic name of Baghdad. Passing the gate is therefore not evidence that a pair is false. 4. Human review. Every retained pair was reviewed by one annotator against the full source post, in two sheets: a blind sheet of 200 pairs, and a second sheet of 390 in which the source-relation and retention columns were prefilled and the type column was left blank. All judgments are in annotations_round2.csv, and the five rows left open are settled with their reasons in adjudication.json. Some counterparts were regenerated rather than discarded: 43 retained pairs carry a documented regeneration, 42 of them with changed text. Every pair excluded from Version 1 is listed with its reason in changelog_from_v1.csv. 5. Assembly and audit. Retained pairs were merged into one file and checked for unique identifiers, one pair per source post, non-empty and non-identical claims, claim text unique across records, a public permalink form, CSV/JSON identity, long-format correspondence, label consistency, recomputed counts and SHA-256 digests. reproduce_package.py runs these checks offline. 6. Verifying this deposit. Every count can be re-derived from claims.csv or claims.json alone: group by verified_type for the reviewed classes, by requested_type for the generation request, by source_channel for per-channel counts, and by the first seven characters of source_date for the monthly distribution.
Institutions
- Nahrain UniversityBaghdad, Baghdad