Query-Free OpenWebText - Part 2: SPARQL Injection Corpus
Description
This dataset is derived from OpenWebText and filtered so that it does not contain any SQL, SPARQL or Cypher in the plain text. This is Part 2 of the Query-Free OpenWebText dataset, which contains SPARQL-injected versions. SPARQL injected versions are derived from OpenWebText-clean The OpenWebText SPARQL Injection Corpus consists of two variations (Injected-10% and Injected-30%) derived from the Clean OpenWebText dataset and augmented with SPARQL queries. This process was designed to simulate controlled pre-training bias (RQ2). Dataset Format: Arrow
Files
Steps to reproduce
The base data for injection is the Clean OpenWebText corpus. This corpus was created using the original OpenWebText dataset (Gokaslan and Cohen, 2019) after applying a sequential, rule-based filtering workflow to ensure the resulting dataset contained no prior exposure to SQL, SPARQL, or Cypher syntax. Source corpora: - Plain text: OpenWebText dataset (Gokaslan and Cohen, 2019) - SPARQL queries: LSQ 2.0 (Large Scale SPARQL Query dataset) (Stadler et al., 2024) Filtering Mechanism: Sequential application of regular expression filters (matching SELECT...FROM, SELECT...WHERE, MATCH...RETURN, as detailed in Table 2) and a list of query keyword filters (e.g., "JOIN", "FILTER", "SQL"). Software/Workflow Custom Python scripts using the re library for pattern matching, performed prior to T5 tokenization. The injection was performed on the previously prepared Clean OpenWebText corpus (https://data.mendeley.com/datasets/m2gdxppkvj/1): Given the clean OpenWebText corpus, where every sample is free of relational or graph query syntax, 1) Randomly select 10% (30%) of samples for replacement 2) Replaced entirely by the sampled SPARQL queries from LSQ2.0. The remaining 90% (or 70%) of the documents remained as clean natural language text.
Institutions
- Yokohama Kokuritsu Daigaku