Corpus and coding record for: Word Embedding Techniques in NLP: A Comprehensive Review and Historical Perspective

Description

Supporting data for "Word Embedding Techniques in Natural Language Processing: A Comprehensive Review and Historical Perspective" (Expert Systems with Applications). The paper's central claim is a negative one: the movement of word embeddings from standalone research output to internal system layer is a continuous accumulation rather than a watershed. Fitting year continuously shows sustained change, with the share of techniques that are variants rather than distinct proposals rising at an odds ratio of 19.5 per decade and team size at 1.7, yet of four candidate breakpoints (Transformer, BERT, GPT-3, ChatGPT) none improves on the smooth trend for five of seven outcomes, and each surviving result breaks at a different year. That claim is only as good as the corpus it is measured on, because a corpus assembled by search would carry the field's own assumptions about which years mattered into the result. The corpus here is therefore built by a stated rule rather than by search: 30 seed reviews, expanded to a register of 95 techniques and 134 anchor papers, under a written protocol fixed before screening began. This deposit is the record of every decision that rule produced. Contents: the protocol with every query as executed; the codebook with its binding worked examples; a run log carrying every count the manuscript reports, one row per screening stage; the coded technique register and corpus manifest; and the analysis scripts that re-derive every reported quantity from them offline. Any count in the paper traces to a row of run_log.csv, and any technique-level code to its row in the register with the coder's note attached. This is a systematized review (Grant & Booth), not a systematic review, and the protocol was fixed prior to screening, not pre-registered. Full-text PDFs are not redistributed (copyright); every paper is identified by DOI or OpenAlex id in data/corpus_manifest.csv.

Steps to reproduce

1. Unzip the deposit and change into its root directory. 2. Install the dependencies. Versions are pinned in analysis/requirements.txt so that a re-run reproduces the reported numbers exactly rather than approximately. Tested on Python 3.13.6. pip install -r analysis/requirements.txt 3. Run the analysis chain: ./analysis/run_analysis.sh Fourteen steps, a few seconds, and no network access: the OpenAlex responses it reads are cached in data/openalex_cache.json, so the run is deterministic and does not depend on what the index holds today. 4. Outputs are written to results/ and figures/, overwriting the deposited copies. That is intentional. If a number moves against the deposited file, the deposit was wrong. 5. To check a specific claim: every count in the manuscript is a row of run_log.csv, one row per stage with its exclusion reason. Every technique-level code traces to its row in data/phase2_technique_register.csv with the note the coder attached to it. What cannot be reproduced from this deposit, by design: the Phase 1-6 retrieval scripts that queried OpenAlex, fetched PDFs and built the coding sheets are not included, since re-running them is a retrieval exercise against a moving index rather than a check on anything the paper claims. Their outputs are in data/. Two further checks read the full-text PDFs, which are copyrighted and not redistributed: the page-1 title verification and the first-author record-integrity check. Both results are deposited, in run_log.csv and results/record_integrity.csv. The pdftotext (poppler) system dependency noted in requirements.txt is needed only for those two steps and not for anything reproducible here.

Institutions

• Texas Tech University

  Texas, Lubbock
• University of Pittsburgh

  Pennsylvania, Pittsburgh
• Florida Atlantic University

  Florida, Boca Raton

Categories

Licence