WNSimRep: a framework and replication dataset for ontology-based semantic similarity measures and information content models
Description
The WNSImRep v1 dataset is provided as supplementary material of the paper by Lastra-Díaz, J. J., & García-Serrano, A. (2016). HESML: a scalable ontology-based semantic similarity measures library with a set of reproducible experiments and a replication dataset. Information Systems. In the aforementioned work, we introduce a scalable Java software library of ontology-based semantic similarity measures and IC models, called HESML, and a set of reproducible experiments on word similarity. The WNSimRep v1 dataset is detailed in the enclosed file called "appendixB_WNSimRep_dataset_LastraGarcia_v1.pdf". This work introduces a framework whose aim is to allow the exact replication of most intrinsic Information Content (IC) models and ontology-based similarity measures reported in the literature by using the publicly available accompanying dataset, called the WNSimRep v1 dataset. This work has been carried-out in the context of a large evaluation campaign of ontology-based semantic similarity measures and IC models on WordNet based on HESML. Our work is encouraged by the identification of several reproducibility problems in a series of recent experimental surveys carried-out by the authors, together with the lack of a framework and gold standard to assist in the replication of ontology-based similarity measures and IC models. To bridge this gap, we introduce herein a replication framework defined by three different types of data file: (a) node-based data files which contain an explicit representation of the WordNet taxonomy together with a specific IC model and a collection of node-based taxonomical features, (b) edge-based data files which contain a family of edge-valued IC models based on the conditional probability between child and parent concepts, and (c) synset-pair-based data files which contain the synset pairs of the Rubenstein-Goodenough word similarity benchmark, together with a collection of taxonomical features based on synset pairs and all the ontology-based similarity measures evaluated on them. The framework is implemented in the accompanying dataset which includes a collection of intrinsic and corpus-based IC models based on WordNet 3.0, enriched with a broad set of taxonomical features used by most intrinsic IC models and ontology-based similarity measures.
Files
Steps to reproduce
The WNSimRep v1 dataset has been created by using the HESML library introduced in the main companion paper below: [1] Lastra-Díaz, J. J., & García-Serrano, A. (2016). HESML: a scalable ontology-based semantic similarity measures library with a set of reproducible experiments and a replication dataset. Information Systems. Thus, in order to reproduce the WNSimRep v1 dataset, you should follow the next steps: (1) Obtain a copy of the paper [1] (2) Follow the instructions in the section on reproducible experiments.