Evaluating Open Information Extraction on Scientific and Medical Text

Published: 16 Feb 2018 | Version 2 | DOI: 10.17632/6m5dyx4b58.2

Description of this data

This dataset is the result of applying crowd sourcing to the extractions of two open information extraction tools (Open IE 4 and MinIE) linked below. Extractions were performed on both a set of random sentences from Wikipedia and randomly selected sentences from the OA-STM corpus.

The aim is to evaluate the effectiveness of open information extraction tools on scientific and medical text.

The initial datasets, the code for applying information, the HITS, labelling instructions, and analysis code are all included above.

Experiment data files

Steps to reproduce

The datasets used as input can be found under the data folder. Note that we use the Wiki.zip file linked below as one input.

The code for applying the information extraction tools can be found above. After installing the dependencies listed in build.sbt the extractions can be generated. This will generate the HITs used for the experiment that can be uploaded to Amazon Mechanical Turk using turktools (http://turktools.net). The hits we generated are in the file hittest-20171108-publicexperiment.zip. The labelling instructions we used are provided above.

After running the crowd sourcing experiment, one can preform the analytics that we did for our experiments using AnalysisAnnotationResults.ipynb.

Related links

Latest version

  • Version 2


    Published: 2018-02-16

    DOI: 10.17632/6m5dyx4b58.2

    Cite this dataset

    Groth, Paul; Lauruhn, Mike; Scerri, Antony; Daniel, Ron (2018), “Evaluating Open Information Extraction on Scientific and Medical Text”, Mendeley Data, v2 http://dx.doi.org/10.17632/6m5dyx4b58.2


Views: 2017
Downloads: 111

Previous versions

Compare to version


Elsevier BV


Natural Language Processing, Information Extraction


CC BY 4.0 Learn more

The files associated with this dataset are licensed under a Creative Commons Attribution 4.0 International licence.

What does this mean?

You can share, copy and modify this dataset so long as you give appropriate credit, provide a link to the CC BY license, and indicate if changes were made, but you may not do so in a way that suggests the rights holder has endorsed you or your use of the dataset. Note that further permission may be required for any content within the dataset that is identified as belonging to a third party.