Human-Annotated Nursery Rhyme Emotions

Description

This dataset contains 308 English nursery rhymes independently annotated by three human annotators using six dominant-emotion categories: Happy, Relaxed, Sad, Tense/Fear, Powerful, and Longing/Nostalgia. Each nursery rhyme was treated as a single annotation unit and assigned exactly one dominant-emotion label based on its complete lyrical, narrative, and contextual meaning rather than on isolated emotional words or expressions. The nursery-rhyme texts were sourced from the publicly available English Nursery Rhymes dataset created by Terence Broad and distributed through Kaggle. The present dataset constitutes a derived annotation resource in which the original rhyme texts were retained and enriched with human-generated dominant-emotion labels, annotator-level decisions, agreement information, and reliability analyses. The final emotion label was determined through unanimous agreement or majority voting. Of the 308 records, 242 (78.57%) achieved unanimous agreement, while 66 (21.43%) were resolved through majority voting. No cases of complete disagreement occurred. Inter-annotator reliability was assessed using Fleiss’ kappa, yielding a coefficient of 0.7886. The repository contains the final dataset in CSV and XLSX formats, a merged annotation-audit workbook, annotation instructions, three analysis scripts, and supporting documentation. The final dataset files provide the consolidated emotion labels for reuse, while the audit workbook preserves annotator-level decisions, agreement patterns, final labels, and resolution information. The annotation instructions document the category definitions and decision procedures, and the analysis scripts support reproduction of Fleiss’ kappa, pairwise Cohen’s kappa, and pairwise confusion-matrix analyses. The README file provides an overview of the dataset structure, deposited files, emotion labels, and intended uses. The original class distribution was retained without resampling or synthetic augmentation. The dataset can support research in textual emotion classification, affective computing, computational linguistics, children’s literature, nursery-rhyme analysis, annotation reliability, and the development or evaluation of natural language processing models. The supporting materials enable users to inspect the individual annotation decisions, verify the final-label resolution process, reproduce the reported reliability analyses, and conduct further methodological or computational investigations.

Steps to reproduce

1. Download all files from the repository and place them in the same working directory. 2. Use Annotation_audit_decoding_merged_final.xlsx as the input file for the agreement analyses. The workbook contains the decoded emotion labels assigned independently by the three annotators. 3. Run 01_fleiss_kappa_analysis to calculate the overall inter-annotator agreement using Fleiss’ kappa. 4. Run 02_pairwise_cohens_kappa_analysis to calculate pairwise Cohen’s kappa for Annotator 1–Annotator 2, Annotator 1–Annotator 3, and Annotator 2–Annotator 3. 5. Run 03_pairwise_confusion_matrix_analysis to generate pairwise confusion matrices and examine agreement and disagreement across the six emotion categories. 6. Ensure that each script refers to the correct worksheet and annotator-label columns in Annotation_audit_decoding_merged_final.xlsx. 7. Compare the generated outputs with the results reported in the accompanying manuscript and repository documentation. The expected overall Fleiss’ kappa coefficient is approximately 0.7886. 8. The consolidated dataset for reuse is provided in Nursery_rhyme_final_dataset.csv and Nursery_rhyme_final_dataset.xlsx. These files are not required as input for reproducing the inter-annotator agreement analyses.

Institutions

• State University of Malang

  East Java, Malang

Categories

Licence