CVI/CVR Dataset Expert Study

Description

This dataset supports a Bangladesh-based expert content-validation study of the AI-informed Writing Assessment Rubric for EFL (AIWAR-EFL). The study examined whether a proposed Co-authorship Integrity domain, designed to address undisclosed generative AI contributions (“AI-giarism”), was a relevant and appropriate addition to a conventional analytic EFL writing rubric. Ten EFL, applied linguistics, language assessment, and academic integrity experts from Bangladeshi higher education evaluated 41 rubric domain, descriptor, and structural decision items via an online survey. Each item was rated using Lawshe’s three-category essentiality scale: Essential, Useful but not essential, or Not necessary. The dataset includes de-identified participant codes, item-level ratings, binary coding for Content Validity Ratio (CVR) calculation, item descriptions, and qualitative comments. CVR credits Essential ratings only and was interpreted against a .62 threshold for a ten-member panel. Item-level Content Validity Indices (I-CVIs) count both Essential and Useful but not essential responses as affirmative; the criterion was .78. The data show a limited, strict essentiality consensus: three of 41 items met the CVR threshold. However, relevance consensus was strong: 40 of 41 items met the I-CVI criterion and the overall S-CVI/Ave was .91. The Co-authorship Integrity domain-inclusion item achieved an I-CVI of 1.00, while Epistemic Transparency received the strongest essentiality support. These results indicate expert recognition of the domain’s relevance, alongside caution about treating every new descriptor as indispensable. The data may be used to replicate the content-validity calculations, assess alternative retention rules, or adapt the rubric in comparable EFL higher-education settings.

Institutions

• Jahangirnagar University

  Dhaka Division, Dhaka

Categories

Licence