Argentina and Chinese university students quiz attemps and responses on ten D-48 test and LLM perceptions
Description
Would ChatGPT outperform your outstanding performance? Overestimation of LLMs' capabilities in a standardized abstract reasoning test in Argentina and China. The study was conducted during 2025. With Chinese and Argentine University students. A self-administered online test was conducted to gather the data. This is a .csv of anonymized quiz attempts and participant metadata collected from an interactive web-based quiz, containing a standardized cognitive test activity (ten domino-style quizzes from D-48 test). The survey had four parts: -An informed consent form. -An initial questionnaire. -A standardized cognitive assessment. -A second questionnaire featuring the main research question (perceptions regarding the results a GenAI tool would achieve on the same cognitive test), along with others designed to explore the same topic in greater depth.
Files
Steps to reproduce
Steps to Reproduce the Study and Data Collection Procedure To replicate the experimental protocol, user experience, and data structure generated via domino2025english.netlify.app, follow next steps. Phase 1: Participant Onboarding & Informed Consent Access Web Application: Participants open domino2025english.netlify.app on a desktop or tablet web browser. Informed Consent & Instructions: Participants read the study protocol, data anonymization policy, and instructions explaining that they will solve a non-verbal sequence logic test using domino rules. Demographic Questionnaire: Participants complete background queries regarding age, gender, institution/country, university major, programming experience, and prior frequency of LLM/GenAI tool usage. Phase 2: Human Abstract Reasoning Task (D-48 Test) Practice Item: Participants solve a guided sample item to familiarize themselves with the interactive interface (selecting top/bottom domino values from 0 to 6). Standardized Test Execution: Participants independently solve 10 sequential abstract logic items selected from the standard D-48 Domino test under non-timed or loosely timed conditions. Phase 3: Metacognitive Prediction & AI Expectation Task Self-Performance Assessment: Upon finishing the 10 items, participants are given feedback about their performance.. LLM Capability Prediction: Participants are prompted with the question:"How many items out of 10 do you predict ChatGPT (GPT-4) would answer correctly if given the exact same visual domino reasoning tasks?" Data Submission: Participants submit their response (0 to 10) and the last part of the survey is conducted. The system logs responses directly to the secure backend database.
Institutions
- Universidad de Buenos AiresBuenos Aires F.D., Buenos Aires