The Transparency Paradox: Decoupling Perceived and Behavioral Trust through Trust Calibration in Human-AI Interaction

Description

The research data contained in Dataset_TrustCalibration.csv represents the experiment results used to investigate the dynamic process of trust calibration in human-AI interaction. This dataset captures how participants' perceptions and behaviors shift over multiple interaction rounds, especially following a trust-damaging system failure. Data Structure and Variable Descriptions The file Dataset_master_TrustCalibration.csv is organized into columns that correspond to demographic data, psychometric scales, and behavioral performance metrics: Experimental Grouping: -Condition: Indicates whether the participant was in the XAI condition (AI predictions + explanations) or the baseline condition (AI predictions only). - Demographics -(Q1 Series):Q1_1 (Gender): 1 for Male, 2 for Female. -Q1_2 (Age): Categorical ranges (e.g., 18~25, 31~40). -Q1_4 (AI-related Job): 1 for Yes, 2 for No. Psychometric Scales (Q2, Q4, Q6 Series): - Q2_1 to Q2_18 (AI Literacy): 18 items from the Meta AI Literacy Scale (MAILS) measuring conceptual understanding and tool usage. - Q4_1 to Q4_3 (Perceived Interpretability): Items measuring how transparent the AI's logic appeared to the user. - Q6_1 to Q6_5 (Perceived Trust): Measures the user's psychological confidence in the AI after the system error.Behavioral Performance (R Series): - R1 to R4 (Rounds 1–4): Each round contains: _I (Initial Estimation): The participant's first age estimate. _A (AI Prediction): The suggestion provided by the DEX model. _F (Final Judgment): The participant's revised estimate after seeing the AI's feedback. - Trust Calibration Metric (WoA Series): - WoA_R1 to WoA_R4: The Weight of Advice for each round. This is the primary measure of behavioral trust. How to Interpret the Data When using Dataset_TrustCalibration.csv, the following patterns are critical for interpretation based on the study's findings: 1. The Perception–Behavior Gap: Look for instances where Q6 (Perceived Trust) scores remain high while WoA (Behavioral Trust) scores drop significantly in Rounds 3 and 4. 2. The Impact of System Error: Round 2 introduced a deliberate error. A successful trust calibration is indicated by a lower WoA in Round 3 (reflecting warranted skepticism) paired with a lower Mean Absolute Error (MAE) (reflecting better decision accuracy). 3.Moderation Analysis: Compare the Q2 (AI Literacy) total scores with Q4 (Interpretability) to observe how literacy acts as a cognitive scaffold for novices in the XAI condition. 4.Cognitive Effort: While not a direct column, the increased Total Completion Time reported in the study for the XAI group indicates the activation of analytical System 2 processing. Researchers can utilize Dataset_TrustCalibration.csv to replicate the path analysis or mixed ANOVA used to identify the Transparency Paradox, where making an AI more understandable leads users to be more critical of its failures.

Institutions

• Seoul National University

  Seoul, Seoul

Categories

Licence