Second-by-Second Sensitivity Analysis of Response Time in Online Psychological Research
Description
The two data files contain the datasets used in the two studies reported in the article How Fast Is Too Fast? Experimental Validation and a Second-by-Second Sensitivity Analysis of Response Time Thresholds for Detecting Unreliable Responding to Online Surveys. Study1_data.xlsx contains data from 235 participants, comprising 185 respondents who completed the survey under standard instructions and 50 who were instructed to respond as quickly and randomly as possible, together with their survey and questionnaire completion times. The file also includes timing information for the EAI-3, EDS-R, OCI-R, SCOFF, and TIPI measures used to experimentally compare attentive and instructed-careless responding. Data5062all100percentComplete(1).xlsx contains the Study 2 dataset of 5,062 fully completed responses from physically active adults across 54 countries and 13 survey languages, including response-time, demographic, and psychometric data. These data were used for the second-by-second sensitivity analyses of response-time thresholds from 1 to 10 seconds per item and the associated comparisons of scale scores, internal consistency, intra-individual response variability, and Mahalanobis distance.
Files
Steps to reproduce
To reproduce Study 1, compare total and questionnaire-specific completion times between the standard-instruction group and the instructed-careless group using two-sided Mann–Whitney U tests, with Benjamini–Hochberg correction for multiple page-level comparisons. Assess robustness by repeatedly drawing 1,000 random control-group subsamples of 50 participants and recomputing the same group comparisons against all 50 instructed-careless participants. For Study 2, calculate each respondent’s seconds-per-item value from total completion time and classify respondents as “fast” or “adequate” at each integer threshold from 1 to 10 seconds per item. At every threshold, compare the groups on scale means using Mann–Whitney U tests with rank-biserial correlations, compute Cronbach’s alpha separately by group, and compare intra-individual response variability and Mahalanobis distance. Finally, repeat the threshold analysis in the English-language subsample (n = 400) to assess whether the pattern is reproduced within a single-language group.
Institutions
- Széchenyi István UniversityGyőr-Moson-Sopron, Győr
- University of PaduaVeneto, Padua