Reducing Endogeneity Within a Small-n Design Using Cross-Validation with Cluster-Robust Variance Estimation When Comparing Mixed Effect Models: A New Analytic Methodology Applied in Neurobiology

Published: 16 September 2024| Version 1 | DOI: 10.17632/hxv4mwfmry.1
Contributors:
,
,
,
,

Description

Data from this pilot study was purposed initially to assess post-synaptic behavioural effects of neuromodulation via intermittent theta-burst stimulation in combination with N-methyl-D-aspartate (NMDA) and Gamma-aminobutyric Acid (GABAa) receptor-modulating psychopharmacology on implicit motor learning outcomes. In the current study, a secondary exploratory analysis using cross-validation (both Replicated k-Fold and Leave-One-Cluster-Out) was conducted to compare linear mixed effect models to reduce errors caused by clustered endogeneity. Afterwards, cluster-robust variance estimation was added to this finalized working model to control for endogeneity present in the study design. Initially, the study's design comprised a small sample size (n = 13) in a clustered randomized trial design, which was collected at Butler Psychiatric Hospital in Providence, Rhode Island, under the principal investigation of Dr. Joshua Brown for the Department of Psychiatry at Brown University. Following the preprocessing and organizational stages of data preparation, data from a limited number of participants (n = 8) was retained as a change was made in the block order protocol during data collection. Participants were randomly assigned to four different drug conditions (Lorazepam (2.5 mg; LZP) vs. D-Cycloserine (100 mg; DCS) vs. D-Cycloserine (100 mg) + Dextromethorphan (150 mg; DCS + DXM) vs. Placebo) across four sessions and were tasked with completing the Serial Reaction Time Task (SRTT) before (referred to as the ‘skill acquisition’ phase or “pre-iTBS”) and after (referred to as the ‘retention’ phase or “post-iTBS”) an intermittent theta-burst stimulation (iTBS) intervention, a form of transcranial magnetic stimulation. Results of these data via linear mixed effect modelling revealed a reduction in response times when in the DCS condition paired with the iTBS as compared to all other drug conditions. Oppositely, there was an increase in response times in the LZP condition. The process of cross-validation yielded a design-driven LMER that was attuned at the 'session'-level by random assignment of the drug condition over a number of trials (96 per session for each participant) and each block order of the SRTT. Below is an R-Script includes instructions for replication should any researcher wish to pursue such with this small-n design pilot study.

Files

Steps to reproduce

Please refer to the R-Script featured for the steps to reproduce the results. The R-Script, "Official_R_Script.r," features detailed instructions to reproduce the analysis steps in the manuscript mentioned above. These steps include the order of arrangement for the utilized R-Packages (including 'lme4', 'lmerTest,' 'cv,' 'cvms,' 'parameters,' 'sjstats,' and 'performance'), linear mixed effect models, cross-validation for model development and comparison, and cluster-robust variance estimation.

Institutions

University of Manitoba, Harvard Medical School, McLean Hospital

Categories

Transcranial Magnetic Stimulation, Randomized Controlled Trial, Controlled Clinical Trial, Cross-Validation, Endogeneity Problem, Mixed Effect Regression, Applied Machine Learning

Funders

  • Brain & Behavior Research Foundation Young Investigator Grant
    Grant ID: #31748
  • the McLean Hospital Center of Excellence in Depression and Anxiety Disorders
    Grant ID: R01 AA027760
  • Marlene Zuckerman Fund
    Grant ID: R01 AA027760
  • Cindy & Paul Gamble Fund
    Grant ID: R01 AA027760
  • Department of Defense Advanced Research Projects Agency
    Grant ID: HR00112320037
  • National Institute of General Medical Sciences
    Grant ID: P20GM130452

Licence