TY - JOUR
T1 - Tell Us What You Really Think
T2 - A Think Aloud Protocol Analysis of the Verbal Cognitive Reflection Test
AU - Byrd, Nick
AU - Joseph, Brianna
AU - Gongora, Gabriela
AU - Sirota, Miroslav
N1 - Publisher Copyright:
© 2023 by the authors.
PY - 2023/4
Y1 - 2023/4
N2 - The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of mathematical reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol in-person and online to test how this assumption is satisfied by the new, validated, less familiar, and non-mathematical verbal Cognitive Reflection Test (vCRT). Verbalized thoughts in both studies revealed that most (but not all) correct responses involved reflection and that most (but not all) lured responses lacked reflection. The think-aloud protocols seemed to reflect business-as-usual performance: thinking aloud did not disrupt test performance compared to a control group. These data suggest that the vCRT usually satisfies the standard interpretation of the reflection tests (albeit not without exceptions) and that the vCRT can be a good measure of the construct theorized by the two-factor explication of ‘reflection’ (as deliberate and conscious).
AB - The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of mathematical reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol in-person and online to test how this assumption is satisfied by the new, validated, less familiar, and non-mathematical verbal Cognitive Reflection Test (vCRT). Verbalized thoughts in both studies revealed that most (but not all) correct responses involved reflection and that most (but not all) lured responses lacked reflection. The think-aloud protocols seemed to reflect business-as-usual performance: thinking aloud did not disrupt test performance compared to a control group. These data suggest that the vCRT usually satisfies the standard interpretation of the reflection tests (albeit not without exceptions) and that the vCRT can be a good measure of the construct theorized by the two-factor explication of ‘reflection’ (as deliberate and conscious).
KW - cognitive reflection test
KW - heuristics and biases
KW - judgment- and decision-making
KW - psychometrics
KW - think-aloud protocol analysis
UR - http://www.scopus.com/inward/record.url?scp=85153790193&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85153790193&partnerID=8YFLogxK
U2 - 10.3390/jintelligence11040076
DO - 10.3390/jintelligence11040076
M3 - Article
AN - SCOPUS:85153790193
VL - 11
JO - Journal of Intelligence
JF - Journal of Intelligence
IS - 4
M1 - 76
ER -