Item Response Theory analysis of the Cognitive Reflection Test: Testing the psychometric properties of the original scale and a newly developed 8-item version.

The Cognitive Reflection Test (CRT) is a short measure of a person’s ability to resist intuitive response tendencies, and to produce a normative response which is based on effortful reasoning. The CRT correlates strongly with important real-life outcomes, such as time preferences, risk-taking, and rational thinking. Although the CRT is a very popular measure, there is virtually no available data about its psychometric properties. The present study aimed at investigating the psychometric properties of the CRT, and to verify the suitability of a longer version of the test, which was obtained by adding five new items to the three original ones. We applied Item Response Theory analyses. The two-parameter logistic model was used in order to estimate item parameters (difficulty and discrimination), and the Test Information Function was computed to assess the measurement precision of both the original and the longer versions of the test. The results confirmed the suitability of the original tems for measuring the cognitive reflection ability trait. Furthermore, the results demonstrated that the longer version of the scale measures with high precision a wider range of the cognitive reflection latent trait.