Abstract
Matrix-reasoning tasks are commonly used in education, hiring, and admissions, and have
significant implications for test-takers. Analyzing cognitive strategies provides valuable insights
into reasoning processes and task evaluation. Previous research has identified a positive relation
between constructive matching and reasoning ability, and a negative relation with response
elimination. However, several questions remain unanswered regarding the generalizability of
matrix-reasoning tasks and the interplay between cognitive strategies and reasoning ability based
on test characteristics. To address these questions, we employed a matrix-reasoning task that
has not yet been studied in the context of cognitive strategies. A total of 119 university students
(71 women, 76% psychology students) completed 12 items of the Free Response Matrices
(FRM) and 12 stem-equivalent items adapted to a single-choice (SC) response format while eyetracking data was collected. Results revealed limited generalizability in cognitive strategy and
performance across different matrix-reasoning tasks. Additionally, we investigated whether eye
tracking indicators of strategy could explain variance in the SC score, even when controlling for the
FR score. The latency until the first response inspection emerged as the only significant predictor
(β = -0.18, 95% CI [-0.35, -0.02], p = .028) of SC score. Data supports that response elimination may
introduce construct-irrelevant variance into the SC score. The discussion focuses on response
format and potential influences due to the test characteristics of the FRM. The implications for
test construction, considering the contributions of this study and existing findings, are proposed.
significant implications for test-takers. Analyzing cognitive strategies provides valuable insights
into reasoning processes and task evaluation. Previous research has identified a positive relation
between constructive matching and reasoning ability, and a negative relation with response
elimination. However, several questions remain unanswered regarding the generalizability of
matrix-reasoning tasks and the interplay between cognitive strategies and reasoning ability based
on test characteristics. To address these questions, we employed a matrix-reasoning task that
has not yet been studied in the context of cognitive strategies. A total of 119 university students
(71 women, 76% psychology students) completed 12 items of the Free Response Matrices
(FRM) and 12 stem-equivalent items adapted to a single-choice (SC) response format while eyetracking data was collected. Results revealed limited generalizability in cognitive strategy and
performance across different matrix-reasoning tasks. Additionally, we investigated whether eye
tracking indicators of strategy could explain variance in the SC score, even when controlling for the
FR score. The latency until the first response inspection emerged as the only significant predictor
(β = -0.18, 95% CI [-0.35, -0.02], p = .028) of SC score. Data supports that response elimination may
introduce construct-irrelevant variance into the SC score. The discussion focuses on response
format and potential influences due to the test characteristics of the FRM. The implications for
test construction, considering the contributions of this study and existing findings, are proposed.
| Original language | English |
|---|---|
| Publication status | Published - 21 Oct 2024 |
Fields of Science and Technology Classification 2012
- 501 Psychology