Adapting multiple-choice comprehension question formats in a test of second language listening comprehension

Stefan O'Grady*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


The current study explores the impact of varying multiple-choice question preview and presentation formats in a test of second language listening proficiency targeting different levels of text comprehension. In a between-participant design, participants completed a 30-item test of listening comprehension featuring implicit and explicit information comprehension questions under one of four multiple-choice question preview and presentation conditions. Interactions between preview, presentation and comprehension in the participants' test scores were analysed using many facet Rasch analysis. The results suggest that the measurement of participants' listening ability was directly influenced by the presentation of multiple-choice questions. Test scores were highest when participants were able to preview the multiple-choice question stems before the sound file and listened to the options after the text had completed. However, interactions between preview and presentation conditions and comprehension level were only statistically significant in an analysis of the low scoring students' item responses, which were more frequently correct when preview of item stems was available for questions targeting comprehension of implicit information. The research underscores the importance of accounting for test design when making inferences about language learners' listening ability and will be of interest to teachers, practitioners and researchers developing listening assessment tasks.

Original languageEnglish
Article number1362168820985367
Number of pages25
JournalLanguage Teaching Research
Early online date23 Jan 2021
Publication statusE-pub ahead of print - 23 Jan 2021


  • English as a second language
  • Listening comprehension
  • Multiple-choice questions
  • Rasch measurement


Dive into the research topics of 'Adapting multiple-choice comprehension question formats in a test of second language listening comprehension'. Together they form a unique fingerprint.

Cite this