Colloquium on Multiple-Document Reading in the Digital Age
Children and adolescents today grow up in an information-rich world where reading increasingly involves navigating multiple sources. Finding relevant information, comparing different documents, and evaluating their content are essential skills — but they are also challenging ones. The results of the Programme for International Student Assessment (PISA) show a continued decline in the reading proficiency of 15-year-olds, including in the Netherlands, raising growing concerns about students’ ability to understand and evaluate information effectively. See the Dutch PISA 2022 results.
This colloquium focuses on reading across multiple sources and the cognitive demands involved. PISA assesses both single- and multiple-document reading and includes extensive questionnaire and process data from students and schools. These publicly available data provide a rich international resource for studying reading comprehension and the factors that contribute to successful reading.
In this talk, the speaker and co-authors present a methodological approach for analysing large-scale assessment data, combining generalised linear mixed-effects models with meta-analytic techniques to test theory-driven hypotheses about multiple-source reading.
Using PISA 2018 data from 44 countries (N = 341,591), the researchers examined whether reading across multiple sources is more cognitively demanding than reading a single source, as reflected in lower comprehension accuracy. They also investigated whether engagement, operationalised as time on task, plays a particularly important role when students work with multiple documents.
The analyses examined the effects of source type (single versus multiple), time on task, and their interaction on reading comprehension, while accounting for relevant student- and item-level characteristics.
The findings support the theoretical predictions: comprehension was lower when students had to integrate information across multiple sources, and the positive relationship between time on task and comprehension was stronger for multiple-source reading.
The talk concludes by discussing the methodological challenges of analysing process data from large-scale international assessments, as well as the use of random-effects meta-analysis to synthesise country-specific findings.
The colloquium offers both substantive insights into how students read across multiple sources and a closer look at the methodological opportunities and challenges of working with large-scale international assessment data.
A colloquium for anyone interested in reading education, large-scale research, and the statistical challenges of international comparative data.
