Donders Institute for Brain, Cognition and Behaviour
Zoek in de site...
Theme 1: Language and Communication

Multimodal Language and Cognition


This PI group tries to understand cognitive and social pressures underlying language use, emergence and language learning in its multimodal and face to face context – that is the context languages are naturally are learned by children, evolved historically and are used mosty frequently.

In doing so it particularly investigates how adults and child speakers use and perceive hand gestures ( e.g, saying “drinking” with a drink gesture ) in a coordinated fashion with spoken utterances and what this reveals about their linguistic , cognitive and social processes . For example we try to unravel how speakers use gestures to disambiguate degraded speech and recruit them in first and second language learning. We also conduct research on how speakers process and integrate semantic information from speech and gesture in the brain recruiting different aspects of the language network and beyond and how they design their message for their addressees by modulating their speech and gesture such as in teaching contexts. We are also interested in how general non linguistic cognitive abilities such as memory, imagery and visual attention are related to the use and perception of multimodal aspects of language use in face to face interactions by adults and children.

Asli_02Another focus of this PI group is on sign languages, the natural languages of deaf communities. Here we try to understand the role modality (visual vs. vocal) plays in shaping languages in general , cognitive and social biases that might play a role in the emergent patterns and structures in new sign languages ( in the wild or laboratory) and language acquisition patterns by deaf and hearing children and adults as well as in bimodal bilinguals ( hearing speakers who can sign and speak at the same time).

Our general mission therefore is to understand embodied and social foundations of language and cognition in multimodal and situated contexts in adults and children and provide new insights into understanding relations between language and other cognitive systems such as action-perception.

Please visit for further info.

Name: Asli Özyürek
Telephone: 024-3521304
Fax: 024-3616066
Visiting address: Donders Centre for Cognition
Montessorilaan 3
6525 HR Nijmegen
The Netherlands
Postal address: Donders Centre for Cognition
P.O. Box 9104
6500 HE Nijmegen
The Netherlands
Key publications
  • Ortega, G., Sumer, B., & Ozyurek, A. (2017). Type of iconicity matters in the vocabulary development of signing children. Developmental Psychology, 53(1), 89-99. doi:10.1037/dev0000161
  • Drijvers, L., & Ozyurek, A. (2017). Visual context enhanced: The joint contribution of iconic gestures and visible speech to degraded speech comprehension. Journal of Speech, Language, and Hearing Research., 60, 212-222. doi:10.1044/2016_JSLHR-H-16-0101
  • Ozyurek, A. (2014). Hearing and seeing meaning in speech and gesture: Insights from brain and behaviour. Philosophical Transactions of the Royal Society of London, Series B: Biological Sciences, 369(1651): 20130296. doi:10.1098/rstb.2013.0296.
  • Kelly, S. D., Ozyurek, A., & Maris, E. (2010). Two sides of the same coin: Speech and gesture mutually interact to enhance comprehension. Psychological Science, 21, 260-267. doi:10.1177/0956797609357327.
  • Ozyurek, A., Willems, R. M., Kita, S., & Hagoort, P. (2007). On-line integration of semantic information from speech and gesture: Insights from event-related brain potentials. Journal of Cognitive Neuroscience, 19(4), 605-616. doi:10.1162/jocn.2007.19.4.605
  • Senghas, A., Kita, S., & Ozyurek, A. (2004). Children creating core properties of language: Evidence from an emerging sign language in Nicaragua. Science, 305(5691), 1779-1782. doi:10.1126/science.1100199.

Back to:
Theme 1:
Language and Communication

Multimodal Language
and Cognition

Principal Investigator
Prof. Asli Özyürek

Group members

Wim Pouw