Dr J.A.M. Holler (Judith)

I investigate human multimodal communication. In particular, I am interested in how communicative intentions and the pragmatics of conversation influence the use of and interplay between bodily signals and speech, as well as how social signals and context may modulate the perception of bodily signals and their integration with speech. My projects use paradigms that embed multimodal language production and comprehension in situated contexts with a special focus on the interplay between different communicative modalities, including speech, gesture, facial signals and eye gaze. In addition to investigating how these modalities interact in the process of semantic comprehension, I explore the role of the visual articulators in pragmatic communication, including turn-taking and social action formation and recognition. I have recently been awarded an ERC Consolidator grant entitled "Communication in Action (CoAct): Towards a model of contextualised action and language processing" which investigates the role of visual communicative signals in situated, face-to-face language production and comprehension using a variety of novel techniques (including Virtual Reality and motion capture). Furthermore, I am interested in the cognitive representations underlying the production of gestures, the development of gestural communication (in childhood and ageing, as well as in evolutionary terms), and the use of gesture in the context of impaired (e.g. Parkinson’s) and verbally challenging (e.g. pain sensation) communication.