Thesis defense James Trujillo (Donders series 419)

13 February 2020

Promotors: prof. dr. H. Bekkering, prof. dr. A. Özyürek
Co-promotor: dr. I. Simanova

Movement Speaks for Itself: The Neural and Kinematic Dynamics of Communicative Action and Gesture

As social creatures, humans rely heavily on the ability to understand what others are doing and why. Similarly, we make ourselves understandable to others. This is the glue that allows our complex social structure to function. Besides conventionalized communicative behaviors such as speaking or giving a “thumbs up”, we also regularly act on objects, or simulate doing so using hand gestures, in order to demonstrate how to do something or to instruct someone to act. The way that we perform these actions and gestures changes depending on whether we are doing them for ourselves or as a demonstration, and furthermore depending on for whom we are demonstrating (e.g. a child or an adult). Specifically, different social contexts lead to changes in the kinematics (e.g. velocity, size, complexity) of our movements. If our actions and gestures are shaped by the context in which they are produced, this means that information about our intentions, both in terms of what we are trying to convey and why, is externalized in our behavior.

In this thesis I have demonstrated that the intention to communicate affects the kinematics of our actions and gestures, and that these kinematic differences both enhance the comprehensibility of what we are doing and act as a signal that what we are doing is intended for our addressee. Our addressee recognizes this intention because the exaggeration is unexpected based on previous experience. This leads them to infer that our intention was to use the action or gesture communicatively. Using brain imaging, I have shown that this process of intention inference is supported by a similar neural mechanism as is used to rationalize unusual or inefficient behavior observed in others. Extending this model of communicative kinematic exaggeration into noisy, co-speech gestures, I demonstrated a similar effect. Specifically, I found that increased noise leads to an increase in the visual information conveyed in the gesture.

This thesis shows that communication is not just what we say or do. It is also the way we move, providing a glimpse into our intentions and giving shape to the ideas we wish to communicate.