Dynamic Signs and Signals (Dynamos)

The Dynamos research program is about how communication emerges from (multiple) time-varying bodily systems that coordinate in productive ways so as to allow for novel, efficient, or more stable communicative behavior to emerge. An overarching goal in this research is identifying biological or and basic cognitive constituents and constraints on multimodal communication, in the hope to come to a more continuous understanding of communicative practices in humans. Continuous, in the sense of doing justice to the time-varying nature of communication. But, also in the sense of grounding human communicative practices in a wider basis of communicative practices shared with other animals.

Dynamos is aimed to develop quantitative approaches of bodily time-varying signs and signals (e.g., acoustic analyses, motion tracking methodology, as captured by time series analysis) which can be further related to semantic and pragmatic aspects which imbue signals with communicative potential. Dynamos combines a wide range of theoretical (embodied cognitive science, dynamical systems, ecological psychology, evolutionary biology, behavioral biologie) and methodological (e.g., time series analysis, computer vision, experimental psychology, data science) interests in its research outlined below.

Manual gestures

A human movement approach to signifying gestures

Manual gestures are often defined as meaningful movements –movement forms that contain a message. The theoretical conundrum is how a movement can contain such content: What allows movements to become meaningful? In this line of research, a bottom-up approach is developed, where it is investigated whether gestures are meaningful in virtue of relating consistently with other gestures over time. Using tools from data science to understand the constellated patterning of gestural movements, this line of research aims to unravel the process of signification of movements, without going beyond movement itself. That is, by only analyzing the kinematic trajectory of gesture, can we gain insights into how gestures come to function communicatively?

Communicative rhythms

Communicative rhythms: Timing and multimodal/interpersonal coordination

Adults speak and move communicatively in such an unreflective way, that is easy to forget that composing a multimodal utterance requires exquisite timing. The same is true for many instances of animal communication. Firstly, it requires timing between bodily segments that have their own intrinsic rhythms, such as manual movements versus much more mobile speech/communication effectors. But it also requires timing between bodies, such as in conversations (turn-taking), music-making and dance. This line of research is focusing specifically on the temporal aspects of communication. For example, looking at timing between gesture and speech, the rhythms are present in the kinematics of gesture, as well as the timing of turn taking between persons in conversation. Again our goal is to search for stable communicative behavior through basic principles of coordination, such as by synchronizing to stabilized communication.

Gesture-speech biomechanics

Gesture-speech biomechanics

Gestures are often understood as visual communication. Yet we should not forget that moving our hands is a biomechanical process that interacts with the whole body, affecting processes that are mechanically important for producing speech. Specifically, the biomechanical relations between manual gestures and the respiratory-vocal aspects of speech are investigated.  We have found that interactions arise when manual movements accelerate or decelerate thereby producing physical impulses on the body that recruit counter-stabilizing muscle groups that also involve respiratory muscles. Wim Pouw is currently working under a VENI grant that is dedicated to further investigating this phenomenon.

Multimodal signal processing

Multimodal signal processing

Studying multimodal communication comes with unique challenges for quantitative analysis. Gestures are postures in movement through 3D space, which is wholly different in nature from the speech signal. In this line of research, we seek to innovate methods to capture the complexity of gesture as well as speech, and to utilize state-of-the art developments in computer science for analyzing body movements. Specifically, the aim is to develop methods that are reproducible and automatable and that are nevertheless tailored to be combined with more qualitative methods of inquiry. We think we should strive for a symbiosis between quantitative and qualitative methods if we want to make progress in the study of multimodal communication.

Research group information

Click on one of the links below for more information about this research group or contact one of the members of this group.

Contact information

Postal address
Postbus 9104
6500HE NIJMEGEN
Contact person
Dr W.T.J.L. Pouw (Wim)