Gestures aid in communication from a distance or with face masks

Wearing face masks will be obligatory in public transport from June 1. This has an effect our communication with each other, for both deaf and hearing people. Asli Özyürek's research investigates the relations between cognition, language, communication and development. Her research helps us understand how we can use gestures to get around these new barriers in spoken communication.

In a new interview, Özyürek addresses the consequences of wearing face masks on communication for the Deaf and Hearing (older & second language speakers) alike. You can read the interview in Dutch on Nemo Kennislink or Radboud Recharge.

Additionally, Özyürek wrote an article on the effect of Covid-19 on our daily communication. You can read this article in English below.

How is Covid19 Affecting Our Communication?:

As Talking Gets Harder, Gestures are Filling the Gap

Waving at your neighbor from afar. Discerning which colleague is talking in the Zoom call by what they’re doing with their hands. How to show your affection to your elderly loved ones through a window with a heart gesture. We need to figure out how to communicate effectively not only in hospitals but also in restaurants , trains etc; wearing mouth masks and from a distance in the “new normal”. Spontaneous gestures we make with our hands have long been thought to be ornaments of communication, perhaps even distractions. But everyday life in the age of the coronavirus has made hand gestures crucial. Fortunately, there’s a long tradition of research on gesture to show us how gestures can bridge unexpected communication gaps and can indeed help.

The coronavirus has peeled us away from many of our communicative norms. Physical proximity is dangerous. Even talking is likely to spread the virus (a recent study tested the size of fluid particles that came out of a mouth ranging in size from 20 to 500 microns during talking). We’ve responded with social distancing and working, communicating, and even socializing in virtual environments, all of which seem impoverished. Other aspects of the pandemic have forced us to adapt. Sometimes, when doctors are in full suits , the only way for them to communicate is with their hands. Relatives or clinicians trying to communicate through windows or other barriers can’t necessarily use sound, and masks, from medical-grade N95 masks to home-sewn wonders, are barriers that block sound but also keep us from seeing lip movements. Lip-covering masks challenge not only deaf as has been  mostly mentioned in media but hearing people alike. Even if we do device masks that show the lips, the blocking of sound will remain as a barrier.

Gestures critical to our survival

In the age of coronavirus, we’re gesturing with our hands more, and do so more grandly. Sometimes it’s for social niceties. With handshaking out of bounds, international leaders have found new ways of greeting and agreeing with one another We normally use shared eye gaze to regulate aspects of face-to-face conversation. In virtual environments, eye gaze isn’t useful, so people try to take turns by raising their hands. But gesturing can also be more critical. In emergency situations, where medical staff are wearing masks and a patient is receiving oxygen through a breathing aid device, gesturing can move meaning more quickly and help save lives.

Surprisingly, our ability to re-purpose our hands this way comes easily. That’s because transmitting messages through visible aspects of our body, including hand gestures, evolved along with spoken language. As far as we know, no spoken language lacks hand gestures. Before babies learn to speak, they use their bodies and hands to communicate. Furthermore, deaf people who have no access to spoken language can invent and learn a sign language in very similar patterns to spoken language. This adaptability of human language is coming to our aid when physical distance between speakers is critical to our survival.

Many scientific studies have demonstrated that hand gestures are important in enhancing speech communication, both for speakers and listeners. Speakers from all cultures use gestures—albeit to different degrees—to augment what they are saying. Some conventional gestures can be understood without speech, such as the thumbs-up sign for “OK.” Other gestures enhance speech, such as when doctors point to the left or right lung when asking “does it hurt here?” or offer an iconic drinking gesture with the words “Would you like to drink?” For a patient foggy from drugs or stress, the additional information in another channel isn’t redundant at all and even necessary.

Studies show that our brains benefit more from hand gestures than we do from information in another channel, like lip movements. When we asked people to understand words in noise just seeing the head of someone with access  to their lips versus seeing also the gesture, the comprehension of the word increased. This might be of great help when we can not  hear someone in virtual environments. This enhancement occurs because understanding the iconic hand gesture for “drink” involves the same brain areas (the Broca’s area of the brain) and processes as understanding the sound of the spoken word “drink.” The effect of these gestures is amplified whenever the speaker looks directly at or turns her body toward the person they are talking to or when she makes gestures bigger.

We also know that when people are asked to communicate only with their hands, speakers of different languages can invent, on the fly, ways to convey complex messages that are easily understood. When we asked Dutch and Mexican speakers in an experiment to convey the object words such as “pillow” with their hands, they did so in remarkably similar ways and most used mimicking of actions associated with objects- a common pattern we found. They had no experience with any sign language to draw on but still could rely on the icon-making potential of their hands (which may lie at the root of sign languages as well). Moreover, others found these made-up gestures easy to understand. Gesture invention happens in other settings, as with scuba divers or people on stock market trading floors. To communicate with coronavirus patients through glass walls, nurses in Wuhan hospitals are reported to have invented a few gestures such as " changing IV bottle", " giving injection" and others.

If they followed the gestural patterns that humans naturally use and understand, they likely worked well.

Though gesturing comes easily, we can’t expect this enhanced use to come naturally. Societies must be able to harness the full potential of our talking hands’ expressive range. It is possible that gesture as a communicative tool is more “naturally” available in gesture-rich cultures such as Mediterranean ones. But we can predict that other Western cultures will adapt too. We need research to develop communication protocols to help us adapt to mask-wearing, social distancing, and virtual environments and help us fully exploit the potential of visual communication. Gestures will undoubtedly play a role in those protocols, bringing us into the future by being the speaking and gesturing species that we’ve always been.

Want to know more? Please contact

• Prof. Asli Özyürek asli.ozyurek@mpi.nl. She is a member of the research group Multimodel Language and Cognition, at the Centre for Language Studies (CLS)
• Communication Faculty of Arts, communicatie@let.ru.nl