Visual information helps (extra) when listening in a busy environment

Understanding someone in a busy environment becomes easier if the background noise is in a different language than the one you are conversing in. It also helps if you can see the lips of the person you are talking with. What happens when you combine these two factors? Linguist Susanne Brouwer and her colleagues at Penn State University found out.

If you are in conversation with someone, and around you other people are also talking to each other, it is sometimes difficult to understand what your conversational partner is saying. The other people's speech 'masks' your conversation partner's speech.

Research has shown that there are factors that can make understanding easier. For example, it helps if you also get visual information: if you can see the lips of the person you are conversing with moving, you are more likely to also understand what they are saying. This is called visual speech information (VSI). It is also easier to understand your partner's speech if the people around you speak in a different language than the one you are communicating in. This effect is called Linguistic Release from Masking (LRM) in linguistics.

Experiments

Susanne Brouwer, linguist at the Centre for Language Studies (CLS), and her colleagues investigated whether VSI and LRM work together when trying to understand a conversation partner. In their experiments, native English speakers performed an intelligibility task. They heard sentences in English and simultaneously heard Dutch or English spoken in the background. Sometimes the participants could only hear the speaker they had to understand, and sometimes they could also see the person while he/she was pronouncing the sentence.

Working together

The results showed that it was easier for English-speaking participants to understand the sentences if the people in the background spoke Dutch instead of English. This is consistent with previous research. Interestingly, it does not matter whether the participants can see their conversational partner or not: the effect persists. But: the effect of Linguistic Release from Masking was smaller in the situation where participants could see the speaker. According to Brouwer and her colleagues, this means that people use both VSI and LRM to help them understand better.

Want to know more? Read the article The Effect of Visual Speech Information on Linguistic Release from Masking or contact Susanne Brouwer at susanne.brouwer@ru.nl