PhD defence: Interplay between pronunciation, human, computer and foreign language

When people speak, they often adapt the way they do so to their interlocutor, whether human or computer. Linguist Aurora Troncoso Ruiz studied why we do this: do we simply repeat what we have just heard or do we adapt  to ensure that the interlocutor understands us? Troncoso Ruiz will defend her thesis on 13 June.

Photo by Christina Morillo

When we talk to someone, we adapt the way we do so to our listener. For example, a speaker from Limburg can adapt his pronunciation when talking to a non-Limburger. Someone who normally uses the word 'rich' may  switch to 'wealthy' when talking to someone who says 'wealthy'. People also adapt their speech when speaking in a language other than their mother tongue. In order to learn the pronunciation of a foreign language, it can help to repeat the pronunciation of the interlocutor in that language.

In linguistics, adapting pronunciation is called 'accommodation'. ‘Why people accommodate has been studied many times’, says linguist Aurora Troncoso Ruiz. ‘But mainly in contexts where the speakers use their mother tongue. In my research, I specifically focused on the linguistic factors that can cause Dutch speakers of English to adapt. I not only studied this in interaction with people, but also in interaction with computers.'

New methodology

To study accommodation of pronunciation, Troncoso Ruiz and her colleagues developed a new methodology for a laboratory setting. ‘Until now, the spontaneity and naturalness of the conversation was often sacrificed for this type of experiment, in order to have full control over the interaction in the lab’, explains Troncoso Ruiz. For example, the voice of the interlocutor is often recorded beforehand and presented to the participants via a computer. So in those experiments, the subjects are not communicating with a person, but with a computer.'

Troncoso Ruiz's method creates a more natural setting for a conversation between two people. The test subject sits in a room with another person, an accomplice who is part of the experiment. Both persons are wearing headphones with a microphone and they are playing a game together. The test subject thinks she is listening live to the accomplice's voice through the headphones, but in reality she is listening to pre-recorded speech fragments. When the accomplice wants to 'speak', she partly hides his face behind a monitor and presses a button to play the fragment. ‘About 80 per cent of the participants thought they were speaking directly to the interlocutor and did not notice the manipulation’, says Troncoso Ruiz. 'This way we can have more natural conversations and at the same time control the speech input the participants receive, so I can better control whether participants were exposed to a specific sound of English, and how often.'

Why do people accommodate?

Troncoso Ruiz used the new method for a number of experiments. In one of the experiments, Dutch students of English participated in an interaction with a human. For native speakers of Dutch, some English vowels are difficult to pronounce because they do not exist in Dutch, such as the English vowels in bad and bed (/bæd-bɛd/). During the experiment, some participants heard their conversation partner pronounce that specific English contrast, while others did not. In addition, some participants received feedback from their interlocutor that the specific vowels were unclear. Troncoso Ruiz: 'Subsequently, 107 native English speakers evaluated the pronunciation and with the help of Speech Alignment technology I was able to evaluate the subjects' pronunciation before and after the interaction. We discovered with this experiment that accommodation can be the result of repeating what you have heard, but also of changing the way you speak when people misunderstand you.'

Computer versus human as listener

Another angle in the research was interaction with computers. In an experiment, Troncoso Ruiz investigated whether people accommodated differently when speaking to a human than when speaking to a computer (in the style of Siri or Alexa). 'To do this, I created two identical setups: one where non-native speakers were interacting with a human interlocutor and one where they were interacting with a computer,' Troncoso Ruiz explains. 'Using my method, I was able to keep the speech of the accomplice or computer constant. Interestingly, speakers appeared to accommodate more to humans than to computers as a result of repetition, but more to computers than to humans when communicative success was compromised.'

Accommodation thus appears to be both an automatic phenomenon that facilitates the speaker's role and a process that facilitates the listener's task. Moreover, the results of these studies show that the degree of accommodation strongly depends on the type of interlocutor, namely whether it is a human or a computer.

Aurora Troncoso Ruiz will defend her thesis – titled Non-native phonetic accommodation in interactions with humans and with computers - on 13 June from 10:30.