Automatic speech recognition technology is ideally suited for educational applications because it can be directly applied to the development of two fundamental skills in children's education: speaking and reading. Game company 8D Games and the Centre for Language and Speech Technology (CLST) have therefore collaborated in recent years to develop a digital 'language buddy'. “Interviews showed that children regret that reading is often done alone and that they have no influence on the course of the story,” says Giel Hekkert, from 8D Games closely involved in the project. “That touches on an important principle from the gaming industry: people are generally more motivated to do a task when they experience autonomy.” This is how the idea was born to have children take turns speaking sentences on the smartphone and thus have them create a story together. Each spoken message is converted live to text. In the process, of course, you have to read your classmates' contributions carefully to add a new twist to the story yourself. Players are instructed to incorporate certain words or phrases in their contribution and can then vote for the best fragment.
Playfully practicing speaking and reading
The Centre for Language and Speech Technology at Radboud University Nijmegen and the game company 8D Games have jointly investigated the possibilities of speech technology embedded in games that promote reading and speaking skills. During the project, the game concept and functional prototype DigiJuf was developed and tested at various stages on primary school-aged children.
Speech technology from CLST
Researchers at Radboud University Nijmegen's Centre for Language and Speech Technology (CLST) developed the speech technology for the game. “We are currently doing several projects on reading and spelling and each application requires its own technology,” explains Helmer Strik of CLST. “In standard speech recognition, speech is converted to text. In the case of DigiJuf, the speech recogniser must additionally be able to assess whether the words are pronounced correctly by the child. Speech recognition of children is difficult anyway because of the wide variation and pitch of their voices. If you then limit yourself to reading predetermined sentences, it is easier, but at DigiJuf, the children were given a lot of freedom. In this project, we worked on optimising speech recognition for that.”
Testing in primary schools
An initial prototype was tested with over forty primary school pupils. Hekkert: “During these tests, we once again tested the creative concept and - in a technical sense - mapped out how CLST's speech technology can add as much value as possible in a game like this. The results were remarkable: children started speaking differently after only four sentences. Thus, they started articulating better to see their contribution to the story appear in the game in the best possible way. We also saw that this game form stimulates creativity and involves a lot of fun, making children forget that they are actually practising reading and speaking.”
The data from the test phases was analysed by Strik and his CLST colleagues, and the results were in turn used to improve the prototype. In addition, they investigated how children rated the pronunciation of their peers, compared to adults and the ASR system. Strik: “Previous studies show that getting feedback from peers has all kinds of benefits, such as increasing engagement and learning to think critically. Our research shows that children are as reliable assessors as adults. This means that primary school-aged children can provide feedback suitable to complement DigiJuf's feedback.”
Further development
Currently, 8D Games and CLST are exploring options for further development. Hekkert: “The potential of the game is clear and - besides a valuable learning experience for children - offers many starting points for researchers. CLST's speech technology is not dependent on tech giants, so we keep control over what data on reading and speaking behaviour is retrieved and what happens to it next. We are motivated to develop the prototype together into a beautiful final product, so that both education and the research field can benefit from it.”
DigiJuf is part of ST.CART, which is funded by the European Regional Development Fund and the European Union. CLST staff involved in this research have written several papers on the subject and made presentations at international conferences.
Contact information
- Contact
- Dr W.A.J. Strik (Helmer)
- Organizational unit
- Centre for Language Studies, Faculty of Arts
