Intelligent machine translation: from text to sign language and vice versa
Sign language project EASIER has started. Goal: to enable barrier-free communication between deaf and hearing citizens in Europe using language technology. Onno Crasborn is involved in the project on behalf of Radboud University.
With some creativity, the name of the project EASIER can be taken from the subtitle of the project: intElligent Automatic SIgn languagE tRanslation. The project was awarded 4 million euros at the end of last year by Horizon 2020, the research and innovation program of the European Union.
EASIER has now officially started. For three years, an interdisciplinary research consortium will work on the automatic translation from (written) language to sign language and vice versa. This technology can be used, for example, to translate written web texts into sign language. The consortium unites the European Union for the Deaf (EUD) with research groups in sign language technology and experts in technology and humanities. Deaf employees are part of the project throughout the entire research cycle.
Widely functioning systems of machine translation to and from sign languages do not yet exist. There are systems that can be used in very limited areas, such as weather reports, but technology that is able to deal with a wide range of languages and communication scenarios is still a thing of the future. In addition, the current avatars - virtual people - are not really lifelike and, moreover, are unable to communicate the meaningful non-verbal parts of sign language. Crasborn: ‘In the deaf communities, everyone remembers with horror the first sign language avatars who moved their hands with a stiff, straight face. It's better now, but they still don't have many meaningful head movements and facial expressions.’
The plans are ambitious: EASIER wants to not only develop tools for the translation between written language and sign languages, but also guidelines and standards to enable future research into the collection and annotation of data for sign languages for which fewer sources are available. The project therefore starts with seven sign languages (British, French, German, German-Swiss, Dutch, Greek and Italian) for which a relatively large amount of data is available, and the six corresponding spoken languages (English, French, German, Dutch, Greek and Italian). Crasborn: ‘Based on the system we are building, we want to ensure that the technology can also be used for other (sign) languages, such as Hungarian and Finnish.’
Crasborn is pleased with the award of the grant and, above all, with what it stands for. ‘Sign languages are usually left out in language technology projects for a variety of reasons. We have had a lot of trouble getting a foothold in those ICT projects of the European Commission. Thanks to a lot of lobbying and perhaps also changing times - because Irma was on the news here, but in other European countries there were Irmas too - there is now finally a specific effort from the European Commission for the deaf communities again.’
Another sign language project - SignOn - recently received a grant from Horizon 2020. The Centre for Language and Speech Technology (CLST) is involved in that project.