Current neural language models are very good at producing grammatically correct sentences. But how good is their knowledge of syntax really? In this project, we create a large collection of Dutch sentence pairs, one of which is grammatically correct and the other minimally different but incorrect. The question is how well language models are able to choose the correct sentence.
We also collect judgements from people about these sentences, to investigate whether models have similar judgements. Finally, we measure eye movements of people reading these sentences, to investigate the extent to which language models can explain these eye movements.
