About this project
Recent years have seen remarkable progress in the fluency of chatbots (e.g., Chat-GPT), spurred on by the development of large language models. These chatbots often reply in human-like ways to even the most outlandish queries. This apparent success has led to vigorous debate about whether chatbots truly understand language, or whether they merely go through the motions.
In order to adequately answer this question, we will develop a tool that measures language understanding by, e.g., measuring to what extent intended meanings are grasped that go beyond literal meaning. We will then compare chatbot performance to human performance in order to answer the question of to what extent chatbots truly understand language.
