Live Session
Wednesday Posters
Research
Evaluation and simplification of text difficulty using LLMs in the context of recommending texts in French to facilitate language learning
Henri Jamet (Faculty of Business and Economics, University of Lausanne), Maxime Manderlier (Faculty of Engineering, University of Mons (UMONS)), Yash Raj Shrestha (Faculty of Business and Economics, University of Lausanne) and Michalis Vlachos (Faculty of Business and Economics, University of Lausanne)
Abstract
We develop a recommendation system for foreign language learning. This recommends text or video content. It ranks digital content considering both the content’s difficulty and how the topic aligns to the learners’ interests. To achieve this, we automatically apply the following operations to any text: a. Classify its subject. b. Evaluate its linguistic difficulty. c. Potentially simplify its language level, while preserving its semantic content for adaptation to the reader’s language level. Once these three operations have produced a set of texts adapted to the reader’s interests and level, they are ranked by relevance using a recommendation system based on the reading and satisfaction of other users. In this paper, we focus on using Large Language Models (LLMs) to automatically perform these tasks on any set of texts. We present an approach for training and evaluation and compare both zero-shot and fine-tuned performance of state-of-the-art models. Our findings indicate a marked improvement in the prediction of French content difficulty (improvement range of 18-56%), a 27% enhancement in topic prediction accuracy with fine-tuned models compared to zero-shot models, and up to an 18% increase in NDCG in recommendation performance.