Cross-Layered Sentiment Analysis for Identifying Learner Intent in AI Chatbots

Abstract

The rapid growth of artificial intelligence (AI) and machine learning has led to the development and widespread adoption of chatbots in different applications, including education. Chatbots have the potential to facilitate personalised learning and enhance knowledge acquisition. At the same time, these AI tools have demonstrated adverse effects on the learning process and academic integrity. Hence, the primary objective of this work is to develop a novel approach to determine if a user interacting with the chatbot exhibits genuine learning intent, which is under-explored in the literature. The outcome is SentimentGPT, a sentiment analysis framework that leverages natural language processing transformers fine-tuned with emotion recognition to assess the actual learning motivation during user interactions. By discerning authentic learning mindsets from superficial engagements, SentimentGPT enables AI chatbots to deliver tailored responses without compromising educational ethics. Experimental evaluations demonstrate that our approach achieves 77.94% accuracy and an F1 score of 0.91 in identifying a learning mindset, underscoring its potential to enhance user learning experiences within accredited educational settings.