Schedule of Events | Symposia

Sketchpad Series

From a Large Language Model to Three-Dimensional Sentiment

Poster Session E - Monday, March 31, 2025, 2:30 – 4:30 pm EDT, Back Bay Ballroom/Republic Ballroom

Patrick Clarke¹, Carly Leininger¹, Cristiana Principato¹, Patrick Staples¹, Guy Goodwin¹, Gregory Ryslik¹, Robert Dougherty¹; ¹Compass Pathways

Understanding sentiment in language with subtlety and precision offers insight into the psychological state of a speaker. Quantitative sentiment applied to therapy transcripts has already proved itself valuable in forecasting outcomes. We present a novel automated model of sentiment analysis that assigns three key emotional dimensions—valence, arousal, and confidence (VAC)—to any arbitrary text. Grounded in the three-dimensional framework of emotion by Mehrabian and Russell, this model offers a nuanced and versatile tool for measuring sentiment in text. Our model utilizes a convex combination of points in a three-dimensional emotion cube, with weights derived from the publicly available BART large language model (LLM), fine-tuned on the Multi-Genre Natural Language Inference (MNLI) dataset. The model demonstrates strong correlations with human ratings for individual words and arguably surpasses human raters in the dimension of confidence. Leveraging the capabilities of LLMs, our model can process text of any size, handle idiomatic expressions, adapt to evolving language trends, and deliver robust sentiment analysis for sentence-length inputs. To highlight its real-world relevance, we apply the model to assess emotional content in sentences spoken during psychological therapy sessions. Additionally, we introduce a new, cleaned version of the EmoBank dataset on which we evaluate the model, thus providing a performance measure on a now high-quality dataset whose text samples resemble the intended real-world application of the model.

Topic Area: LANGUAGE: Other