Authors:

Isabella Summe. Ziwei Zhang

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

You might remember feeling surprised by a plot twist in a story. Surprise is a cognitive process that occurs when reality deviates from our expectations, and can arise in a variety of contexts — encountering an unexpected conversation, watching an unexpected play in a sports game, or reaching an unexpected outcome in a board game. Previous research identified a brain-based model that predicts surprise in a learning task and, importantly, generalizes to predict surprise while subjects watched a basketball game [1]. Here, | extend this framework to language and ask a novel question: what are the behavioral, neural, and computational mechanisms of linguistic surprise? Participants (n=90) listened to short narratives and provided continuous self-ratings of surprise, while we collected functional near-infrared spectroscopy (fNIRS) data. We quantified large language model (LLM) measures of narrative surprise by prompting an LLM to generate text predictions as additional story context is revealed. We computed two metrics: (1) surprisal, defined as how unlikely the model judges the upcoming word to be given the preceding context, and (2) representational dissimilarity, defined as how different the model's internal semantic representations are between the actual text and its own predicted text as the story unfolds. We observe stronger alignment between representational dissimilarity and human surprise ratings than surprisal, suggesting that narrative surprise may be better captured by changes in internal semantic representations rather than word-level improbability. Furthermore, fNIRS analyses reveal brain regions associated with human-rated and LLM-derived surprise dynamics. Together, these findings shed light on the neural dynamics underlying linguistic surprise and position LLMs as a useful computational framework for studying narrative expectations and comprehension.

Source:

University of Chicago

Topics:

No topics listed

Co-authors:

Isabella Summe. Ziwei Zhang