Authors:

low) Author

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

A large language model (LLM), upon being prompted with a question, should respond with follow-up questions when the model has insufficient information to answer the user query. This paper investigates whether desirable behaviors like follow- up questions are subject to systematic biases. To this end, we analyze LLM responses from Cohere Command R+ and Llama 3.1-8B to twenty demographic-neutral queries about food, health, clothing, and more. Each prompt was prefixed with a gendered persona. These prompts were intentionally under- specified, requiring further clarification from the LLM for a full response to the request. We classified the responses into two categories: those that were direct replies to the query and those that sought further clarification. We generated 590 responses for each query and applied uniform manifold approximation and projection (UMAP) to reduce the dimensionality of the data, followed by clustering to investigate patterns. Preliminary results show the model is more likely to make assumptions about the user when the prompt contains a female persona and more likely to ask clarifying questions when the persona is male. Furthermore, responses to men included more epistemic modal verbs like "can" while responses to women included more deontic modal verbs like "should." If these preliminary results hold, these trends imply that female users receive lower quality, less personalized responses from LLMs than male users. Interpreting and Manipulating Language Model Embeddings via Concept Manifolds Carl Scandelius, Pranav Misra, Haim Sompolinsky Harvard College | Winthrop House | Mathematics | 2027 Representations in language models (LMs) remain largely uninterpretable. We propose a geometric framework for analysing LM representations using concept manifolds—point clouds formed by the final-token residual-stream embeddings of sentences representing a single dominant concept. These manifolds enable the definition of theoretically grounded geometric measures that directly relate to downstream task performance. This framework builds on successful applications to vision and vision-language models. Our project has three main stages: (1) characterise the internal structure and inter-concept alignments of the concept manifolds at each transformer layer; (2) analyse the interpretability of the basis vectors of these manifolds; (3) use this framework to guide causal intervention on LM embeddings to generate predictably altered output. Insights from this project hope to inform safety-minded efforts like discovering misaligned latent knowledge, as well as fine- tuning strategies or perturbation-based methods to mitigate against misaligned model behaviour. Harvard Summer Undergraduate Research Village Kempner Research in Artificial and Natural Intelligence for Undergraduates with Mentorship

Source:

Harvard / Mathematics / 2028

Topics:

No topics listed

Co-authors:

low) Author