Authors:

Leonardo Marciaga

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

Longitudinal surveys are a crucial component of behavioral research. Such surveys, however, struggle to maintain analytical continuity when survey questions must change over time. Missing responses in longitudinal surveys lead to data sparsity issues, further restricting applicable modeling methods. Using a 15-wave vaccination survey as a testbed, we address these issues through a framework involving deep learning, LLM-derived semantic question embeddings, and wave-local cluster analysis. We leverage LLM-generated semantic embeddings of survey questions to encode question meaning, enabling a Deep & Cross Network used for imputation to jointly model responses across item semantics, individual characteristics, and temporal dynamics. This structure directly addresses survey evolution by operating in learned semantic space. To overcome data scarcity, we use cluster-informed synthetic data generation via hierarchical prompting that produces synthetic responses preserving distributional properties and empirical cluster structure. Our approach achieves a strong improvement in semantic gap tasks and 80-90% synthetic data fidelity, providing practical solutions for evolving longitudinal studies.

Source:

Illinois Institute of Technology

Topics:

No topics listed

Co-authors:

Leonardo Marciaga