Nick
Cheng
Multimodal Data Fusion Models Pretrained With VICReg
Abstract profile. Full document pending author claim.
Authors:
Nick Cheng
Date Created:
Not specified
Course Title:
Professor:
Not specified
About Paper:
Prediction models can be applied to hospital ICUs in order to improve patient care, through predicting patient behavior through the duration of their stay. The current field for mortality and length of stay predictions in the ICU consists of mainly single modal models, such as Shukla & Marlin's Interpolation Network, or Futoma et al.s' Multitask Gaussian Network. However, they are incapable of leveraging inter-modal patterns where each mode is strongest, which should allow for improved model performance when compared to single modal models. This is especially applicable in a hospital setting, as different modes of time series data are gathered when patients are admitted, such as clinical notes and machine output. Multimodal fusion models for this context have been proposed, and offer a notable performance improvement when compared to their single modal cousins. I believe that the performance of these multimodal models can be further improved through a pretraining step that leverages the large amount of unlabeled data that the hospital accumulates daily. Supervised models can only use a small amount of hospital data that includes the mortality or length of stay labels, while an unsupervised step allows the model to process unlabelled data. The unsupervised step is also expected to increase model performance when transferred to hospitals with different operating conditions or little labelled data when compared to standard supervised multimodal models. My results show that VICReg failed to create any noticeable performance benefit when compared to baseline multimodal models. Despite this outcome, I still believe that VICReg can be used to boost multimodal model performance, and I will discuss potential steps that could create a performance boost. Poster #19 The Impact of Novelty on the Context-Dependence of Avoidance Denise Carriaga
Source:
Texas A&M University / 2023
Topics:
No topics listed
Co-authors:
Nick Cheng