Authors:

Hyungjun Doh

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

Visual understanding and action prediction in computer vision present fundamental challenges. Recent advancements in diffusion models have achieved state-of-the-art synthesis results in image data. Conversely, despite the emergence of transformers as the dominant mechanism in large language models, they have not been used effectively generating an animation. Animation from a single image remains a complex task. This research addresses this challenge by proposing a diffusion model that generates a sequence of frames from a single image. To enhance the smoothness and accuracy of the animation generated by our approach, we leverage the NW-UCLA dataset. This dataset encompasses 10 action categories and includes RGB, depth, and human skeleton data. Keyframe information is extracted from each action category, consisting of an image frame, a text description, and human skeleton data. The diffusion model is trained using sequences of frames as ground truth, with the keyframe information serving as conditioning. Perceptual metrics, specifically the Structural Similarity Index (SSIM), are employed to evaluate the performance of our implementation. The results demonstrate that our project achieves realistic animations that faithfully depict the text descriptions based on the provided images. Furthermore, our research holds potential beyond applications in animation, encompassing diverse domains such as human-computer interaction and virtual reality.

Source:

Purdue University / 2023

Topics:

No topics listed

Co-authors:

Hyungjun Doh