Authors:

Michael Cheng

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

The exploitation of software vulnerabilities can lead to severe consequences, including data breaches, financial loss, and system failure. As such, it is essential to identify and flag these vulnerabilities during the development cycle. Methods employing large language models (LLMs), which have demonstrated promise in many text- related tasks, have recently gained popularity in the context of vulnerability detection. However, these models often struggle to distinguish between a vulnerable function and a syntactically similar benign function. t-SNE visualization of the model's internal representation of the data reveals a clear inability to disentangle such functions. This research aims to improve upon existing techniques by training LLMs with a contrastive learning approach. Using DeepSeek-Coder as our base LLM, we incorporate triplet loss to encourage the model to cluster functions from the same Common Weakness Enumeration (CWE) class more closely in the embedding space. We train our model on real-world software vulnerability data and evaluate its performance against state- of-the-art approaches. Future work includes gathering additional high-quality vulnerability data and exploring alternative architectures and training methodologies to further enhance performance. Keywords: Machine Learning; Large Language Models; Software Security

Source:

Purdue University / 2024

Topics:

No topics listed

Co-authors:

Michael Cheng