Authors:

Evelyn Colon

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

Transformer models, as the backbone of Large Language Models (LLMs), have complex and energy-intensive computing requirements. Optimizing the efficiency and latency of Transformer AI hardware is essential for reducing costs, improving sustainability, and democratizing access to powerful LLM technology. While previous research has yielded significant advances in computing- in-memory (CIM) technology to make running Transformer models more efficient with reduced data movement, these projects often lack a robust Instruction Set Architecture (ISA-the set of commands a processor can execute) and, critically, a compiler that allows programmers to execute high-level code (especially Python) on the CIM chips. This research aims to bridge the gap between high-level code and NanoX's innovative CIM chip by designing a compiler that translates high-level code into the chip's custom ISA. The compiler development process began from a hardware-independent perspective. The first task was to write an application that breaks down a given transformer model into matrix computations. Next, the compiler must select which of these computations the chip can execute. Finally, the compiler will be developed from a low-level, hardware-specific perspective, translating the identified computations into commands that the chip can execute directly. The compiler's functionality will be verified by executing high-level instructions on the chip and analyzing the system's performance through metrics such as computational accuracy and latency. Keywords: Transformer; Computing-in-Memory; Artificial Intelligence; Accelerator; Compiler

Source:

Purdue University / 2025

Topics:

No topics listed

Co-authors:

Evelyn Colon