Publication:
Enabling Automatic Compiler-Driven Vectorization of Transformers

Thumbnail Image
Files
cgo26main-p71-p-a00f13bfad-168217-preprint.pdf(662.1 KB)
Date
2026-02-23
relationships.isAuthorOfPublication
Profile Picture
Person
Profile Picture
Person
Profile Picture
Person
relationships.isSecondaryAuthorOf
relationships.isDirectorOf
Authors
Alladi, Shreya ; Ros Bardisa, Alberto ; Jimborean, Alexandra
item.page.secondaryauthor
item.page.director
Publisher
IEEE
publication.page.editor
publication.page.department
Ingeniería y Tecnología de Computadores
DOI
10.1109/CGO68049.2026.11395226
item.page.type
info:eu-repo/semantics/article
Description
Abstract
Compiling neural networks and Transformers for edge devices faces significant challenges due to resource constraints and the reliance on manually optimized operations for performance among others. These limitations hinder the scalability and portability of neural networks on resource-constrained platforms, such as edge devices utilizing the RISC-V ecosystem. Addressing these issues, this paper introduces innovative techniques to overcome the inefficiencies of current compilation methods and reduce dependence on manual optimizations. This work proposes a novel compilation flow, ONNXMLIR- LLVM (OML), which leverages MLIR and LLVM IR to enable automatic optimizations and generate stand-alone RISC-V binaries. Through comprehensive analysis, we identify key barriers preventing the auto-vectorizer from handling vectorization-friendly operators, particularly reduction operations and vectorization-unfriendly data layouts. We address these through a versatile MLIR reduction detection pass and a compiletime transpose pass, respectively. Our automatic transformations (OML-vect) unlock the capabilities of the MLIR affine super-vectorizer, reducing reliance on manual vectorization. Evaluations on both x86 and RISC-V across eight neural networks and Transformer models demonstrate that automatic vectorization via OML-vect achieves, on average, 5% and 59% on x86 and RISC-V, respectively, compared to baseline (manually vectorized libraries), offering an efficient and portable solution for edge device deployments.
publication.page.subject
Vectorization , Neural networks , Reduction detection , Data layout optimizations , Neural networks
Citation
2026 IEEE/ACM International Symposium on Code Generation and Optimization (CGO) pp.: 319-333
URI
http://hdl.handle.net/10201/220721
item.page.embargo
Collections
Artículos
Ir a Estadísticas