Benchmarking Programming Models for GPU and AI Accelerator Kernels

Abstract

High-performance kernels for GPUs and AI accelerators can be implemented through low-level representations such as PTX and SASS, general-purpose programming such as CUDA and domain specific languages (DSLs) such as Triton, cuTile, TileLang and AWS Neuron Kernel Interface (NKI). These DSLs abstract many low-level hardware optimizations, allowing programmers to focus more on algorithmic optimizations. However, there is limited prior research that has compared these DSLs’ programming models, and how their performance may differ among kernel operations. We present a benchmark suite of 45 kernels implemented comparably across Triton, cuTile, TileLang, and AWS NKI. Triton primarily uses pointer-based indexing, while cuTile, TileLang, and NKI use tile-based programming models. The benchmark kernels span convolutions, matrix multiplications, elementwise operations, reductions, and data layout transformations. Each kernel includes a PyTorch baseline and supports sweeps across input sizes, data types, and default and autotune configurations. NVIDIA kernels are implemented for the B200 GPU which includes support for 5^th generation tensor-core operations such as tcgen05.mma, Tensor Memory for MMA accumulators, and Tensor Memory Accelerator (TMA). This benchmark helps compare how the underlying programming models of these DSLs affect kernel performance.