K-Search Adapts Decades of GPU Kernel Expertise for Apple Silicon Using MLX

Researchers have developed a novel approach, K-Search, to translate decades of GPU kernel optimization expertise from NVIDIA's CUDA ecosystem to Apple Silicon, leveraging Apple's MLX framework. This method aims to automatically adapt existing CUDA kernels into high-quality kernels for Apple's custom hardware, rather than requiring a complete rebuild. The K-Search framework, originally developed at UC Berkeley Sky Lab, uses AI to iteratively optimize GPU kernels. It employs a large language model to reason about optimizations, generate candidate kernels, and benchmark them on real hardware. This process is guided by a domain-specific document encoding hardware rules and optimization patterns to ensure generated code is valid and efficient. The system maintains a persistent reasoning state, called a world model, structured as a decision tree that explores and refines optimization plans. This approach has demonstrated near-expert level performance on Apple Silicon, achieving a 0.97x speedup compared to the native MLX Attention kernel and up to a 20x prefill speedup over the community mlx-lm implementation for the Mamba SSM kernel. The researchers highlight that while they focused on MLX for Apple Silicon, the underlying method is transferable to other hardware ecosystems where CUDA expertise is relevant. The adoption of MLX, especially for local AI inference on hundreds of millions of Apple devices, presents an opportunity to bridge the performance gap in critical kernels that are often underdeveloped or naive in newer hardware ecosystems.