Semantic Affinity Engine

Overview

Research repository for Semantic Affinity Engine (SAE), a semantic-aware batching approach for optimizing memory bandwidth in Large Language Model inference. Zenodo publication and supporting materials. Open-source project by Emmanuel Sánchez Pache, published on GitHub.

Highlights

• Open source — view the code and contribute on GitHub

Built with

• Code