Fiddler
Fiddler is a fast inference system for LLMs based on Mixture-of-Experts (MoE) architecture at local devices.
- Preprint: Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models
- GitHub: Fiddler
People
Baris Kasikci
Associate Professor