vllm

github.com

saved by: 0 people
related read on: Accelerating Gemma 4: faster inference with multi-token p... from the discussion