Exllamav2: Inference library for running LLMs locally on consumer-class GPUs

news.ycombinator.com

saved by: 0 people
related read on: Exllamav2: Inference library for running LLMs locally on ... discussion