Serving 70B-scale LLMs efficiently on low-resource edge devices [pdf]

news.ycombinator.com

saved by: 0 people
related read on: Serving 70B-scale LLMs efficiently on low-resource edge d... discussion