vLLM与LMCache实现重复提示2.8倍加速，无需GPU

vLLM and LMCache delivered a 2.8x speedup for repeated prompts without any GPU.

Their GitHub has 10K stars.

LLMs repeatedly calculate KV cache tensors for prompt tokens before generating each new token.

Shared system prompts and documents therefore force identical work across many requests.

LMCache stores those tensors in memory as L1 or external systems as L2.

LMCache is vendor-neutral. i.e. it can be used as a KV cache layer for a range of mainstream open-source serving engines, inference frameworks, hardware vendors, storage systems, and infrastructure providers.