GhostWriter: Exploiting GPU-Cache Contention to Steal and Steer Multi-tenant Large-Language-Model Inference
摘要
Cloud providers routinely batch unrelated user prompts on shared Graphics Processing Units (GPUs) to amortize the escalating operational costs of large-language-model (LLM) inference. This paper demonstrates that such efficiency-driven optimizations introduce a potent microarchitectural side channel. We present GhostWriter, the first attack framework that (i) extracts private prompt tokens from co-located victims and (ii) covertly biases their generated text, all without requiring code execution privileges within the victim’s process or access to the LLM’s internal parameters. The attack exploits a subtle phenomenon: the residency of key–value (KV) cache entries in the GPU’s L2 cache persists across batched tenants. This persistence manifests as minute, per-token latency skews—often only a few milliseconds—which, as we show, are statistically amplifiable. We formalize the threat model under realistic cloud deployment conditions and develop a robust statistical detector based on a frequentist z-test, which recovers target tokens with up to \(96.2\) % accuracy on a GPT-Neo-125M model. Furthermore, we introduce an adversarial cache pre-loading strategy capable of increasing victim perplexity by up to \(11\) % and flipping sentiment in \(42\) % of news-style generations. Our end-to-end evaluation spans contemporary NVIDIA T4, A100, and consumer-grade RTX 4090 GPUs, popular inference stacks (including Triton Inference Server, vLLM, and Hugging Face Text Generation Inference), and realistic cloud batch sizes, confirming cross-tenant leakage under default configurations.