serving-llms-vllm
•published 2026/02/01
Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.
📦 Install Skill
Command line install
Install directly with the `skills` CLI.
npx skills add Orchestra-Research/AI-research-SKILLs/12-inference-serving/vllmFile Explorer
No file selected
Select a file to view its content