Cloudflare cuts its AI code reviewer's cost per run 68% with shared cache prefixes and model routing

llm-d is a router and orchestration layer that spreads requests across vLLM or SGLang replicas in a Kubernetes cluster, routing by cached prefix and load. The quickstart deploys the router in standalone mode plus two vLLM replicas of Qwen3-32B on NVIDIA GPUs.

1. Install kubectl and helm, then clone the repo and check out the release-1.0 branch.
2. Source guides/env.sh, set GUIDE_NAME and NAMESPACE, install the Gateway API Inference Extension CRDs and create the namespace.
3. Create the llm-d-hf-token secret with your Hugging Face token.
4. Helm-install the standalone router with the optimized-baseline values, then apply the vLLM model server kustomization.
5. Get the router ClusterIP and send a request to /v1/completions from a debug pod.

export branch="release-1.0"
git clone https://github.com/llm-d/llm-d.git && cd llm-d && git checkout ${branch}
export REPO_ROOT=$(realpath $(git rev-parse --show-toplevel))
source ${REPO_ROOT}/guides/env.sh
export GUIDE_NAME="quickstart"
export NAMESPACE=llm-d-quickstart
helm install ${GUIDE_NAME} \
    ${ROUTER_STANDALONE_CHART} \
    -f guides/recipes/router/base.values.yaml \
    -f guides/optimized-baseline/router/optimized-baseline.values.yaml \
    -n ${NAMESPACE} --version ${ROUTER_CHART_VERSION}
kubectl apply -n ${NAMESPACE} -k guides/optimized-baseline/modelserver/gpu/vllm/base/