{
 "slug": "vllm",
 "name": "vLLM + Open WebUI",
 "tagline": "High-throughput OpenAI-compatible LLM inference server with a browser chat UI",
 "category": "ai-llm",
 "use_cases": [
  "llm-inference",
  "openai-compatible-api",
  "chat-ui"
 ],
 "aliases": [
  "open webui"
 ],
 "status": "live",
 "upstream": {
  "vendor": "vLLM Team",
  "url": "https://docs.vllm.ai",
  "license": "Apache-2.0",
  "license_url": "https://github.com/vllm-project/vllm/blob/main/LICENSE"
 },
 "image": {
  "os": "Ubuntu 24.04 LTS",
  "cis_level": 1,
  "ports": [
   80,
   443
  ],
  "artifacts": [
   "/etc/lynxroute/sbom.json",
   "/etc/lynxroute/cis-report.html",
   "/usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md"
  ]
 },
 "clouds": {
  "aws": {
   "status": "live",
   "version": "0.32.0",
   "last_build": "2026-10-25",
   "entity_id": "prod-ub33kxqn3lfxi",
   "ami_alias": "/aws/service/marketplace/prod-ub33kxqn3lfxi/latest",
   "listing_url": "https://aws.amazon.com/marketplace/pp/prodview-wfiyb274g2jdc",
   "recommended_instance_type": "m6i.xlarge",
   "price_usd_per_hour": 0.05,
   "free_trial_days": 5,
   "unit": "instance-hour"
  },
  "azure": {
   "status": "live",
   "version": "0.30.0",
   "last_build": "2026-10-05",
   "offer_id": "vllm",
   "plan_id": "payg-gen2",
   "image_urn": "lynxroute:vllm:payg-gen2:latest",
   "listing_url": "https://azuremarketplace.microsoft.com/en-us/marketplace/apps/lynxroute.vllm?tab=Overview",
   "recommended_vm_size": "Standard_D4s_v3",
   "price_usd_per_vcpu_hour": 0.028,
   "unit": "vcpu-hour"
  },
  "alicloud": {
   "status": "not_offered"
  }
 },
 "urls": {
  "page": "https://lynxroute.com/products/vllm/",
  "json": "https://lynxroute.com/products/vllm/product.json"
 },
 "release_notes": [
  {
   "version": "0.32.0",
   "title": "vLLM 0.32.0 + Open WebUI 0.12.4",
   "notes": "* vLLM upgraded to 0.32.0 (from 0.30.0) - OpenAI-compatible LLM inference server. The default model, the CPU runtime and the API endpoints this image exposes are unchanged. Apache-2.0 unchanged\n* Security (vLLM): fixes for chat-template resource exhaustion (GHSA-4hhp-h66f-j5j7) and a tokenizer bounds error (GHSA-q97c-p368-75c4); untrusted per-request multimodal options are refused by default; cross-origin browser calls to the API are no longer allowed by default (scripts and SDKs are unaffected)\n* Security (Open WebUI): bundled Open WebUI updated to 0.12.4 (from 0.11.4), with the channel message-spoofing fix (GHSA-3vvh-g22g-jq6r) and further upstream security and access-control fixes. Adds optional two-factor sign-in. Open WebUI is source-available under the Open WebUI License, not MIT: its branding may not be altered or removed in a deployment exceeding fifty end users in any rolling thirty-day period\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base, picking up every operating-system security update published since the previous build\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-10-25"
   }
  },
  {
   "version": "0.30.0",
   "title": "vLLM 0.30.0 + Open WebUI 0.11.4",
   "notes": "* Security: bundled Open WebUI updated to 0.11.4, fixing published advisories including session-token theft by any website (GHSA-vpq8-f445-hcq7), token theft through DOCX previews (GHSA-f9xp-mfmq-x6cg) and citation links (GHSA-qpqv-xwg8-cqpj), and a terminal-proxy privilege escalation (GHSA-q46m-r89w-j74p)\n* vLLM remains at 0.30.0. Apache-2.0 unchanged; Open WebUI is source-available under the Open WebUI License, not MIT: its branding may not be altered or removed in a deployment exceeding fifty end users in any rolling thirty-day period\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base, picking up every operating-system security update published since the previous build\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-10-23",
    "azure": "2026-10-05"
   }
  },
  {
   "version": "0.30.0",
   "title": "vLLM 0.30.0 + Open WebUI 0.11.1",
   "notes": "* vLLM upgraded to 0.30.0 (from 0.29.0) - OpenAI-compatible LLM inference server. The API surface this image exposes is unchanged, as are the default model and the CPU runtime. Apache-2.0 unchanged\n* Security: validation-error responses are now size-bounded, closing a large response-amplification path, client-supplied sparse embeddings are bounded before expansion, and cache-salt values are validated so a single request cannot take down the engine\n* Upstream removals: GPTQ activation-ordering support and the settings deprecated in 0.29 are gone. The default model shipped here, Qwen2.5-0.5B-Instruct, is unaffected; any HuggingFace model can still be selected by editing /etc/vllm/server.env\n* Open WebUI stays at 0.11.1. It is source-available under the Open WebUI License, not MIT: its branding may not be altered or removed in a deployment exceeding fifty end users in any rolling thirty-day period\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the current CIS Level 1 hardened Ubuntu 24.04 LTS base for the latest Ubuntu security updates\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-10-23",
    "azure": "2026-10-05"
   }
  },
  {
   "version": "0.29.0",
   "title": "vLLM 0.29.0 + Open WebUI 0.11.1",
   "notes": "* vLLM upgraded to 0.29.0 (from 0.28.0) - OpenAI-compatible LLM inference server. The API surface this image exposes is unchanged, as are the default model and the CPU runtime. Apache-2.0 unchanged\n* Security: the API key and any HuggingFace token are now redacted from the server startup log, oversized media is rejected before it is fully downloaded, a cache-salt length limit closes a request path that could exhaust scheduler CPU, and prompt-length validation is enforced on decoders that previously skipped it\n* Ten long-unmaintained model architectures were removed upstream. The default model shipped here, Qwen2.5-0.5B-Instruct, and the current Qwen, Llama and Mistral families are unaffected; any HuggingFace model can still be selected by editing /etc/vllm/server.env\n* Open WebUI stays at 0.11.1, the release carrying the September 2026 sign-in, access-control and server-side-request fixes. It is source-available under the Open WebUI License, not MIT: its branding may not be altered or removed in a deployment exceeding fifty end users in any rolling thirty-day period\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the current CIS Level 1 hardened Ubuntu 24.04 LTS base for the latest Ubuntu security updates\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-09-11",
    "azure": "2026-09-14"
   }
  },
  {
   "version": "0.28.0",
   "title": "vLLM 0.28.0 + Open WebUI 0.11.1",
   "notes": "* vLLM upgraded to 0.28.0 (from 0.27.1) - OpenAI-compatible LLM inference server. The OpenAI API surface this image exposes is unchanged; upstream raises the default max_num_batched_tokens to 16384, moves bitsandbytes support to an out-of-tree plugin, and removes the deprecated calculate_kv_scales and override_attention_dtype options. Apache-2.0 unchanged\n* Open WebUI upgraded to 0.11.1 (from 0.9.6). Replies now stream in increments instead of resending the whole message on every update, a reply in progress survives a page refresh, folders can be shared with users or groups under a new administrator permission that is off by default, and chat lists, file access checks and tool listings no longer slow down as an instance grows\n* FIXED: the browser chat UI now works. The previous default model (facebook/opt-125m) is a base language model that defines no chat template, so the endpoint Open WebUI's chat box calls returned an error on every message while the direct API kept working. The default is now Qwen2.5-0.5B-Instruct (Apache-2.0, ~1GB, instruction-tuned), which answers in the chat UI out of the box. Any HuggingFace model can still be selected by editing /etc/vllm/server.env\n* Open WebUI is source-available under the Open WebUI License, not MIT. Its branding may not be altered or removed in a deployment exceeding fifty end users in any rolling thirty-day period. This term is unchanged in 0.11.1\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the current CIS Level 1 hardened Ubuntu 24.04 LTS base to pick up the latest Ubuntu security updates\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-08-28",
    "azure": "2026-08-29"
   }
  },
  {
   "version": "0.27.1",
   "title": "vLLM 0.27.1 + Open WebUI 0.9.6",
   "notes": "* vLLM upgraded to 0.27.1 (from 0.26.0) - OpenAI-compatible LLM inference server with a chat UI. Adds support for quantized DSpark Markov heads on top of the 0.27.0 feature release. The OpenAI API surface is unchanged. Open WebUI remains at 0.9.6.\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-08-18",
    "azure": "2026-08-19"
   }
  },
  {
   "version": "0.26.0",
   "title": "vLLM 0.26.0 + Open WebUI 0.9.6",
   "notes": "* vLLM upgraded to 0.26.0 (from 0.25.1) - OpenAI-compatible LLM inference server with a chat UI; upstream bugfixes, dependency updates, and security fixes. The OpenAI API surface is unchanged. Open WebUI remains at 0.9.6.\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base\n",
   "source": "backfill-git",
   "live": {
    "aws": "2026-07-25"
   }
  },
  {
   "version": "0.25.1",
   "title": "vLLM 0.25.1 + Open WebUI 0.9.6",
   "notes": "* vLLM upgraded to 0.25.1 (from 0.24.0) - OpenAI-compatible LLM inference server with a chat UI. Model Runner V2 is now the default engine; the OpenAI-compatible API surface is unchanged. Open WebUI remains at 0.9.6. Apache-2.0.\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-07-20",
    "azure": "2026-07-20"
   }
  },
  {
   "version": "0.24.0",
   "title": "vLLM 0.24.0 + Open WebUI 0.9.6",
   "notes": "* vLLM upgraded to 0.24.0 (from 0.23.0) - OpenAI-compatible LLM inference server with a chat UI. Additive API server changes (API-key auth, CORS, model metadata); no torch/CUDA bump (torch 2.11.0 unchanged). Open WebUI remains at 0.9.6.\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-07-11",
    "azure": "2026-07-13"
   }
  },
  {
   "version": "0.23.0",
   "title": "vLLM + Open WebUI 0.23.0",
   "notes": "* vLLM 0.23.0 + Open WebUI 0.9.6 - OpenAI-compatible LLM inference server with a chat UI (update from vLLM 0.22.1)\n* Certbot pre-installed - enable a trusted HTTPS certificate with one command: sudo certbot --nginx -d yourdomain.com\n* Rebuilt on the latest CIS Level 1 hardened Ubuntu 24.04 LTS base\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-06-27",
    "azure": "2026-06-21"
   }
  },
  {
   "version": "0.22.1",
   "title": "vLLM + Open WebUI 0.22.1 - June 2026",
   "notes": "* vLLM 0.22.x: performance improvements, Model Runner V2 advances, and expanded model/hardware support; deprecated internal APIs removed. Open WebUI updated to the latest release.\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-06-06",
    "azure": "2026-06-07"
   }
  },
  {
   "version": "0.21.0",
   "title": "vLLM + Open WebUI 0.21.0 - Initial release (May 2026)",
   "notes": "* vLLM 0.21.0 (CPU build from upstream wheel) + Open WebUI 0.9.5 on Ubuntu 24.04 LTS\n* CIS Level 1 hardening applied (ansible-lockdown/UBUNTU24-CIS)\n* CVE-scanned before every release\n* Random 32-byte API key generated at first boot - written to /root/vllm-credentials.txt and shared between vLLM and Open WebUI\n* vLLM API server bound to 127.0.0.1:8000 with --api-key Bearer auth on every /v1/* request\n* Open WebUI bound to 127.0.0.1:8080, pre-wired to local vLLM via OPENAI_API_BASE_URL\n* First registered user in Open WebUI becomes the workspace administrator\n* Default model facebook/opt-125m preloaded - chat and API work immediately, no HuggingFace token required\n* Nginx reverse proxy with self-signed TLS, HTTP-to-HTTPS redirect, WebSocket upgrade for streaming chat\n* Loading splash page during model warmup\n* Anonymous telemetry disabled (VLLM_NO_USAGE_STATS, DO_NOT_TRACK, ANONYMIZED_TELEMETRY)\n* UFW firewall pre-configured (TCP 22, 80, 443 only; 8000 and 8080 explicitly denied)\n* fail2ban, auditd, AppArmor pre-configured\n* SBOM (CycloneDX 1.6) at /etc/lynxroute/sbom.json\n* CIS Conformance Report (OpenSCAP) at /etc/lynxroute/cis-report.html\n* IMDSv2 enforced\n",
   "source": "backfill-aws-listing",
   "live": {
    "aws": "2026-05-19"
   }
  }
 ],
 "release_history": [
  {
   "version": "0.32.0",
   "aws": "2026-10-25"
  },
  {
   "version": "0.30.0",
   "aws": "2026-10-23",
   "azure": "2026-10-05"
  },
  {
   "version": "0.29.0",
   "aws": "2026-09-11",
   "azure": "2026-09-14"
  },
  {
   "version": "0.28.0",
   "aws": "2026-08-28",
   "azure": "2026-08-29"
  },
  {
   "version": "0.27.1",
   "aws": "2026-08-18",
   "azure": "2026-08-19"
  },
  {
   "version": "0.26.0",
   "aws": "2026-07-25"
  },
  {
   "version": "0.25.1",
   "aws": "2026-07-20",
   "azure": "2026-07-20"
  },
  {
   "version": "0.24.0",
   "aws": "2026-07-11",
   "azure": "2026-07-13"
  },
  {
   "version": "0.23.0",
   "aws": "2026-06-27",
   "azure": "2026-06-21"
  },
  {
   "version": "0.22.1",
   "aws": "2026-06-06",
   "azure": "2026-06-07"
  },
  {
   "version": "0.21.0",
   "aws": "2026-05-19"
  }
 ],
 "deploy": {
  "aws_cli": "# subscribe once on the AWS Marketplace listing, then:\naws ec2 run-instances \\\n  --image-id resolve:ssm:/aws/service/marketplace/prod-ub33kxqn3lfxi/latest \\\n  --instance-type m6i.xlarge --key-name my-key --security-group-ids sg-xxxxxxxx",
  "azure_cli": "# accept the plan terms once, then:\naz vm image terms accept --publisher lynxroute --offer vllm --plan payg-gen2\naz vm create -g my-rg -n vllm --size Standard_D4s_v3 \\\n  --image lynxroute:vllm:payg-gen2:latest --admin-username azureuser --generate-ssh-keys"
 },
 "description": {
  "what_is": "vLLM is an open-source, high-throughput inference and serving engine for large language models, built in Python on top of PyTorch. It implements PagedAttention, continuous batching, and tensor parallelism to serve any HuggingFace-compatible transformer model (Llama, Mistral, Qwen, Phi, Gemma, OPT, GPT-J, MPT, Falcon, and 100+ more) through a fully OpenAI-compatible REST API. Any client built for OpenAI (openai-python, openai-node, LangChain, LlamaIndex, AnythingLLM, the OpenAI ChatGPT SDK) connects unchanged - just point the base URL at this instance and pass the local Bearer token. This image ships the CPU build of vLLM 0.26.0, bundled with Open WebUI 0.9.6 as a browser chat front end pre-wired to the local vLLM. Persists nothing externally - models cache to /var/lib/vllm/hf-cache, chats and accounts to /var/lib/open-webui. No vendor lock-in.\n\nLICENSING NOTE\n\nvLLM is Apache-2.0 licensed. Open WebUI is distributed under the source-available Open WebUI License (not an OSI-approved open-source license). The license permits self-hosted use without restriction; deployments with more than 50 end users in any 30-day window that also modify Open WebUI branding (name, logo) require a commercial Enterprise License from Open WebUI Inc. This image ships Open WebUI with the original branding preserved - the 50-user clause does not apply unless the operator rebrands.",
  "image_adds": "Security hardening:\n* Random 32-byte API key generated at first boot, written to /root/vllm-credentials.txt - never baked into the image; the same key is injected into Open WebUI so the chat UI authenticates to vLLM transparently\n* vLLM API server bound to 127.0.0.1:8000 only - reachable only through Nginx with TLS, with --api-key Bearer auth enforced on every /v1/* request\n* Open WebUI bound to 127.0.0.1:8080 only - reachable only through Nginx with TLS\n* First registered user in Open WebUI becomes the workspace administrator; no admin baked in\n* Nginx reverse proxy with TLS, HTTP-to-HTTPS redirect, WebSocket upgrade for streaming chat, security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy)\n* Loading splash page served while the model warms up on first request\n* Anonymous telemetry disabled (VLLM_NO_USAGE_STATS, DO_NOT_TRACK, ANONYMIZED_TELEMETRY)\n* UFW firewall pre-configured - only TCP 22, 80, 443 are exposed; 8000 and 8080 explicitly denied\n* fail2ban, AppArmor\n* CVE scan - every image is scanned for vulnerabilities before release\n\nOut of the box, with no external services:\n* Default tiny model facebook/opt-125m (~250 MB) loaded at first boot - chat and API work immediately, no HuggingFace token required\n* Swap to any HuggingFace-compatible model by editing /etc/vllm/server.env and restarting the service - no rebuild needed\n* No third-party LLM API keys baked in or required - everything runs on-instance\n\nOS hardening (CIS Level 1):\n* CIS Ubuntu 24.04 LTS Level 1 benchmark applied via ansible-lockdown\n* auditd, SSH hardening, kernel hardening, IMDSv2 enforced\n\nCompliance artifacts:\n* SBOM - CycloneDX 1.6 at /etc/lynxroute/sbom.json\n* CIS Conformance Report at /etc/lynxroute/cis-report.html\n* CIS Tailored Profile at /usr/share/doc/lynxroute/CIS_TAILORED_PROFILE.md"
 },
 "generated_at": "2026-10-31T09:36:34Z"
}