12 posts tagged with "Releases"

Published on August 6, 2026

We are excited to announce the release of KServe v0.20. This release delivers major new capabilities across the platform with key highlights including:

• Confidential model serving with TEE-based encrypted model decryption
• Traffic splitting API for progressive LLMInferenceService deployments
• Canary rollout support for InferenceService in RawDeployment mode
• KV cache offloading with CPU memory tiering for vLLM workloads
• Distributed tracing API for LLMInferenceService components
• Managed DRA (Dynamic Resource Allocation) for GPU provisioning
• Anthropic Messages API routing support
• Native OCI ImageVolume mounting (oci+native://) for model storage
• vLLM as a standalone runtime for InferenceService
• AutoGluon server with time series inference support

Published on June 14, 2026

We are excited to announce the release of KServe v0.19. This release focuses heavily on LLMInferenceService maturity with key highlights including:

• Static LoRA adapter support for efficient fine-tuned model serving
• Model name-based routing for shared-gateway deployments
• Graceful pod shutdown with vLLM drain to prevent in-flight request drops
• HPA/KEDA scaling status surfaced directly in service conditions
• LocalModelCache integration for LLMInferenceService
• Dual-protocol (REST/gRPC) routing for InferenceService Standard mode
• Seamless llm-d v0.6 to v0.7 migration with version-gated controller logic

Published on April 29, 2026

We are excited to announce the release of KServe v0.18. This release brings multi-node inference support without Ray, LeaderWorkerSet (LWS)-based autoscaling for multi-node workloads, OpenAI Responses API routing, namespace-scoped ModelCache, vLLM upgrade to v0.19.0, llm-d v0.6 integration, enhanced security hardening, and GKE Gateway compatibility improvements.

Published on March 13, 2026

We are excited to announce the release of KServe v0.17, a landmark release that brings LLMInferenceService to production readiness with a GenAI-first architecture built on the llm-d framework. This release introduces KV-cache aware intelligent routing, disaggregated prefill-decode, distributed inference with tensor/data/expert parallelism, Envoy AI Gateway integration with token-based rate limiting, and a completely restructured modular Helm chart architecture.

Published on May 27, 2025

We are thrilled to announce the release of KServe v0.15, marking a significant leap forward in serving both predictive and generative AI models. This release introduces enhanced support for generative AI workloads, including advanced features for serving large language models (LLMs), improved model and KV caching mechanisms, and integration with Envoy AI Gateway.

Published on December 23, 2024

We are excited to announce KServe v0.14. In this release we are introducing a new Python client designed for KServe, and a new model cache feature; we are promoting OCI storage for models as a stable feature; and we added support for deploying models directly from Hugging Face.

Published on May 15, 2024

We are excited to unveil KServe v0.13, marking a significant leap forward in evolving cloud native model serving to meet the demands of Generative AI inference. This release is highlighted by three pivotal updates: enhanced Hugging Face runtime, robust vLLM backend support for Generative Models, and the integration of OpenAI protocol standards.

Published on October 8, 2023

We are excited to announce the release of KServe 0.11. In this release we introduced Large Language Model (LLM) runtimes, made enhancements to the KServe control plane, Python SDK Open Inference Protocol support and dependency management. For ModelMesh we have added features PVC, HPA, payload logging to ensure feature parity with KServe.

Published on February 5, 2023

We are excited to announce KServe 0.10 release. In this release we have enabled more KServe networking options, improved KServe telemetry for supported serving runtimes and increased support coverage for Open(aka v2) inference protocol for both standard and ModelMesh InferenceService.

Published on July 21, 2022

Today, we are pleased to announce the v0.9.0 release of KServe! KServe has now fully onboarded to LF AI & Data Foundation as an Incubation Project! 🎉

In this release we are excited to introduce the new InferenceGraph feature which has long been asked from the community. Also continuing the effort from the last release for unifying the InferenceService API for deploying models on KServe and ModelMesh, ModelMesh is now fully compatible with KServe InferenceService API!