Skip to main content
Version: Next

LocalModel Installation

This guide covers installation of the LocalModel controller for efficient model caching.

Prerequisites​

Required: LocalModel MUST be installed after KServe controller is already installed.

note

LocalModel is an optional add-on component that requires KServe controller to be installed first. It currently only supports InferenceService workloads. Support for LLMInferenceService is planned for future releases.

Overview​

LocalModel provides efficient model caching capabilities:

  • Controller: Manages model cache lifecycle and policies
  • Agent: DaemonSet deployed on worker nodes for local caching
  • Node Selection: Target specific nodes for model caching
  • Cache Policies: Configurable cache size, eviction, and preloading

Installation Methods​

Method 1: Kustomize​

# Clone KServe repository
git clone https://github.com/kserve/kserve.git
cd kserve

# Install LocalModel (requires KServe to be already installed)
kubectl apply -k config/overlays/addons/localmodel

Method 2: Helm​

Install CRDs​

# Using OCI registry (recommended)
helm install localmodel-crd oci://ghcr.io/kserve/charts/kserve-localmodel-crd \
  --version v0.17.0-rc1 \
  --namespace kserve \
  --create-namespace

# Or using local charts
helm install localmodel-crd ./charts/kserve-localmodel-crd \
  --namespace kserve \
  --create-namespace

Install LocalModel Resources​

# Using OCI registry (recommended)
helm install localmodel-resources oci://ghcr.io/kserve/charts/kserve-localmodel-resources \
  --version v0.17.0-rc1 \
  --namespace kserve

# Or using local charts
helm install localmodel-resources ./charts/kserve-localmodel-resources \
  --namespace kserve

Configuration Options​

For detailed configuration options including node selector, resource limits, and storage configurations, see the kserve-localmodel-resources Helm Chart README.

Uninstallation​

Helm​

# Remove resources
helm uninstall localmodel-resources -n kserve
helm uninstall localmodel-crd -n kserve

Kustomize​

# Remove LocalModel
kubectl delete -k config/overlays/standalone/localmodel

Cleanup​

# Remove node labels
kubectl label nodes -l kserve/localmodel=worker kserve/localmodel-

# Clean up cached models (optional)
kubectl delete localmodelcache --all
kubectl delete localmodelnodegroup --all

Next Steps​