Skip to content

Kubernetes Deployment Installation Guide

KServe supports RawDeployment mode to enable InferenceService deployment with Kubernetes resources Deployment, Service, Ingress and Horizontal Pod Autoscaler. Comparing to serverless deployment it unlocks Knative limitations such as mounting multiple volumes, on the other hand Scale down and from Zero is not supported in RawDeployment mode.

Kubernetes 1.20 is the minimally required version and please check the following recommended Istio versions for the corresponding Kubernetes version.

Kubernetes Version Recommended Istio Version
1.21 1.10, 1.11
1.22 1.11, 1.12
1.23 1.12, 1.13
1.24 1.13, 1.14

1. Install Istio

The minimally required Istio version is 1.9.5 and you can refer to the Istio install guide.

Once Istio is installed, create IngressClass resource for istio. 

apiVersion: networking.k8s.io/v1beta1
kind: IngressClass
metadata:
  name: istio
spec:
  controller: istio.io/ingress-controller

Note

Istio ingress is recommended, but you can choose to install with other Ingress controllers and create IngressClass resource for your Ingress option.

2. Install Cert Manager

The minimally required Cert Manager version is 1.3.0 and you can refer to Cert Manager installation guide.

Note

Cert manager is required to provision webhook certs for production grade installation, alternatively you can run self signed certs generation script.

3. Install KServe

Note

The default KServe deployment mode is Serverless which depends on Knative. The following step changes the default deployment mode to RawDeployment before installing KServe.

i. Install KServe

kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.9.0/kserve.yaml

Install KServe default serving runtimes:

kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.9.0/kserve-runtimes.yaml

ii. Change default deployment mode and ingress option

First in ConfigMap inferenceservice-config modify the defaultDeploymentMode in the deploy section,

kubectl patch configmap/inferenceservice-config -n kserve --type=strategic -p '{"data": {"deploy": "{\"defaultDeploymentMode\": \"RawDeployment\"}"}}'

then modify the ingressClassName in ingress section to point to IngressClass name created in step 1.

```yaml
ingress: |-
{
    "ingressClassName" : "your-ingress-class",
}
```
Back to top