Serverless Installation Guide¶
KServe Serverless installation enables autoscaling based on request volume and supports scale down to and from zero. It also supports revision management and canary rollout based on revisions.
Kubernetes 1.22 is the minimally required version and please check the following recommended Knative, Istio versions for the corresponding Kubernetes version.
Recommended Version Matrix¶
| Kubernetes Version | Recommended Istio Version | Recommended Knative Version |
|---|---|---|
| 1.22 | 1.11, 1.12 | 1.4-1.7 |
| 1.23 | 1.12, 1.13 | 1.4-1.8 |
| 1.24 | 1.13, 1.14 | 1.4-1.9 |
| 1.25 | 1.15, 1.16 | 1.4-1.9 |
1. Install Knative Serving¶
Please refer to Knative Serving install guide.
Note
If you are looking to use PodSpec fields such as nodeSelector, affinity or tolerations which are now supported in the v1beta1 API spec, you need to turn on the corresponding feature flags in your Knative configuration.
Warning
In Knative 1.8, The cluster domain suffix is changed to svc.cluster.local as the default domain. As routes using the cluster domain suffix are not exposed through Ingress, you will need to configure DNS in order to expose their services (most users probably already are).
2. Install Networking Layer¶
The recommended networking layer for KServe is Istio as currently it works best with KServe, please refer to the Istio install guide. Alternatively you can also choose other networking layers like Kourier or Contour, see how to install Kourier with KServe guide.
3. Install Cert Manager¶
The minimally required Cert Manager version is 1.9.0 and you can refer to Cert Manager.
Note
Cert manager is required to provision webhook certs for production grade installation, alternatively you can run self signed certs generation script.
4. Install KServe¶
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.10.0/kserve.yaml
5. Install KServe Built-in ClusterServingRuntimes¶
kubectl apply -f https://github.com/kserve/kserve/releases/download/v0.10.0/kserve-runtimes.yaml
Note
ClusterServingRuntimes are required to create InferenceService for built-in model serving runtimes with KServe v0.8.0 or higher.