Skip to main content

Store Logs in Blob Storage

The log messages can be stored directly to blob storage. Similar to the model storage configuration, the logger supports access to the blob storage via service account.

Prerequisites​

Configuration Process​

1. Create Service Account and Secret​

The service account must exist and contain the credentials for the blob storage. First, create a secret with the credentials that the logger agent will use to access the blob storage. The secret must be in the same namespace as the InferenceService.

apiVersion: v1
kind: Secret
metadata:
  annotations:
      serving.kserve.io/s3-region: [YOUR_REGION]
  name: agent-logger-secret
  namespace: default
type: Opaque
data:
  AWS_ACCESS_KEY_ID: [YOUR_ACCESS_KEY_ID]
  AWS_DEFAULT_REGION: [YOUR_REGION]
  AWS_SECRET_ACCESS_KEY: [YOUR_SECRET_ACCESS_KEY]

Next, create a service account that provides the secret.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: logger-sa
  namespace: default
secrets:
- name: agent-logger-secret

2. Create the InferenceService​

Create the inference service and configure the blob storage logger.

When specifying the logger configuration you must specify the logger format for the data that will be stored. Valid values are:

  • json - The log messages will be stored as JSON files.

Additional supported formats are planned for future releases, such as parquet and csv.

note

Currently, the blob storage implementation is limited to S3 storage.

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: sklearn-iris
spec:
predictor:
  logger:
    mode: all
    url: s3://[YOUR_BUCKET_NAME]
    storage:
      path: /logs
      parameters:
        type: s3
        format: json
      key: logger-credentials
      serviceAccountName: logger-sa
  model:
    modelFormat:
      name: sklearn
    storageUri: gs://kfserving-examples/models/sklearn/1.0/model

Storage Format​

When inferences occur they will get stored as JSON files in the specified bucket. There will be a file for the request and a file for the response, and the response will contain the actual prediction in BASE64 encoding.

Benefits of Blob Storage Logging​

  • Persistence: Log data is stored durably in blob storage, even if the cluster fails
  • Scalability: Cloud storage can handle large volumes of log data
  • Separation of Concerns: Logging system is decoupled from your serving infrastructure
  • Data Analytics: Stored logs can be analyzed using big data processing tools