Skip to content

ModelServing

laktory.models.resources.databricks.ModelServing ¤

Bases: ModelServingBase

Databricks model serving endpoint

Examples:

import io

from laktory import models

model_serving_yaml = '''
name: my-endpoint
config:
  served_entities:
    - entity_name: my-catalog.my_schema.my_model
      entity_version: "1"
      workload_size: Small
      scale_to_zero_enabled: true
access_controls:
- group_name: role-engineers
  permission_level: CAN_QUERY
'''
model_serving = models.resources.databricks.ModelServing.model_validate_yaml(
    io.StringIO(model_serving_yaml)
)
References
BASE DESCRIPTION
ai_gateway

A block with AI Gateway configuration for the serving endpoint. Note: only external model endpoints are supported as of now.

TYPE: ModelServingAiGateway | None | VariableType DEFAULT: None

budget_policy_id

(Optiona) The Budget Policy ID set for this serving endpoint

TYPE: str | None | VariableType DEFAULT: None

config

The config for the external model, which must match the provider. Note that API keys could be provided either as a reference to the Databricks Secret (parameters without _plaintext suffix) or in plain text (parameters with _plaintext suffix)!

TYPE: ModelServingConfig | None | VariableType DEFAULT: None

description

The description of the model serving endpoint

TYPE: str | None | VariableType DEFAULT: None

email_notifications

A block with Email notification setting

TYPE: ModelServingEmailNotifications | None | VariableType DEFAULT: None

name

The name of a served model. It must be unique across an endpoint. If not specified, this field will default to modelname-modelversion. A served model name can consist of alphanumeric characters, dashes, and underscores

TYPE: str | VariableType

rate_limits

Block describing rate limits for AI gateway. For details see the description of rate_limits block above

TYPE: list[ModelServingRateLimits] | None | VariableType DEFAULT: None

route_optimized

A boolean enabling route optimization for the endpoint. Note: only available for custom models.

TYPE: bool | None | VariableType DEFAULT: None

tags

Tags to be attached to the serving endpoint and automatically propagated to billing logs

TYPE: list[ModelServingTags] | None | VariableType DEFAULT: None

telemetry_config

TYPE: ModelServingTelemetryConfig | None | VariableType DEFAULT: None

timeouts

TYPE: ModelServingTimeouts | None | VariableType DEFAULT: None

LAKTORY DESCRIPTION
access_controls

List of access controls

TYPE: list[AccessControl | VariableType] | VariableType DEFAULT: []

ATTRIBUTE DESCRIPTION
additional_core_resources
  • permissions

TYPE: list

additional_core_resources property ¤

  • permissions

laktory.models.resources.databricks.modelserving.ModelServingAiGateway ¤

Bases: BaseModel

PARAMETER DESCRIPTION
fallback_config

block with configuration for traffic fallback which auto fallbacks to other served entities if the request to a served entity fails with certain error codes, to increase availability

TYPE: ModelServingAiGatewayFallbackConfig | None | VariableType DEFAULT: None

guardrails

Block with configuration for AI Guardrails to prevent unwanted data and unsafe data in requests and responses. Consists of the following attributes:

TYPE: ModelServingAiGatewayGuardrails | None | VariableType DEFAULT: None

inference_table_config

Block describing the configuration of usage tracking. Consists of the following attributes:

TYPE: ModelServingAiGatewayInferenceTableConfig | None | VariableType DEFAULT: None

rate_limits

Block describing rate limits for AI gateway. For details see the description of rate_limits block above

TYPE: list[ModelServingAiGatewayRateLimits] | None | VariableType DEFAULT: None

usage_tracking_config

Block with configuration for payload logging using inference tables. For details see the description of auto_capture_config block above

TYPE: ModelServingAiGatewayUsageTrackingConfig | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayFallbackConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
enabled

boolean flag specifying if usage tracking is enabled

TYPE: bool | VariableType

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayGuardrails ¤

Bases: BaseModel

PARAMETER DESCRIPTION
input

A block with configuration for input guardrail filters:

TYPE: ModelServingAiGatewayGuardrailsInput | None | VariableType DEFAULT: None

output

A block with configuration for output guardrail filters. Has the same structure as input block

TYPE: ModelServingAiGatewayGuardrailsOutput | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayGuardrailsInput ¤

Bases: BaseModel

PARAMETER DESCRIPTION
invalid_keywords

(Deprecated) List of invalid keywords. AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content

TYPE: list[str] | None | VariableType DEFAULT: None

pii

Block with configuration for guardrail PII filter:

TYPE: ModelServingAiGatewayGuardrailsInputPii | None | VariableType DEFAULT: None

safety

the boolean flag that indicates whether the safety filter is enabled

TYPE: bool | None | VariableType DEFAULT: None

valid_topics

(Deprecated) The list of allowed topics. Given a chat request, this guardrail flags the request if its topic is not in the allowed topics

TYPE: list[str] | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayGuardrailsInputPii ¤

Bases: BaseModel

PARAMETER DESCRIPTION
behavior

a string that describes the behavior for PII filter. Currently only BLOCK value is supported

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayGuardrailsOutput ¤

Bases: BaseModel

PARAMETER DESCRIPTION
invalid_keywords

(Deprecated) List of invalid keywords. AI guardrail uses keyword or string matching to decide if the keyword exists in the request or response content

TYPE: list[str] | None | VariableType DEFAULT: None

pii

Block with configuration for guardrail PII filter:

TYPE: ModelServingAiGatewayGuardrailsOutputPii | None | VariableType DEFAULT: None

safety

the boolean flag that indicates whether the safety filter is enabled

TYPE: bool | None | VariableType DEFAULT: None

valid_topics

(Deprecated) The list of allowed topics. Given a chat request, this guardrail flags the request if its topic is not in the allowed topics

TYPE: list[str] | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayGuardrailsOutputPii ¤

Bases: BaseModel

PARAMETER DESCRIPTION
behavior

a string that describes the behavior for PII filter. Currently only BLOCK value is supported

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayInferenceTableConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
catalog_name

The name of the catalog in Unity Catalog. NOTE: On update, you cannot change the catalog name if it was already set

TYPE: str | None | VariableType DEFAULT: None

enabled

boolean flag specifying if usage tracking is enabled

TYPE: bool | None | VariableType DEFAULT: None

schema_name

The name of the schema in Unity Catalog. NOTE: On update, you cannot change the schema name if it was already set

TYPE: str | None | VariableType DEFAULT: None

table_name_prefix

The prefix of the table in Unity Catalog. NOTE: On update, you cannot change the prefix name if it was already set

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayRateLimits ¤

Bases: BaseModel

PARAMETER DESCRIPTION
calls

Used to specify how many calls are allowed for a key within the renewal_period

TYPE: int | None | VariableType DEFAULT: None

key

Key field for a serving endpoint rate limit. Currently, user, user_group, service_principal, and endpoint are supported, with endpoint being the default if not specified

TYPE: str | None | VariableType DEFAULT: None

principal

Principal field for a user, user group, or service principal to apply rate limiting to. Accepts a user email, group name, or service principal application ID

TYPE: str | None | VariableType DEFAULT: None

renewal_period

Renewal period field for a serving endpoint rate limit. Currently, only minute is supported

TYPE: str | VariableType

tokens

Specifies how many tokens are allowed for a key within the renewal_period

TYPE: int | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingAiGatewayUsageTrackingConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
enabled

boolean flag specifying if usage tracking is enabled

TYPE: bool | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
auto_capture_config

Configuration for Inference Tables which automatically logs requests and responses to Unity Catalog

TYPE: ModelServingConfigAutoCaptureConfig | None | VariableType DEFAULT: None

served_entities

A list of served entities for the endpoint to serve. A serving endpoint can have up to 10 served entities

TYPE: list[ModelServingConfigServedEntities] | None | VariableType DEFAULT: None

served_models

(Deprecated, use served_entities instead) Each block represents a served model for the endpoint to serve. A model serving endpoint can have up to 10 served models

TYPE: list[ModelServingConfigServedModels] | None | VariableType DEFAULT: None

traffic_config

A single block represents the traffic split configuration amongst the served models

TYPE: ModelServingConfigTrafficConfig | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigAutoCaptureConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
catalog_name

The name of the catalog in Unity Catalog. NOTE: On update, you cannot change the catalog name if it was already set

TYPE: str | None | VariableType DEFAULT: None

enabled

boolean flag specifying if usage tracking is enabled

TYPE: bool | None | VariableType DEFAULT: None

schema_name

The name of the schema in Unity Catalog. NOTE: On update, you cannot change the schema name if it was already set

TYPE: str | None | VariableType DEFAULT: None

table_name_prefix

The prefix of the table in Unity Catalog. NOTE: On update, you cannot change the prefix name if it was already set

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntities ¤

Bases: BaseModel

PARAMETER DESCRIPTION
burst_scaling_enabled

TYPE: bool | None | VariableType DEFAULT: None

entity_name

The name of the entity to be served. The entity may be a model in the Databricks Model Registry, a model in the Unity Catalog (UC), or a function of type FEATURE_SPEC in the UC. If it is a UC object, the full name of the object should be given in the form of catalog_name.schema_name.model_name

TYPE: str | None | VariableType DEFAULT: None

entity_version

The version of the model in Databricks Model Registry to be served or empty if the entity is a FEATURE_SPEC

TYPE: str | None | VariableType DEFAULT: None

environment_vars

a map of environment variable names/values that will be used for serving this model. Environment variables may refer to Databricks secrets using the standard syntax: {{secrets/secret_scope/secret_key}}

TYPE: dict[str, str] | None | VariableType DEFAULT: None

external_model

The external model to be served. NOTE: Only one of external_model and (entity_name, entity_version, workload_size, workload_type, and scale_to_zero_enabled) can be specified with the latter set being used for custom model serving for a Databricks registered model. When an external_model is present, the served entities list can only have one served_entity object. An existing endpoint with external_model can not be updated to an endpoint without external_model. If the endpoint is created without external_model, users cannot update it to add external_model later

TYPE: ModelServingConfigServedEntitiesExternalModel | None | VariableType DEFAULT: None

instance_profile_arn

ARN of the instance profile that the served model will use to access AWS resources

TYPE: str | None | VariableType DEFAULT: None

max_provisioned_concurrency

The maximum provisioned concurrency that the endpoint can scale up to. Conflicts with workload_size

TYPE: int | None | VariableType DEFAULT: None

max_provisioned_throughput

The maximum tokens per second that the endpoint can scale up to

TYPE: int | None | VariableType DEFAULT: None

min_provisioned_concurrency

The minimum provisioned concurrency that the endpoint can scale down to. Conflicts with workload_size

TYPE: int | None | VariableType DEFAULT: None

min_provisioned_throughput

The minimum tokens per second that the endpoint can scale down to

TYPE: int | None | VariableType DEFAULT: None

name

The name of a served model. It must be unique across an endpoint. If not specified, this field will default to modelname-modelversion. A served model name can consist of alphanumeric characters, dashes, and underscores

TYPE: str | None | VariableType DEFAULT: None

provisioned_model_units

TYPE: int | None | VariableType DEFAULT: None

scale_to_zero_enabled

Whether the compute resources for the served model should scale down to zero. If scale-to-zero is enabled, the lower bound of the provisioned concurrency for each workload size will be 0. The default value is true

TYPE: bool | None | VariableType DEFAULT: None

workload_size

The workload size of the served model. The workload size corresponds to a range of provisioned concurrency that the compute will autoscale between. A single unit of provisioned concurrency can process one request at a time. Valid workload sizes are Small (4 - 4 provisioned concurrency), Medium (8 - 16 provisioned concurrency), and Large (16 - 64 provisioned concurrency)

TYPE: str | None | VariableType DEFAULT: None

workload_type

The workload type of the served model. The workload type selects which type of compute to use in the endpoint. For deep learning workloads, GPU acceleration is available by selecting workload types like GPU_SMALL and others. See the documentation for all options. The default value is CPU

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModel ¤

Bases: BaseModel

PARAMETER DESCRIPTION
ai21labs_config

AI21Labs Config

TYPE: ModelServingConfigServedEntitiesExternalModelAi21labsConfig | None | VariableType DEFAULT: None

amazon_bedrock_config

Amazon Bedrock Config

TYPE: ModelServingConfigServedEntitiesExternalModelAmazonBedrockConfig | None | VariableType DEFAULT: None

anthropic_config

Anthropic Config

TYPE: ModelServingConfigServedEntitiesExternalModelAnthropicConfig | None | VariableType DEFAULT: None

cohere_config

Cohere Config

TYPE: ModelServingConfigServedEntitiesExternalModelCohereConfig | None | VariableType DEFAULT: None

custom_provider_config

Custom Provider Config. Only required if the provider is 'custom'

TYPE: ModelServingConfigServedEntitiesExternalModelCustomProviderConfig | None | VariableType DEFAULT: None

databricks_model_serving_config

Databricks Model Serving Config

TYPE: ModelServingConfigServedEntitiesExternalModelDatabricksModelServingConfig | None | VariableType DEFAULT: None

google_cloud_vertex_ai_config

Google Cloud Vertex AI Config

TYPE: ModelServingConfigServedEntitiesExternalModelGoogleCloudVertexAiConfig | None | VariableType DEFAULT: None

name

The name of a served model. It must be unique across an endpoint. If not specified, this field will default to modelname-modelversion. A served model name can consist of alphanumeric characters, dashes, and underscores

TYPE: str | VariableType

openai_config

OpenAI Config

TYPE: ModelServingConfigServedEntitiesExternalModelOpenaiConfig | None | VariableType DEFAULT: None

palm_config

PaLM Config

TYPE: ModelServingConfigServedEntitiesExternalModelPalmConfig | None | VariableType DEFAULT: None

provider

The name of the provider for the external model. Currently, the supported providers are ai21labs, anthropic, amazon-bedrock, cohere, databricks-model-serving, google-cloud-vertex-ai, openai, and palm

TYPE: str | VariableType

task

The task type of the external model

TYPE: str | VariableType

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelAi21labsConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
ai21labs_api_key

The Databricks secret key reference for an AI21Labs API key

TYPE: str | None | VariableType DEFAULT: None

ai21labs_api_key_plaintext

An AI21 Labs API key provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelAmazonBedrockConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
aws_access_key_id

The Databricks secret key reference for an AWS Access Key ID with permissions to interact with Bedrock services

TYPE: str | None | VariableType DEFAULT: None

aws_access_key_id_plaintext

An AWS access key ID with permissions to interact with Bedrock services provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

aws_region

The AWS region to use. Bedrock has to be enabled there

TYPE: str | VariableType

aws_secret_access_key

The Databricks secret key reference for an AWS Secret Access Key paired with the access key ID, with permissions to interact with Bedrock services

TYPE: str | None | VariableType DEFAULT: None

aws_secret_access_key_plaintext

An AWS secret access key paired with the access key ID, with permissions to interact with Bedrock services provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

bedrock_provider

The underlying provider in Amazon Bedrock. Supported values (case insensitive) include: Anthropic, Cohere, AI21Labs, Amazon

TYPE: str | VariableType

instance_profile_arn

ARN of the instance profile that the served model will use to access AWS resources

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelAnthropicConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
anthropic_api_key

The Databricks secret key reference for an Anthropic API key

TYPE: str | None | VariableType DEFAULT: None

anthropic_api_key_plaintext

The Anthropic API key provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelCohereConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
cohere_api_base

TYPE: str | None | VariableType DEFAULT: None

cohere_api_key

The Databricks secret key reference for a Cohere API key

TYPE: str | None | VariableType DEFAULT: None

cohere_api_key_plaintext

The Cohere API key provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelCustomProviderConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
api_key_auth

API key authentication for the custom provider API. Conflicts with bearer_token_auth

TYPE: ModelServingConfigServedEntitiesExternalModelCustomProviderConfigApiKeyAuth | None | VariableType DEFAULT: None

bearer_token_auth

bearer token authentication for the custom provider API. Conflicts with api_key_auth

TYPE: ModelServingConfigServedEntitiesExternalModelCustomProviderConfigBearerTokenAuth | None | VariableType DEFAULT: None

custom_provider_url

URL of the custom provider API

TYPE: str | VariableType

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelCustomProviderConfigApiKeyAuth ¤

Bases: BaseModel

PARAMETER DESCRIPTION
key

Key field for a serving endpoint rate limit. Currently, user, user_group, service_principal, and endpoint are supported, with endpoint being the default if not specified

TYPE: str | VariableType

value

The value field for a tag

TYPE: str | None | VariableType DEFAULT: None

value_plaintext

The API Key provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelCustomProviderConfigBearerTokenAuth ¤

Bases: BaseModel

PARAMETER DESCRIPTION
token

The Databricks secret key reference for a token

TYPE: str | None | VariableType DEFAULT: None

token_plaintext

The token provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelDatabricksModelServingConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
databricks_api_token

The Databricks secret key reference for a Databricks API token that corresponds to a user or service principal with Can Query access to the model serving endpoint pointed to by this external model

TYPE: str | None | VariableType DEFAULT: None

databricks_api_token_plaintext

The Databricks API token that corresponds to a user or service principal with Can Query access to the model serving endpoint pointed to by this external model provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

databricks_workspace_url

The URL of the Databricks workspace containing the model serving endpoint pointed to by this external model

TYPE: str | VariableType

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelGoogleCloudVertexAiConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
private_key

The Databricks secret key reference for a private key for the service account that has access to the Google Cloud Vertex AI Service

TYPE: str | None | VariableType DEFAULT: None

private_key_plaintext

The private key for the service account that has access to the Google Cloud Vertex AI Service is provided as a plaintext secret

TYPE: str | None | VariableType DEFAULT: None

project_id

This is the Google Cloud project id that the service account is associated with

TYPE: str | VariableType

region

This is the region for the Google Cloud Vertex AI Service

TYPE: str | VariableType

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelOpenaiConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
microsoft_entra_client_id

This field is only required for Azure AD OpenAI and is the Microsoft Entra Client ID

TYPE: str | None | VariableType DEFAULT: None

microsoft_entra_client_secret

The Databricks secret key reference for a client secret used for Microsoft Entra ID authentication

TYPE: str | None | VariableType DEFAULT: None

microsoft_entra_client_secret_plaintext

The client secret used for Microsoft Entra ID authentication provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

microsoft_entra_tenant_id

This field is only required for Azure AD OpenAI and is the Microsoft Entra Tenant ID

TYPE: str | None | VariableType DEFAULT: None

openai_api_base

This is the base URL for the OpenAI API (default: 'https://api.openai.com/v1'). For Azure OpenAI, this field is required and is the base URL for the Azure OpenAI API service provided by Azure

TYPE: str | None | VariableType DEFAULT: None

openai_api_key

The Databricks secret key reference for an OpenAI or Azure OpenAI API key

TYPE: str | None | VariableType DEFAULT: None

openai_api_key_plaintext

The OpenAI API key using the OpenAI or Azure service provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

openai_api_type

This is an optional field to specify the type of OpenAI API to use. For Azure OpenAI, this field is required, and this parameter represents the preferred security access validation protocol. For access token validation, use azure. For authentication using Azure Active Directory (Azure AD) use, azuread

TYPE: str | None | VariableType DEFAULT: None

openai_api_version

This is an optional field to specify the OpenAI API version. For Azure OpenAI, this field is required and is the version of the Azure OpenAI service to utilize, specified by a date

TYPE: str | None | VariableType DEFAULT: None

openai_deployment_name

This field is only required for Azure OpenAI and is the name of the deployment resource for the Azure OpenAI service

TYPE: str | None | VariableType DEFAULT: None

openai_organization

This is an optional field to specify the organization in OpenAI or Azure OpenAI

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedEntitiesExternalModelPalmConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
palm_api_key

The Databricks secret key reference for a PaLM API key

TYPE: str | None | VariableType DEFAULT: None

palm_api_key_plaintext

The PaLM API key provided as a plaintext string

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigServedModels ¤

Bases: BaseModel

PARAMETER DESCRIPTION
burst_scaling_enabled

TYPE: bool | None | VariableType DEFAULT: None

environment_vars

a map of environment variable names/values that will be used for serving this model. Environment variables may refer to Databricks secrets using the standard syntax: {{secrets/secret_scope/secret_key}}

TYPE: dict[str, str] | None | VariableType DEFAULT: None

instance_profile_arn

ARN of the instance profile that the served model will use to access AWS resources

TYPE: str | None | VariableType DEFAULT: None

max_provisioned_concurrency

The maximum provisioned concurrency that the endpoint can scale up to. Conflicts with workload_size

TYPE: int | None | VariableType DEFAULT: None

max_provisioned_throughput

The maximum tokens per second that the endpoint can scale up to

TYPE: int | None | VariableType DEFAULT: None

min_provisioned_concurrency

The minimum provisioned concurrency that the endpoint can scale down to. Conflicts with workload_size

TYPE: int | None | VariableType DEFAULT: None

min_provisioned_throughput

The minimum tokens per second that the endpoint can scale down to

TYPE: int | None | VariableType DEFAULT: None

model_name

The name of the model in Databricks Model Registry to be served

TYPE: str | VariableType

model_version

The version of the model in Databricks Model Registry to be served

TYPE: str | VariableType

name

The name of a served model. It must be unique across an endpoint. If not specified, this field will default to modelname-modelversion. A served model name can consist of alphanumeric characters, dashes, and underscores

TYPE: str | None | VariableType DEFAULT: None

provisioned_model_units

TYPE: int | None | VariableType DEFAULT: None

scale_to_zero_enabled

Whether the compute resources for the served model should scale down to zero. If scale-to-zero is enabled, the lower bound of the provisioned concurrency for each workload size will be 0. The default value is true

TYPE: bool | None | VariableType DEFAULT: None

workload_size

The workload size of the served model. The workload size corresponds to a range of provisioned concurrency that the compute will autoscale between. A single unit of provisioned concurrency can process one request at a time. Valid workload sizes are Small (4 - 4 provisioned concurrency), Medium (8 - 16 provisioned concurrency), and Large (16 - 64 provisioned concurrency)

TYPE: str | None | VariableType DEFAULT: None

workload_type

The workload type of the served model. The workload type selects which type of compute to use in the endpoint. For deep learning workloads, GPU acceleration is available by selecting workload types like GPU_SMALL and others. See the documentation for all options. The default value is CPU

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigTrafficConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
routes

Each block represents a route that defines traffic to each served entity. Each served_entity block needs to have a corresponding routes block

TYPE: list[ModelServingConfigTrafficConfigRoutes] | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingConfigTrafficConfigRoutes ¤

Bases: BaseModel

PARAMETER DESCRIPTION
served_entity_name

The name of the served entity this route configures traffic for. This needs to match the name of a served_entity block

TYPE: str | None | VariableType DEFAULT: None

served_model_name

TYPE: str | None | VariableType DEFAULT: None

traffic_percentage

The percentage of endpoint traffic to send to this route. It must be an integer between 0 and 100 inclusive

TYPE: int | VariableType

laktory.models.resources.databricks.modelserving.ModelServingEmailNotifications ¤

Bases: BaseModel

PARAMETER DESCRIPTION
on_update_failure

a list of email addresses to be notified when an endpoint fails to update its configuration or state

TYPE: list[str] | None | VariableType DEFAULT: None

on_update_success

a list of email addresses to be notified when an endpoint successfully updates its configuration or state

TYPE: list[str] | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingRateLimits ¤

Bases: BaseModel

PARAMETER DESCRIPTION
calls

Used to specify how many calls are allowed for a key within the renewal_period

TYPE: int | VariableType

key

Key field for a serving endpoint rate limit. Currently, user, user_group, service_principal, and endpoint are supported, with endpoint being the default if not specified

TYPE: str | None | VariableType DEFAULT: None

renewal_period

Renewal period field for a serving endpoint rate limit. Currently, only minute is supported

TYPE: str | VariableType

laktory.models.resources.databricks.modelserving.ModelServingTags ¤

Bases: BaseModel

PARAMETER DESCRIPTION
key

Key field for a serving endpoint rate limit. Currently, user, user_group, service_principal, and endpoint are supported, with endpoint being the default if not specified

TYPE: str | VariableType

value

The value field for a tag

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingTelemetryConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
inference_table_config

Block describing the configuration of usage tracking. Consists of the following attributes:

TYPE: ModelServingTelemetryConfigInferenceTableConfig | None | VariableType DEFAULT: None

table_names

TYPE: ModelServingTelemetryConfigTableNames | None | VariableType DEFAULT: None

telemetry_profile_id

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingTelemetryConfigInferenceTableConfig ¤

Bases: BaseModel

PARAMETER DESCRIPTION
name

The name of a served model. It must be unique across an endpoint. If not specified, this field will default to modelname-modelversion. A served model name can consist of alphanumeric characters, dashes, and underscores

TYPE: str | None | VariableType DEFAULT: None

sampling_fraction

TYPE: int | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingTelemetryConfigTableNames ¤

Bases: BaseModel

PARAMETER DESCRIPTION
annotations_table

TYPE: str | None | VariableType DEFAULT: None

logs_table

TYPE: str | None | VariableType DEFAULT: None

metrics_table

TYPE: str | None | VariableType DEFAULT: None

traces_table

TYPE: str | None | VariableType DEFAULT: None

laktory.models.resources.databricks.modelserving.ModelServingTimeouts ¤

Bases: BaseModel

PARAMETER DESCRIPTION
create

TYPE: str | None | VariableType DEFAULT: None

update_

TYPE: str | None | VariableType DEFAULT: None