InstancePool
laktory.models.resources.databricks.InstancePool
¤
Bases: InstancePoolBase
Databricks instance pool
Examples:
import io
from laktory import models
instance_pool_yaml = '''
instance_pool_name: default
node_type_id: Standard_DS3_v2
min_idle_instances: 0
idle_instance_autotermination_minutes: 10
access_controls:
- group_name: role-engineers
permission_level: CAN_ATTACH_TO
'''
instance_pool = models.resources.databricks.InstancePool.model_validate_yaml(
io.StringIO(instance_pool_yaml)
)
References
| BASE | DESCRIPTION |
|---|---|
aws_attributes
|
TYPE:
|
azure_attributes
|
TYPE:
|
custom_tags
|
(Map) Additional tags for instance pool resources. Databricks tags all pool resources (e.g. AWS & Azure instances and Disk volumes). The tags of the instance pool will propagate to the clusters using the pool (see the official documentation). Attempting to set the same tags in both cluster and instance pool will raise an error. Databricks allows at most 43 custom tags.
TYPE:
|
disk_spec
|
TYPE:
|
enable_elastic_disk
|
(Bool) Autoscaling Local Storage: when enabled, the instances in the pool dynamically acquire additional disk space when they are running low on disk space
TYPE:
|
gcp_attributes
|
TYPE:
|
idle_instance_autotermination_minutes
|
(Integer) The number of minutes that idle instances in excess of the min_idle_instances are maintained by the pool before being terminated. If not specified, excess idle instances are terminated automatically after a default timeout period. If specified, the time must be between 0 and 10000 minutes. If you specify 0, excess idle instances are removed as soon as possible
TYPE:
|
instance_pool_fleet_attributes
|
TYPE:
|
instance_pool_id
|
TYPE:
|
instance_pool_name
|
(String) The name of the instance pool. This is required for create and edit operations. It must be unique, non-empty, and less than 100 characters
TYPE:
|
max_capacity
|
(Integer) The maximum number of instances the pool can contain, including both idle instances and ones in use by clusters. Once the maximum capacity is reached, you cannot create new clusters from the pool and existing clusters cannot autoscale up until some instances are made idle in the pool via cluster termination or down-scaling. There is no default limit, but as a best practice, this should be set based on anticipated usage
TYPE:
|
min_idle_instances
|
(Integer) The minimum number of idle instances maintained by the pool. This is in addition to any instances in use by active clusters
TYPE:
|
node_type_flexibility
|
a block describing the alternative driver node types if
TYPE:
|
node_type_id
|
(String) The node type for the instances in the pool. All clusters attached to the pool inherit this node type and the pool's idle instances are allocated based on this type. You can retrieve a list of available node types by using the List Node Types API call
TYPE:
|
preloaded_docker_image
|
TYPE:
|
preloaded_spark_versions
|
(List) A list with at most one runtime version the pool installs on each instance. Pool clusters that use a preloaded runtime version start faster as they do not have to wait for the image to download. You can retrieve them via databricks_spark_version data source or via Runtime Versions API call
TYPE:
|
| LAKTORY | DESCRIPTION |
|---|---|
access_controls
|
List of access controls
TYPE:
|
| ATTRIBUTE | DESCRIPTION |
|---|---|
additional_core_resources |
TYPE:
|
additional_core_resources
property
¤
- permissions
laktory.models.resources.databricks.instancepool.InstancePoolAwsAttributes
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
availability
|
Availability type used for all nodes. Valid values are
TYPE:
|
instance_profile_arn
|
Nodes belonging to the pool will only be placed on AWS instances with this instance profile. Please see databricks_instance_profile resource documentation for extended examples on adding a valid instance profile using Terraform
TYPE:
|
spot_bid_price_percent
|
(Integer) The max price for AWS spot instances, as a percentage of the corresponding instance type's on-demand price. For example, if this field is set to 50, and the instance pool needs a new i3.xlarge spot instance, then the max price is half of the price of on-demand i3.xlarge instances. Similarly, if this field is set to 200, the max price is twice the price of on-demand i3.xlarge instances. If not specified, the default value is 100. When spot instances are requested for this instance pool, only spot instances whose max price percentage matches this field are considered. For safety, this field cannot be greater than 10000.
TYPE:
|
zone_id
|
Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolAzureAttributes
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
availability
|
Availability type used for all nodes. Valid values are
TYPE:
|
spot_bid_max_price
|
The max bid price used for Azure spot instances. You can set this to greater than or equal to the current spot price. You can also set this to
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolDiskSpec
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
disk_count
|
(Integer) The number of disks to attach to each instance. This feature is only enabled for supported node types. Users can choose up to the limit of the disks supported by the node type. For node types with no local disk, at least one disk needs to be specified
TYPE:
|
disk_size
|
(Integer) The size of each disk (in GiB) to attach
TYPE:
|
disk_type
|
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolDiskSpecDiskType
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
azure_disk_volume_type
|
TYPE:
|
ebs_volume_type
|
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolGcpAttributes
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
gcp_availability
|
Availability type used for all nodes. Valid values are
TYPE:
|
local_ssd_count
|
TYPE:
|
zone_id
|
Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolInstancePoolFleetAttributes
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
fleet_on_demand_option
|
TYPE:
|
fleet_spot_option
|
TYPE:
|
launch_template_override
|
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolInstancePoolFleetAttributesFleetOnDemandOption
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
allocation_strategy
|
TYPE:
|
instance_pools_to_use_count
|
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolInstancePoolFleetAttributesFleetSpotOption
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
allocation_strategy
|
TYPE:
|
instance_pools_to_use_count
|
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolInstancePoolFleetAttributesLaunchTemplateOverride
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
availability_zone
|
TYPE:
|
instance_type
|
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolNodeTypeFlexibility
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
alternate_node_type_ids
|
list of alternative node types that will be used if main node type isn't available. Follow the documentation for requirements on selection of alternative node types
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolPreloadedDockerImage
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
basic_auth
|
TYPE:
|
url
|
URL for the Docker image
TYPE:
|
laktory.models.resources.databricks.instancepool.InstancePoolPreloadedDockerImageBasicAuth
¤
Bases: BaseModel
| PARAMETER | DESCRIPTION |
|---|---|
password
|
TYPE:
|
username
|
TYPE:
|