> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://apidocs.polytomic.com/2024-02-08/guides/configuring-your-connections/connections/databricks/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://apidocs.polytomic.com/_mcp/server. # Databricks Supports: * ✅ Models * ✅ Model sync destination * ✅ Bulk sync source * ✅ Bulk sync destination ## Connection ### Configuration
NameTypeDescriptionRequired
cloud_provider string Cloud provider (destination support only)

Accepted values: `aws` ↓, `azure` ↓
optional
databricks_auth_mode string Authentication method

Accepted values: `access_token` ↓, `oauth_service_principal` ↓
required
enable_delta_uniform boolean Enable Delta UniForm tables optional
enforce_query_limit boolean Limit concurrent queries ↓ optional
http_path string HTTP path required
port integer Port required
server_hostname string Server hostname required
ssh boolean Connect over SSH tunnel ↓ optional
unity_catalog_enabled boolean Unity Catalog enabled ↓ optional
use_bulk_sync_staging_schema boolean Use custom bulk sync staging schema ↓ optional
#### `cloud_provider` = `aws`
NameTypeDescriptionRequired
auth_mode string AWS authentication method

How to authenticate with AWS. Defaults to Access Key and Secret. Accepted values: `access_key_and_secret` ↓, `iam_role` ↓
required
s3_bucket_name string S3 bucket name (destinations only)

Name of bucket used for staging data load files
optional
s3_bucket_region string S3 bucket region (destinations only)

Region of bucket
optional
set_retention_properties boolean Configure data retention for tables ↓ optional
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "cloud_provider": "aws", "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": false, "ssh": false, "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` ##### `auth_mode` When `auth_mode` is `access_key_and_secret`:
NameTypeDescriptionRequired
aws_access_key_id string AWS access key ID (destinations only)

See https://docs.polytomic.com/docs/databricks-connections#writing-to-databricks
required
aws_secret_access_key string AWS secret access key (destinations only) required
Read-only propertyTypeDescription
aws_user string User ARN (destinations only)
When `auth_mode` is `iam_role`:
NameTypeDescriptionRequired
iam_role_arn string IAM role ARN required
storage_credential_name string Storage credential name optional
Read-only propertyTypeDescription
external_id string External ID for the IAM role
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "iam_role", "cloud_provider": "aws", "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "iam_role_arn": "", "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": false, "ssh": false, "storage_credential_name": "", "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` ##### `set_retention_properties` When `set_retention_properties` is `true`:
NameTypeDescriptionRequired
deleted_file_retention_days integer Deleted file retention optional
log_file_retention_days integer Log retention optional
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "cloud_provider": "aws", "databricks_auth_mode": "access_token", "deleted_file_retention_days": 0, "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "log_file_retention_days": 0, "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": true, "ssh": false, "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` #### `cloud_provider` = `azure`
NameTypeDescriptionRequired
azure_access_key string Storage account access key (destination support only)

The access key associated with this storage account
optional
azure_account_name string Storage account name (destination support only)

The account name of the storage account
optional
container_name string Storage container name (destination support only)

The container which we will stage files in
optional
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "azure_access_key": "abcdefghijklmnopqrstuvwxyz0123456789/+ABCDEabcdefghijklmnopqrstuvwxyz0123456789/+ABCDE==", "azure_account_name": "account", "cloud_provider": "azure", "container_name": "container", "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "port": 443, "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "ssh": false, "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` #### `databricks_auth_mode` When `databricks_auth_mode` is `access_token`:
NameTypeDescriptionRequired
access_token string Access token required
When `databricks_auth_mode` is `oauth_service_principal`:
NameTypeDescriptionRequired
service_principal_id string Service principal ID required
service_principal_secret string Service principal secret required
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "cloud_provider": "aws", "databricks_auth_mode": "oauth_service_principal", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "service_principal_id": "sp-1234abcd", "service_principal_secret": "abcdefghijklmnopqrstuvwxyz0123456789/+ABCDEabcdefghijklmnopqrstuvwxyz0123456789/+ABCDE==", "set_retention_properties": false, "ssh": false, "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` #### `enforce_query_limit` When `enforce_query_limit` is `true`:
NameTypeDescriptionRequired
concurrent_queries integer Concurrent query limit optional
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "cloud_provider": "aws", "concurrent_queries": 0, "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": true, "http_path": "/sql", "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": false, "ssh": false, "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` #### `ssh` When `ssh` is `true`:
NameTypeDescriptionRequired
ssh_blob_storage boolean Use SSH for cloud storage bucket optional
ssh_host string SSH host required
ssh_port integer SSH port required
ssh_private_key string Private key required
ssh_user string SSH user required
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "cloud_provider": "aws", "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": false, "ssh": true, "ssh_blob_storage": false, "ssh_host": "bastion.example.com", "ssh_port": 22, "ssh_private_key": "", "ssh_user": "root", "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": false } } ``` #### `unity_catalog_enabled` When `unity_catalog_enabled` is `true`:
NameTypeDescriptionRequired
include_hive_metastore boolean Include legacy Hive metastore tables optional
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "cloud_provider": "aws", "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "include_hive_metastore": false, "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": false, "ssh": false, "unity_catalog_enabled": true, "use_bulk_sync_staging_schema": false } } ``` #### `use_bulk_sync_staging_schema` When `use_bulk_sync_staging_schema` is `true`:
NameTypeDescriptionRequired
bulk_sync_staging_schema string Staging schema name required
```json { "name": "Databricks connection", "type": "databricks", "configuration": { "access_token": "isoz8af6zvp8067gu68gvrp0oftevn", "auth_mode": "access_key_and_secret", "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE", "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY", "bulk_sync_staging_schema": "", "cloud_provider": "aws", "databricks_auth_mode": "access_token", "enable_delta_uniform": false, "enforce_query_limit": false, "http_path": "/sql", "port": 443, "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset", "s3_bucket_region": "us-east-1", "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com", "set_retention_properties": false, "ssh": false, "unity_catalog_enabled": false, "use_bulk_sync_staging_schema": true } } ``` ## Model Sync ### Source #### Configuration
NameTypeDescriptionRequired
catalog string Catalog optional
query string Query optional
schema string Schema optional
table string Table optional
#### Example ```json { ... "configuration": { "catalog": "samples", "query": "SELECT * FROM samples.nyctaxi.trips", "schema": "nyctaxi", "table": "trips" } } ``` ### Target Databricks connections may be used as the destination in a model sync. #### All targets ##### Configuration
NameTypeDescriptionRequired
created_column string 'Created at' timestamp column optional
preserve_table_on_resync boolean Preserve destination table when resyncing optional
updated_column string 'Updated at' timestamp column optional
write_record_timestamps boolean Write row timestamp metadata optional
##### Example ```json { ... "target": { "configuration": { "created_column": "", "preserve_table_on_resync": false, "updated_column": "", "write_record_timestamps": false } } } ``` ### Target creation Databricks connections may be used to create a new target for a model sync. The following parameters are required to create a new target:
NAME DESCRIPTION ENUM
name Table name false
## Bulk Sync ### Source Databricks connections may be used as a bulk sync source. No additional configuration options are required. ### Destination #### Configuration
NameTypeDescriptionRequired
advanced object optional
catalog string Catalog optional
external_location_name string External location optional
mirror_schemas boolean Mirror schemas optional
schema string Output schema optional
#### Example ```json { ... "destination_configuration": { "advanced": { "deleted_file_retention_days": 0, "empty_strings_null": false, "enable_change_data_feed": false, "hard_deletes": false, "initial_execution": "rebuild", "log_file_retention_days": 0, "set_retention_properties": false, "table_prefix": "", "truncate_existing": false }, "catalog": "samples", "external_location_name": "", "mirror_schemas": false, "schema": "nyctaxi" } } ``` ## Type handling ### Destination types | POLYTOMIC TYPE | DATABRICKS TYPE | |-----------------------------|----------------------------| | `array<>` | `ARRAY<>` | | `bigint` | `BIGINT` | | `boolean` | `BOOLEAN` | | `date` | `DATE` | | `datetime` | `TIMESTAMP` | | `decimal(precision, scale)` | `DECIMAL(precision,scale)` | | `double` | `DOUBLE` | | `int` | `INT` | | `json` | `STRING` | | `jsonarray` | `STRING` | | `number` | `DECIMAL(38,18)` | | `object{}` | `STRING` | | `single` | `FLOAT` | | `smallint` | `SMALLINT` | | `string` | `STRING` | | `time` | `TIMESTAMP` | ### Source types | DATABRICKS TYPE | POLYTOMIC TYPE | |-----------------------------|-----------------------------| | `ARRAY<>` | `array<>` | | `BIGINT` | `bigint` | | `BYTE` | `smallint` | | `DATE` | `date` | | `DECIMAL(precision, scale)` | `decimal(precision, scale)` | | `DOUBLE` | `double` | | `FLOAT` | `single` | | `INT` | `int` | | `INTERVAL` | `string` | | `LONG` | `bigint` | | `MAP<>` | `object{}` | | `SHORT` | `smallint` | | `SMALLINT` | `smallint` | | `STRUCT<>` | `object{}` | | `TIMESTAMP` | `datetime_tz` | | `TIMESTAMP_NTZ` | `datetime` | | `TINYINT` | `smallint` | | `VARCHAR` | `string` |