> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://apidocs.polytomic.com/2022-12-12/configuring-your-connections/connections/databricks/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://apidocs.polytomic.com/_mcp/server.

# Databricks

Supports:
* ✅ Models
* ✅ Model sync destination
* ✅ Bulk sync source
* ✅ Bulk sync destination




## Connection

### Configuration
<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>cloud_provider</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Cloud provider (destination support only)<br/><br/>Accepted values: `aws` <a href="#cloud-provider-aws">↓</a>, `azure` <a href="#cloud-provider-azure">↓</a></td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>databricks_auth_mode</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Authentication method<br/><br/>Accepted values: `access_token` <a href="#databricks-auth-mode-access-token">↓</a>, `oauth_service_principal` <a href="#databricks-auth-mode-oauth-service-principal">↓</a></td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>enable_delta_uniform</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Enable Delta UniForm tables</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>enforce_query_limit</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Limit concurrent queries <a href="#enforce-query-limit">↓</a></td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>http_path</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">HTTP path</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>port</code></td>
<td><code>integer</code></td>
<td style="vertical-align: top;">Port</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>server_hostname</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Server hostname</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>ssh</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Connect over SSH tunnel <a href="#ssh">↓</a></td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>unity_catalog_enabled</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Unity Catalog enabled <a href="#unity-catalog-enabled">↓</a></td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>use_bulk_sync_staging_schema</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Use custom bulk sync staging schema <a href="#use-bulk-sync-staging-schema">↓</a></td>
<td><code>false</code></td>
</tr>
</tbody>
</table>

<h4 id="databricks-auth-mode">`databricks_auth_mode`</h4>

<h5 id="databricks-auth-mode-access-token">When `databricks_auth_mode` is `access_token`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>access_token</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Access token</td>
<td><code>true</code></td>
</tr>
</tbody>
</table>

<h5 id="databricks-auth-mode-oauth-service-principal">When `databricks_auth_mode` is `oauth_service_principal`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>service_principal_id</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Service principal ID</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>service_principal_secret</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Service principal secret</td>
<td><code>true</code></td>
</tr>
</tbody>
</table>


<h4 id="cloud-provider">`cloud_provider`</h4>

<h5 id="cloud-provider-aws">When `cloud_provider` is `aws`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>auth_mode</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">AWS authentication method<br/><br/>How to authenticate with AWS. Defaults to Access Key and Secret. Accepted values: `access_key_and_secret`, `iam_role`</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>s3_bucket_name</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">S3 bucket name (destinations only)<br/><br/>Name of bucket used for staging data load files</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>s3_bucket_region</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">S3 bucket region (destinations only)<br/><br/>Region of bucket</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>set_retention_properties</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Configure data retention for tables</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>

<h5 id="cloud-provider-azure">When `cloud_provider` is `azure`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>azure_access_key</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Storage account access key (destination support only)<br/><br/>The access key associated with this storage account</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>azure_account_name</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Storage account name (destination support only)<br/><br/>The account name of the storage account</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>container_name</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Storage container name (destination support only)<br/><br/>The container which we will stage files in</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


<h4 id="use-bulk-sync-staging-schema">`use_bulk_sync_staging_schema`</h4>

<h5 id="use-bulk-sync-staging-schema-true">When `use_bulk_sync_staging_schema` is `true`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>bulk_sync_staging_schema</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Staging schema name</td>
<td><code>true</code></td>
</tr>
</tbody>
</table>


<h4 id="enforce-query-limit">`enforce_query_limit`</h4>

<h5 id="enforce-query-limit-true">When `enforce_query_limit` is `true`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>concurrent_queries</code></td>
<td><code>integer</code></td>
<td style="vertical-align: top;">Concurrent query limit</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


<h4 id="unity-catalog-enabled">`unity_catalog_enabled`</h4>

<h5 id="unity-catalog-enabled-true">When `unity_catalog_enabled` is `true`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>include_hive_metastore</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Include legacy Hive metastore tables</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


<h4 id="ssh">`ssh`</h4>

<h5 id="ssh-true">When `ssh` is `true`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>ssh_blob_storage</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Use SSH for cloud storage bucket</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>ssh_host</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">SSH host</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>ssh_port</code></td>
<td><code>integer</code></td>
<td style="vertical-align: top;">SSH port</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>ssh_private_key</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Private key</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>ssh_user</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">SSH user</td>
<td><code>true</code></td>
</tr>
</tbody>
</table>


<h4 id="auth-mode">`auth_mode`</h4>

<h5 id="auth-mode-access-key-and-secret">When `auth_mode` is `access_key_and_secret`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>aws_access_key_id</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">AWS access key ID (destinations only)<br/><br/>See https://docs.polytomic.com/docs/databricks-connections#writing-to-databricks</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>aws_secret_access_key</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">AWS secret access key (destinations only)</td>
<td><code>true</code></td>
</tr>
</tbody>
</table>

<h5 id="auth-mode-iam-role">When `auth_mode` is `iam_role`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>iam_role_arn</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">IAM role ARN</td>
<td><code>true</code></td>
</tr>
<tr>
<td><code>storage_credential_name</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Storage credential name</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


<h4 id="set-retention-properties">`set_retention_properties`</h4>

<h5 id="set-retention-properties-true">When `set_retention_properties` is `true`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>deleted_file_retention_days</code></td>
<td><code>integer</code></td>
<td style="vertical-align: top;">Deleted file retention</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>log_file_retention_days</code></td>
<td><code>integer</code></td>
<td style="vertical-align: top;">Log retention</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>



#### Example

```json
{
  "name": "Databricks connection",
  "type": "databricks",
  "configuration": {
    "access_token": "isoz8af6zvp8067gu68gvrp0oftevn",
    "auth_mode": "access_key_and_secret",
    "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "cloud_provider": "aws",
    "databricks_auth_mode": "access_token",
    "enable_delta_uniform": false,
    "enforce_query_limit": false,
    "http_path": "/sql",
    "port": 443,
    "s3_bucket_name": "s3://polytomic-databricks-results/customer-dataset",
    "s3_bucket_region": "us-east-1",
    "server_hostname": "dbc-1234dsafas-d0001.cloud.databricks.com",
    "set_retention_properties": false,
    "ssh": false,
    "unity_catalog_enabled": false,
    "use_bulk_sync_staging_schema": false
  }
}
```

### Read-only properties
<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
</tbody>
</table>

<h4 id="auth-mode">`auth_mode`</h4>

<h5 id="auth-mode-access-key-and-secret">When `auth_mode` is `access_key_and_secret`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>aws_user</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">User ARN (destinations only)</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>

<h5 id="auth-mode-iam-role">When `auth_mode` is `iam_role`</h5>

<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>external_id</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">External ID for the IAM role</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>



## Model Sync
### Source
#### Configuration
<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>catalog</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Catalog</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>query</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Query</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>schema</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Schema</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>table</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Table</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


#### Example
```json
{
  ...
  "configuration": {
    "catalog": "samples",
    "query": "SELECT * FROM samples.nyctaxi.trips",
    "schema": "nyctaxi",
    "table": "trips"
  }
}
```

### Target
Databricks connections may be used as the destination in a model sync.
#### All targets

##### Configuration
<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>created_column</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">'Created at' timestamp column</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>preserve_table_on_resync</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Preserve destination table when resyncing</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>updated_column</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">'Updated at' timestamp column</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>write_record_timestamps</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Write row timestamp metadata</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


##### Example
```json
{
  ...
  "target": {
      "configuration": {
        "created_column": "",
        "preserve_table_on_resync": false,
        "updated_column": "",
        "write_record_timestamps": false
    }
  }
}
```





### Target creation

Databricks connections may be used to create a new target for a model sync. The
following parameters are required to create a new target:

<table>
<thead>
<tr>
  <th>NAME</th>
  <th>DESCRIPTION</th>
  <th>ENUM</th>
</tr></thead>
<tbody>
<tr>
  <td>name</td>
  <td>Table name</td>
  <td>false</td>
</tr>

</tbody>
</table>




## Bulk Sync
### Source
Databricks connections may be used as a bulk sync source. No additional configuration options are required.

### Destination
#### Configuration
<table>
<thead>
<tr>
<th>Name</th><th>Type</th><th>Description</th><th>Required</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>advanced</code></td>
<td><code>object</code></td>
<td style="vertical-align: top;"></td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>catalog</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Catalog</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>external_location_name</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">External location</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>mirror_schemas</code></td>
<td><code>boolean</code></td>
<td style="vertical-align: top;">Mirror schemas</td>
<td><code>false</code></td>
</tr>
<tr>
<td><code>schema</code></td>
<td><code>string</code></td>
<td style="vertical-align: top;">Output schema</td>
<td><code>false</code></td>
</tr>
</tbody>
</table>


#### Example
```json
{
  ...
  "destination_configuration": {
    "advanced": {
        "deleted_file_retention_days": 0,
        "empty_strings_null": false,
        "enable_change_data_feed": false,
        "hard_deletes": false,
        "initial_execution": "",
        "log_file_retention_days": 0,
        "set_retention_properties": false,
        "table_prefix": "",
        "truncate_existing": false
    },
    "catalog": "samples",
    "external_location_name": "",
    "mirror_schemas": false,
    "schema": "nyctaxi"
  }
}
```



## Type handling
### Destination types
|       POLYTOMIC TYPE        |      DATABRICKS TYPE       |
|-----------------------------|----------------------------|
| `array<>`                   | `ARRAY<>`                  |
| `bigint`                    | `BIGINT`                   |
| `boolean`                   | `BOOLEAN`                  |
| `date`                      | `DATE`                     |
| `datetime`                  | `TIMESTAMP`                |
| `decimal(precision, scale)` | `DECIMAL(precision,scale)` |
| `double`                    | `DOUBLE`                   |
| `int`                       | `INT`                      |
| `json`                      | `STRING`                   |
| `jsonarray`                 | `STRING`                   |
| `number`                    | `DECIMAL(38,18)`           |
| `object{}`                  | `STRING`                   |
| `single`                    | `FLOAT`                    |
| `smallint`                  | `SMALLINT`                 |
| `string`                    | `STRING`                   |
| `time`                      | `TIMESTAMP`                |


### Source types
|       DATABRICKS TYPE       |       POLYTOMIC TYPE        |
|-----------------------------|-----------------------------|
| `ARRAY<>`                   | `array<>`                   |
| `BIGINT`                    | `bigint`                    |
| `BYTE`                      | `smallint`                  |
| `DATE`                      | `date`                      |
| `DECIMAL(precision, scale)` | `decimal(precision, scale)` |
| `DOUBLE`                    | `double`                    |
| `FLOAT`                     | `single`                    |
| `INT`                       | `int`                       |
| `INTERVAL`                  | `string`                    |
| `LONG`                      | `bigint`                    |
| `MAP<>`                     | `object{}`                  |
| `SHORT`                     | `smallint`                  |
| `SMALLINT`                  | `smallint`                  |
| `STRUCT<>`                  | `object{}`                  |
| `TIMESTAMP`                 | `datetime_tz`               |
| `TIMESTAMP_NTZ`             | `datetime`                  |
| `TINYINT`                   | `smallint`                  |
| `VARCHAR`                   | `string`                    |