Redshift Serverless | Polytomic Docs

Supports:

✅ Models
✅ Model sync destination
✅ Bulk sync source
✅ Bulk sync destination

Connection

Configuration

Name	Type	Description	Required
`connection_method`	`string`	Connection method Accepted values: `data_api` ↓, `endpoint` ↓	`required`
`database`	`string`	Database	`required`
`iam_role_arn`	`string`	IAM role ARN	`required`
`region`	`string`	Region	`optional`
`s3_bucket_name`	`string`	S3 bucket name (destination/unload support only) (required if `use_unload` is “true”)	`optional`
`s3_bucket_region`	`string`	S3 bucket region (destination/unload support only) (required if `use_unload` is “true”)	`optional`
`use_bulk_sync_staging_schema`	`boolean`	Use custom bulk sync staging schema ↓	`optional`
`use_unload`	`boolean`	Read data using unload	`optional`
`workgroup`	`string`	Workgroup	`required`

Read-only property	Type	Description
`external_id`	`string`	External ID

`connection_method` = `data_api`

Name	Type	Description	Required
`override_endpoint`	`boolean`	Override Redshift Data API endpoint ↓	`optional`

{
  "name": "Redshift Serverless connection",
  "type": "redshiftserverless",
  "configuration": {
    "connection_method": "data_api",
    "database": "users",
    "iam_role_arn": "arn:aws:iam::012345678910:role/role",
    "override_endpoint": false,
    "region": "us-west-2",
    "s3_bucket_name": "my-bucket",
    "s3_bucket_region": "us-west-2",
    "use_bulk_sync_staging_schema": false,
    "use_unload": false,
    "workgroup": "default"
  }
}

`override_endpoint`

When override_endpoint is true:

Name	Type	Description	Required
`data_api_endpoint`	`string`	Redshift Data API endpoint Example: https://redshift-data.us-west-2.amazonaws.com	`optional`

{
  "name": "Redshift Serverless connection",
  "type": "redshiftserverless",
  "configuration": {
    "connection_method": "data_api",
    "data_api_endpoint": "",
    "database": "users",
    "iam_role_arn": "arn:aws:iam::012345678910:role/role",
    "override_endpoint": true,
    "region": "us-west-2",
    "s3_bucket_name": "my-bucket",
    "s3_bucket_region": "us-west-2",
    "use_bulk_sync_staging_schema": false,
    "use_unload": false,
    "workgroup": "default"
  }
}

`connection_method` = `endpoint`

Name	Type	Description	Required
`endpoint`	`string`	Redshift Serverless endpoint	`optional`

{
  "name": "Redshift Serverless connection",
  "type": "redshiftserverless",
  "configuration": {
    "connection_method": "endpoint",
    "database": "users",
    "endpoint": "acme.12345.us-west-2.redshift-serverless.amazonaws.com:5439",
    "iam_role_arn": "arn:aws:iam::012345678910:role/role",
    "region": "us-west-2",
    "s3_bucket_name": "my-bucket",
    "s3_bucket_region": "us-west-2",
    "use_bulk_sync_staging_schema": false,
    "use_unload": false,
    "workgroup": "default"
  }
}

`use_bulk_sync_staging_schema`

When use_bulk_sync_staging_schema is true:

Name	Type	Description	Required
`bulk_sync_staging_schema`	`string`	Staging schema name	`required`

{
  "name": "Redshift Serverless connection",
  "type": "redshiftserverless",
  "configuration": {
    "bulk_sync_staging_schema": "",
    "connection_method": "data_api",
    "database": "users",
    "iam_role_arn": "arn:aws:iam::012345678910:role/role",
    "override_endpoint": false,
    "region": "us-west-2",
    "s3_bucket_name": "my-bucket",
    "s3_bucket_region": "us-west-2",
    "use_bulk_sync_staging_schema": true,
    "use_unload": false,
    "workgroup": "default"
  }
}

Model Sync

Source

Configuration

Name	Type	Description	Required
`query`	`string`		`optional`
`schema`	`string`	Schema	`optional`
`table`	`string`	Table	`optional`
`view`	`string`	View	`optional`

Example

{
  ...
  "configuration": {
    "query": "SELECT * FROM sampledata.users",
    "schema": "sampledata",
    "table": "users",
    "view": "active_users"
  }
}

Target

Redshift Serverless connections may be used as the destination in a model sync.

All targets

Configuration

Name	Type	Description	Required
`created_column`	`string`	’Created at’ timestamp column	`optional`
`preserve_table_on_resync`	`boolean`	Preserve destination table when resyncing	`optional`
`updated_column`	`string`	’Updated at’ timestamp column	`optional`
`write_record_timestamps`	`boolean`	Write row timestamp metadata	`optional`

Example

{
  ...
  "target": {
      "configuration": {
        "created_column": "",
        "preserve_table_on_resync": false,
        "updated_column": "",
        "write_record_timestamps": false
    }
  }
}

Target creation

Redshift Serverless connections may be used to create a new target for a model sync. The following parameters are required to create a new target:

NAME	DESCRIPTION	ENUM
name	Table name	false

Bulk Sync

Source

Redshift Serverless connections may be used as a bulk sync source. No additional configuration options are required.

Destination

Configuration

Name	Type	Description	Required
`advanced`	`object`		`optional`
`mirror_schemas`	`boolean`	Mirror schemas	`optional`
`schema`	`string`	Output schema	`optional`

Advanced configuration (`advanced`)

Name	Type	Description	Required
`empty_strings_null`	`boolean`	Write empty strings as null	`optional`
`hard_deletes`	`boolean`	Propagate hard deletes instead of soft deletes	`optional`
`initial_execution`	`string`	On first sync	`optional`
`table_prefix`	`string`	Prefix table names with	`optional`
`truncate_existing`	`boolean`	Truncate destination table when source table does not have a primary key	`optional`

Example

{
  ...
  "destination_configuration": {
    "advanced": {
        "empty_strings_null": true,
        "hard_deletes": false,
        "initial_execution": "rebuild",
        "table_prefix": "",
        "truncate_existing": false
    },
    "mirror_schemas": false,
    "schema": "schema"
  }
}

Type handling

Destination types

POLYTOMIC TYPE	REDSHIFT SERVERLESS TYPE
`array<>`	`SUPER`
`bigint`	`BIGINT`
`boolean`	`BOOL`
`date`	`DATE`
`datetime`	`TIMESTAMP`
`decimal(precision, scale)`	`NUMERIC(precision,scale)`
`double`	`FLOAT8`
`int`	`INTEGER`
`json`	`SUPER`
`jsonarray`	`SUPER`
`number`	`NUMERIC(38,18)`
`object{}`	`SUPER`
`single`	`FLOAT4`
`smallint`	`SMALLINT`
`string`	`VARCHAR(MAX)`
`time`	`VARCHAR(255)`

Source types

REDSHIFT SERVERLESS TYPE	POLYTOMIC TYPE
`4000`	`json`
`BIGINT`	`bigint`
`BOOL`	`boolean`
`BOOLEAN`	`boolean`
`BPCHAR`	`string`
`CHAR`	`string`
`CHARACTER`	`string`
`CHARACTER VARYING`	`string`
`DATE`	`date`
`DECIMAL`	`number`
`DECIMAL(precision, scale)`	`decimal(precision, scale)`
`DOUBLE PRECISION`	`double`
`FLOAT`	`double`
`FLOAT4`	`single`
`FLOAT8`	`double`
`INT`	`int`
`INT2`	`smallint`
`INT4`	`int`
`INT8`	`bigint`
`INTEGER`	`int`
`NCHAR`	`string`
`NUMERIC`	`number`
`NUMERIC(precision, scale)`	`decimal(precision, scale)`
`NVARCHAR`	`string`
`REAL`	`single`
`SMALLINT`	`smallint`
`STRING`	`string`
`TEXT`	`string`
`TIME`	`time`
`TIME WITH TIME ZONE`	`time`
`TIME WITHOUT TIME ZONE`	`time`
`TIMESTAMP`	`datetime`
`TIMESTAMP WITH TIME ZONE`	`datetime_tz`
`TIMESTAMP WITHOUT TIME ZONE`	`datetime`
`TIMESTAMPTZ`	`datetime_tz`
`TIMETZ`	`time`
`VARCHAR`	`string`

Connection

Configuration

connection_method = data_api

override_endpoint

connection_method = endpoint

use_bulk_sync_staging_schema

Model Sync

Source

Configuration

Example

Target

All targets

Configuration

Example

Target creation

Bulk Sync

Source

Destination

Configuration

Advanced configuration (advanced)

Example

Type handling

Destination types

Source types

`connection_method` = `data_api`

`override_endpoint`

`connection_method` = `endpoint`

`use_bulk_sync_staging_schema`

Advanced configuration (`advanced`)