Redshift

Supports:

✅ Models
✅ Model sync destination
✅ Bulk sync source
✅ Bulk sync destination

Connection

Configuration

Name	Type	Description	Required
`auth_mode`	`string`	Authentication Method How to authenticate with AWS. Defaults to Access Key and Secret. Accepted values: `access_key_and_secret` ↓, `iam_role` ↓	`true`
`database`	`string`	Database	`true`
`hostname`	`string`	Hostname	`true`
`password`	`string`	Password	`true`
`port`	`integer`	Port	`true`
`s3_bucket_name`	`string`	S3 Bucket Name (destinations only) Name of bucket used for staging data load files	`false`
`s3_bucket_region`	`string`	S3 Bucket Region (destinations only) Region of bucket. Note: must match region of redshift server	`false`
`ssh`	`boolean`	Connect over SSH tunnel ↓	`false`
`use_bulk_sync_staging_schema`	`boolean`	Use custom bulk sync staging schema ↓	`false`
`username`	`string`	Username	`true`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

Name	Type	Description	Required
`aws_access_key_id`	`string`	AWS Access Key ID (destinations only) Access Key ID with read/write access to a bucket. More info: https://docs.polytomic.com/docs/redshift	`false`
`aws_secret_access_key`	`string`	AWS Secret Access Key (destinations only)	`false`

When `auth_mode` is `iam_role`

Name	Type	Description	Required
`iam_role_arn`	`string`	IAM Role ARN	`true`

`use_bulk_sync_staging_schema`

When `use_bulk_sync_staging_schema` is `true`

Name	Type	Description	Required
`bulk_sync_staging_schema`	`string`	Staging schema name	`false`

`ssh`

When `ssh` is `true`

Name	Type	Description	Required
`ssh_host`	`string`	SSH host	`false`
`ssh_port`	`integer`	SSH port	`false`
`ssh_private_key`	`string`	Private key	`false`
`ssh_user`	`string`	SSH user	`false`

Example

1 {
2   "name": "Redshift connection",
3   "type": "redshift",
4   "configuration": {
5     "auth_mode": "access_key_and_secret",
6     "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
7     "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
8     "database": "mydb",
9     "hostname": "mycluster.us-west-2.redshift.amazonaws.com",
10     "password": "password",
11     "port": 5439,
12     "s3_bucket_name": "my-bucket",
13     "s3_bucket_region": "us-west-2",
14     "ssh": false,
15     "use_bulk_sync_staging_schema": false,
16     "username": "redshift_user"
17   }
18 }

Read-only properties

Name	Type	Description	Required

`auth_mode`

When `auth_mode` is `access_key_and_secret`

Name	Type	Description	Required
`aws_user`	`string`	User ARN	`false`

When `auth_mode` is `iam_role`

Name	Type	Description	Required
`external_id`	`string`	External ID for the IAM role	`false`

Model Sync

Source

Configuration

Name	Type	Description	Required
`query`	`string`		`false`
`schema`	`string`	Schema	`false`
`table`	`string`	Table	`false`
`view`	`string`	View	`false`

Example

1 {
2   ...
3   "configuration": {
4     "query": "SELECT * FROM sampledata.users",
5     "schema": "sampledata",
6     "table": "users",
7     "view": "active_users"
8   }
9 }

Target

Redshift connections may be used as the destination in a model sync.

All targets

Configuration

Name	Type	Description	Required
`created_column`	`string`	’Created at’ timestamp column	`false`
`preserve_table_on_resync`	`boolean`	Preserve destination table when resyncing	`false`
`updated_column`	`string`	’Updated at’ timestamp column	`false`
`write_record_timestamps`	`boolean`	Write row timestamp metadata	`false`

Example

1 {
2   ...
3   "target": {
4       "configuration": {
5         "created_column": "",
6         "preserve_table_on_resync": false,
7         "updated_column": "",
8         "write_record_timestamps": false
9     }
10   }
11 }

Target creation

Redshift connections may be used to create a new target for a model sync. The following parameters are required to create a new target:

NAME	DESCRIPTION	ENUM
name	Table name	false

Bulk Sync

Source

Redshift connections may be used as a bulk sync source. No additional configuration options are required.

Destination

Configuration

Name	Type	Description	Required
`advanced`	`object`		`false`
`mirror_schemas`	`boolean`	Mirror schemas	`false`
`schema`	`string`	Output schema	`false`

Example

1 {
2   ...
3   "destination_configuration": {
4     "advanced": {
5         "empty_strings_null": false,
6         "hard_deletes": false,
7         "table_prefix": "",
8         "truncate_existing": false
9     },
10     "mirror_schemas": false,
11     "schema": "schema"
12   }
13 }

Type handling

Destination types

POLYTOMIC TYPE	REDSHIFT TYPE
`array<>`	`SUPER`
`bigint`	`BIGINT`
`boolean`	`BOOL`
`date`	`DATE`
`datetime`	`TIMESTAMP`
`decimal(precision, scale)`	`NUMERIC(precision,scale)`
`double`	`FLOAT8`
`int`	`INTEGER`
`json`	`SUPER`
`jsonarray`	`SUPER`
`number`	`NUMERIC(38,18)`
`object{}`	`SUPER`
`single`	`FLOAT4`
`smallint`	`SMALLINT`
`string`	`VARCHAR(MAX)`
`time`	`VARCHAR(255)`

Source types

REDSHIFT TYPE	POLYTOMIC TYPE
`4000`	`json`
`BIGINT`	`bigint`
`BOOL`	`boolean`
`BOOLEAN`	`boolean`
`BPCHAR`	`string`
`CHAR`	`string`
`CHARACTER`	`string`
`CHARACTER VARYING`	`string`
`DATE`	`date`
`DECIMAL`	`number`
`DECIMAL(precision, scale)`	`decimal(precision, scale)`
`DOUBLE PRECISION`	`double`
`FLOAT`	`double`
`FLOAT4`	`single`
`FLOAT8`	`double`
`INT`	`int`
`INT2`	`smallint`
`INT4`	`int`
`INT8`	`bigint`
`INTEGER`	`int`
`NCHAR`	`string`
`NUMERIC`	`number`
`NUMERIC(precision, scale)`	`decimal(precision, scale)`
`NVARCHAR`	`string`
`REAL`	`single`
`SMALLINT`	`smallint`
`STRING`	`string`
`TEXT`	`string`
`TIME`	`time`
`TIME WITH TIME ZONE`	`time`
`TIME WITHOUT TIME ZONE`	`time`
`TIMESTAMP`	`datetime`
`TIMESTAMP WITH TIME ZONE`	`datetime_tz`
`TIMESTAMP WITHOUT TIME ZONE`	`datetime`
`TIMESTAMPTZ`	`datetime_tz`
`TIMETZ`	`time`
`VARCHAR`	`string`

1	{
2	"name": "Redshift connection",
3	"type": "redshift",
4	"configuration": {
5	"auth_mode": "access_key_and_secret",
6	"aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
7	"aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
8	"database": "mydb",
9	"hostname": "mycluster.us-west-2.redshift.amazonaws.com",
10	"password": "password",
11	"port": 5439,
12	"s3_bucket_name": "my-bucket",
13	"s3_bucket_region": "us-west-2",
14	"ssh": false,
15	"use_bulk_sync_staging_schema": false,
16	"username": "redshift_user"
17	}
18	}

1	{
2	...
3	"configuration": {
4	"query": "SELECT * FROM sampledata.users",
5	"schema": "sampledata",
6	"table": "users",
7	"view": "active_users"
8	}
9	}

1	{
2	...
3	"target": {
4	"configuration": {
5	"created_column": "",
6	"preserve_table_on_resync": false,
7	"updated_column": "",
8	"write_record_timestamps": false
9	}
10	}
11	}

1	{
2	...
3	"destination_configuration": {
4	"advanced": {
5	"empty_strings_null": false,
6	"hard_deletes": false,
7	"table_prefix": "",
8	"truncate_existing": false
9	},
10	"mirror_schemas": false,
11	"schema": "schema"
12	}
13	}

Connection

Configuration

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

use_bulk_sync_staging_schema

When use_bulk_sync_staging_schema is true

ssh

When ssh is true

Example

Read-only properties

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

Model Sync

Source

Configuration

Example

Target

All targets

Configuration

Example

Target creation

Bulk Sync

Source

Destination

Configuration

Example

Type handling

Destination types

Source types

Connection

Configuration

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

use_bulk_sync_staging_schema

When use_bulk_sync_staging_schema is true

ssh

When ssh is true

Example

Read-only properties

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

Model Sync

Source

Configuration

Example

Target

All targets

Configuration

Example

Target creation

Bulk Sync

Source

Destination

Configuration

Example

Type handling

Destination types

Source types

`auth_mode`

When `auth_mode` is `access_key_and_secret`

When `auth_mode` is `iam_role`

`use_bulk_sync_staging_schema`

When `use_bulk_sync_staging_schema` is `true`

`ssh`

When `ssh` is `true`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

When `auth_mode` is `iam_role`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

When `auth_mode` is `iam_role`

`use_bulk_sync_staging_schema`

When `use_bulk_sync_staging_schema` is `true`

`ssh`

When `ssh` is `true`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

When `auth_mode` is `iam_role`