AWS Athena

Supports:

✅ Models
✅ Model sync destination
✅ Bulk sync source
✅ Bulk sync destination

Connection

Configuration

Name	Type	Description	Required
`auth_mode`	`string`	Authentication Method How to authenticate with AWS. Defaults to Access Key and Secret. Accepted values: `access_key_and_secret` ↓, `iam_role` ↓	`true`
`outputbucket`	`string`	AWS S3 output bucket A pre-existing bucket (folder optional) that AWS can use to store query results. ex: s3://polytomic-athena-results/customer-dataset	`true`
`region`	`string`	AWS region	`true`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

Name	Type	Description	Required
`access_id`	`string`	AWS Access ID	`false`
`secret_access_key`	`string`	AWS Secret Access Key	`false`

When `auth_mode` is `iam_role`

Name	Type	Description	Required
`external_id`	`string`	External ID for the IAM role	`false`
`iam_role_arn`	`string`	IAM Role ARN	`true`
`tags`	`object`	Tags Additional tags to apply during role assumption	`false`

Example

1 {
2   "name": "AWS Athena connection",
3   "type": "awsathena",
4   "configuration": {
5     "access_id": "AKIAIOSFODNN7EXAMPLE",
6     "auth_mode": "access_key_and_secret",
7     "outputbucket": "s3://polytomic-athena-results/customer-dataset",
8     "region": "us-east-1",
9     "secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
10   }
11 }

Read-only properties

Name	Type	Description	Required

`auth_mode`

When `auth_mode` is `access_key_and_secret`

Name	Type	Description	Required
`aws_user`	`string`	User ARN	`false`

Model Sync

Source

Configuration

Name	Type	Description	Required
`catalog`	`string`	Catalog	`false`
`database`	`string`	Database	`false`
`query`	`string`		`false`
`table`	`string`	Table	`false`

Example

1 {
2   ...
3   "configuration": {
4     "catalog": "awsdatacatalog",
5     "database": "default",
6     "query": "SELECT * FROM \"webdata\".\"impressions\"",
7     "table": "users"
8   }
9 }

Target

AWS Athena connections may be used as the destination in a model sync.

All targets

Configuration

Name	Type	Description	Required
`created_column`	`string`	’Created at’ timestamp column	`false`
`preserve_table_on_resync`	`boolean`	Preserve destination table when resyncing	`false`
`updated_column`	`string`	’Updated at’ timestamp column	`false`
`write_record_timestamps`	`boolean`	Write row timestamp metadata	`false`

Example

1 {
2   ...
3   "target": {
4       "configuration": {
5         "created_column": "",
6         "preserve_table_on_resync": false,
7         "updated_column": "",
8         "write_record_timestamps": false
9     }
10   }
11 }

Target creation

AWS Athena connections may be used to create a new target for a model sync. The following parameters are required to create a new target:

NAME	DESCRIPTION	ENUM
name	Table name	false

Bulk Sync

Source

AWS Athena connections may be used as a bulk sync source. No additional configuration options are required.

Destination

Configuration

Name	Type	Description	Required
`advanced`	`object`		`false`
`mirror_schemas`	`boolean`	Indicates whether schemas should be mirrored	`false`
`schema`	`string`	Output schema to write to	`false`

Example

1 {
2   ...
3   "destination_configuration": {
4     "advanced": {
5         "table_prefix": "",
6         "truncate_existing": false
7     },
8     "mirror_schemas": false,
9     "schema": "public"
10   }
11 }

Type handling

Destination types

POLYTOMIC TYPE	AWS ATHENA TYPE
`array<>`	`string`
`bigint`	`string`
`boolean`	`boolean`
`date`	`timestamp`
`datetime`	`timestamp`
`decimal(precision, scale)`	`string`
`double`	`string`
`int`	`string`
`json`	`string`
`jsonarray`	`string`
`number`	`string`
`object{}`	`string`
`single`	`string`
`smallint`	`string`
`string`	`string`
`time`	`timestamp`

Source types

AWS ATHENA TYPE	POLYTOMIC TYPE
`ARRAY`	`jsonarray`
`BIGINT`	`bigint`
`BINARY`	`string`
`CHAR`	`string`
`DATE`	`date`
`DECIMAL`	`number`
`DECIMAL(precision, scale)`	`decimal(precision, scale)`
`DOUBLE`	`number`
`FLOAT`	`number`
`INT`	`int`
`INTEGER`	`int`
`INTERVAL DAY TO SECOND`	`string`
`INTERVAL YEAR TO MONTH`	`string`
`IPADDRESS`	`string`
`JSON`	`json`
`MAP`	`json`
`REAL`	`number`
`SMALLINT`	`smallint`
`STRING`	`string`
`TIME`	`time`
`TIME WITH TIME ZONE`	`time`
`TIMESTAMP`	`datetime`
`TIMESTAMP WITH TIME ZONE`	`datetime`
`TINYINT`	`smallint`
`VARBINARY`	`string`
`VARCHAR`	`string`

1	{
2	"name": "AWS Athena connection",
3	"type": "awsathena",
4	"configuration": {
5	"access_id": "AKIAIOSFODNN7EXAMPLE",
6	"auth_mode": "access_key_and_secret",
7	"outputbucket": "s3://polytomic-athena-results/customer-dataset",
8	"region": "us-east-1",
9	"secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
10	}
11	}

1	{
2	...
3	"configuration": {
4	"catalog": "awsdatacatalog",
5	"database": "default",
6	"query": "SELECT * FROM \"webdata\".\"impressions\"",
7	"table": "users"
8	}
9	}

1	{
2	...
3	"target": {
4	"configuration": {
5	"created_column": "",
6	"preserve_table_on_resync": false,
7	"updated_column": "",
8	"write_record_timestamps": false
9	}
10	}
11	}

1	{
2	...
3	"destination_configuration": {
4	"advanced": {
5	"table_prefix": "",
6	"truncate_existing": false
7	},
8	"mirror_schemas": false,
9	"schema": "public"
10	}
11	}

Connection

Configuration

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

Example

Read-only properties

auth_mode

When auth_mode is access_key_and_secret

Model Sync

Source

Configuration

Example

Target

All targets

Configuration

Example

Target creation

Bulk Sync

Source

Destination

Configuration

Example

Type handling

Destination types

Source types

Connection

Configuration

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

Example

Read-only properties

auth_mode

When auth_mode is access_key_and_secret

Model Sync

Source

Configuration

Example

Target

All targets

Configuration

Example

Target creation

Bulk Sync

Source

Destination

Configuration

Example

Type handling

Destination types

Source types

`auth_mode`

When `auth_mode` is `access_key_and_secret`

When `auth_mode` is `iam_role`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

`auth_mode`

When `auth_mode` is `access_key_and_secret`

When `auth_mode` is `iam_role`

`auth_mode`

When `auth_mode` is `access_key_and_secret`