Connection

Description

Configuration

NAME	TYPE	DESCRIPTION	REQUIRED	READONLY
auth_mode	string	How to authenticate with AWS. Defaults to Access Key and Secret	true	false
aws_access_key_id	string	Access Key ID with read/write access to a bucket. (required if `AuthMode` is “aws_access_key_and_secret”)	false	false
aws_secret_access_key	string	(required if `AuthMode` is “aws_access_key_and_secret”)	false	false
aws_user	string		false	false
iam_role_arn	string	(required if `AuthMode` is “iam_role”)	false	false
external_id	string	External ID for the IAM role	false	false
s3_bucket_region	string		true	false
s3_bucket_name	string	Bucket name (folder optional); ex: s3://polytomic/dataset	true	false
is_single_table	boolean	Treat the files as a single table. (required if `is_single_table` is “true”)	false	false
single_table_name	string	(required if `IsSingleTable` is “true”)	false	false
single_table_file_format	string	(required if `IsSingleTable` is “true”)	false	false
skip_lines	integer	Skip first N lines of each CSV file.	false	false

Example

1 {
2   "name": "s3 connection",
3   "type": "s3",
4   "configuration": {
5     "auth_mode": "access_key_and_secret",
6     "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
7     "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
8     "aws_user": "",
9     "external_id": "",
10     "iam_role_arn": "",
11     "is_single_table": false,
12     "s3_bucket_name": "s3://polytomic/dataset",
13     "s3_bucket_region": "us-east-1",
14     "single_table_file_format": "",
15     "single_table_name": "collection",
16     "skip_lines": 0
17   }
18 }

Model Sync

Source

N/A

Target

Configuration

NAME	TYPE	DESCRIPTION	REQUIRED	READONLY
format	string	Output file encoding	false	false

Example

1 {
2   ...
3   "target": {
4       "configuration": {
5         "format": "csv"
6     }
7   }
8 }

Bulk Sync

Source

N/A

Destination

Configuration

NAME	TYPE	DESCRIPTION	REQUIRED	READONLY
format	string	Output file encoding	false	false
subfolder	string	Subfolder to write to (optional)	false	false

Example

1 {
2   ...
3   "destination_configuration": {
4     "format": "csv",
5     "subfolder": "reports"
6   }
7 }

Getting started

Configuring your connections

Code examples

S3

Connection

Description

Configuration

Example

Model Sync

Source

Target

Configuration

Example

Bulk Sync

Source

Destination

Configuration

Example

1	{
2	"name": "s3 connection",
3	"type": "s3",
4	"configuration": {
5	"auth_mode": "access_key_and_secret",
6	"aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
7	"aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
8	"aws_user": "",
9	"external_id": "",
10	"iam_role_arn": "",
11	"is_single_table": false,
12	"s3_bucket_name": "s3://polytomic/dataset",
13	"s3_bucket_region": "us-east-1",
14	"single_table_file_format": "",
15	"single_table_name": "collection",
16	"skip_lines": 0
17	}
18	}

1	{
2	...
3	"destination_configuration": {
4	"format": "csv",
5	"subfolder": "reports"
6	}
7	}