S3

Supports:

• ✅ Models
• ✅ Model sync destination
• ✅ Bulk sync source
• ✅ Bulk sync destination

Connection

Configuration

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

enable_event_notifications

When enable_event_notifications is true

is_single_table

When is_single_table is true

When is_single_table is true AND is_directory_snapshot is false

When is_single_table is true AND is_directory_snapshot is true

is_directory_snapshot

When is_directory_snapshot is true

single_table_file_format

When single_table_file_format is csv OR single_table_file_formats contains csv

Example

1
{
2
  "name": "S3 connection",
3
  "type": "s3",
4
  "configuration": {
5
    "auth_mode": "access_key_and_secret",
6
    "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
7
    "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
8
    "csv_has_headers": false,
9
    "enable_event_notifications": false,
10
    "is_single_table": false,
11
    "s3_bucket_name": "s3://polytomic/dataset",
12
    "s3_bucket_region": "us-east-1"
13
  }
14
}

Read-only properties

auth_mode

When auth_mode is access_key_and_secret

When auth_mode is iam_role

Model Sync

Source

Configuration

Example

1
{
2
  ...
3
  "configuration": {
4
    "file_format": "",
5
    "key": "",
6
    "model_from": "",
7
    "skip_lines": 0,
8
    "subfolder": ""
9
  }
10
}

Target

S3 connections may be used as the destination in a model sync.

All targets

Configuration

Example

1
{
2
  ...
3
  "target": {
4
      "configuration": {
5
        "format": "csv"
6
    }
7
  }
8
}

Bulk Sync

Source

S3 connections may be used as a bulk sync source. No additional configuration options are required.

Destination

Configuration

Example

1
{
2
  ...
3
  "destination_configuration": {
4
    "format": "csv",
5
    "subfolder": "reports"
6
  }
7
}