S3 Compatible

Supports:

• ✅ Models
• ✅ Model sync destination
• ✅ Bulk sync source
• ✅ Bulk sync destination

Connection

Configuration

is_single_table

When is_single_table is true

When is_single_table is true AND is_directory_snapshot is false

When is_single_table is true AND is_directory_snapshot is true

is_directory_snapshot

When is_directory_snapshot is true

single_table_file_format

When single_table_file_format is csv OR single_table_file_formats contains csv

Example

{
  "name": "S3 Compatible connection",
  "type": "s3_compatible",
  "configuration": {
    "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
    "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
    "csv_has_headers": false,
    "endpoint": "https://s3.example.com",
    "is_single_table": false,
    "s3_bucket_name": "s3://polytomic/dataset",
    "s3_bucket_region": "us-east-1"
  }
}

Model Sync

Source

Configuration

Example

{
  ...
  "configuration": {
    "file_format": "",
    "key": "",
    "model_from": "",
    "skip_lines": 0,
    "subfolder": ""
  }
}

Target

S3 Compatible connections may be used as the destination in a model sync.

All targets

Configuration

Example

{
  ...
  "target": {
      "configuration": {
        "format": "csv"
    }
  }
}

Bulk Sync

Source

S3 Compatible connections may be used as a bulk sync source. No additional configuration options are required.

Destination

Configuration

Example

{
  ...
  "destination_configuration": {
    "advanced": {
        "write_incremental": false
    },
    "format": "csv",
    "subfolder": "reports"
  }
}