Cloudflare R2 | Polytomic Docs

Supports:

✅ Models
✅ Model sync destination
✅ Bulk sync source
✅ Bulk sync destination

Connection

Configuration

Name	Type	Description	Required
`account_id`	`string`	Account ID	`true`
`aws_access_key_id`	`string`	Access Key ID with read/write access to a bucket.	`true`
`aws_secret_access_key`	`string`	Secret Access Key	`true`
`bucket_name`	`string`	Bucket name (folder optional); ex: polytomic/dataset	`true`
`directory_glob_pattern`	`string`	Tables glob path	`false`
`is_directory_snapshot`	`boolean`	Multi-directory multi-table	`false`
`is_single_table`	`boolean`	Files are time-based snapshots Treat the files as a single table.	`false`
`single_table_file_format`	`string`	File format Accepted values: `csv`, `json`, `parquet`	`false`
`single_table_name`	`string`	Collection name	`false`
`skip_lines`	`integer`	Skip first lines Skip first N lines of each CSV file.	`false`

Example

1 {
2   "name": "Cloudflare R2 connection",
3   "type": "cloudflare_r2",
4   "configuration": {
5     "account_id": "",
6     "aws_access_key_id": "AKIAIOSFODNN7EXAMPLE",
7     "aws_secret_access_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
8     "bucket_name": "polytomic/dataset",
9     "directory_glob_pattern": "",
10     "is_directory_snapshot": false,
11     "is_single_table": false,
12     "single_table_file_format": "csv",
13     "single_table_name": "collection",
14     "skip_lines": 0
15   }
16 }

Model Sync

Source

Configuration

Name	Type	Description	Required
`file_format`	`string`	File format Accepted values: `csv`, `json`, `parquet`	`false`
`key`	`string`	Object key The key of the object in the bucket to read from.	`false`
`model_from`	`string`	Files The model is generated from a single file or a multi-file archive. Accepted values: `single_file`, `multi_file_archive`	`true`
`skip_lines`	`integer`	Skip first lines Skip first N lines of each CSV file.	`false`
`subfolder`	`string`	Subfolder to read files from from (optional)	`false`

Example

1 {
2   ...
3   "configuration": {
4     "file_format": "",
5     "key": "",
6     "model_from": "",
7     "skip_lines": 0,
8     "subfolder": ""
9   }
10 }

Target

Cloudflare R2 connections may be used as the destination in a model sync.

All targets

Configuration

Name	Type	Description	Required
`format`	`string`	Output format Output file encoding. Accepted values: `csv`, `json-doc`, `json`, `parquet`	`false`

Example

1 {
2   ...
3   "target": {
4       "configuration": {
5         "format": "csv"
6     }
7   }
8 }

Bulk Sync

Destination

Configuration

Name	Type	Description	Required
`format`	`string`	Output file encoding	`false`
`subfolder`	`string`	Subfolder to write to (optional)	`false`

Example

1 {
2   ...
3   "destination_configuration": {
4     "format": "csv",
5     "subfolder": "reports"
6   }
7 }