Google Cloud Storage | Polytomic Docs

Supports:

✅ Models
✅ Model sync destination
✅ Bulk sync source
✅ Bulk sync destination

Connection

Configuration

Name	Type	Description	Required
`bucket`	`string`	Bucket	`true`
`directory_glob_pattern`	`string`	Tables glob path	`false`
`is_directory_snapshot`	`boolean`	Multi-directory multi-table	`false`
`is_single_table`	`boolean`	Files are time-based snapshots Treat the files as a single table.	`false`
`service_account`	`string`	Service account key	`true`
`single_table_file_format`	`string`	File format Accepted values: `csv`, `json`, `parquet`	`false`
`single_table_name`	`string`	Collection name	`false`
`skip_lines`	`integer`	Skip first lines Skip first N lines of each CSV file.	`false`

Example

1 {
2   "name": "Google Cloud Storage connection",
3   "type": "gcs",
4   "configuration": {
5     "bucket": "my-bucket",
6     "directory_glob_pattern": "",
7     "is_directory_snapshot": false,
8     "is_single_table": false,
9     "service_account": "",
10     "single_table_file_format": "csv",
11     "single_table_name": "collection",
12     "skip_lines": 0
13   }
14 }

Read-only properties

Name	Type	Description	Required
`client_email`	`string`	Service account identity	`false`
`project_id`	`string`	Service account project ID	`false`

Model Sync

Source

Configuration

Name	Type	Description	Required
`file_format`	`string`	File format Accepted values: `csv`, `json`, `parquet`	`false`
`key`	`string`	Object key The key of the object in the bucket to read from.	`false`
`model_from`	`string`	Files The model is generated from a single file or a multi-file archive. Accepted values: `single_file`, `multi_file_archive`	`true`
`skip_lines`	`integer`	Skip first lines Skip first N lines of each CSV file.	`false`
`subfolder`	`string`	Subfolder to read files from from (optional)	`false`

Example

1 {
2   ...
3   "configuration": {
4     "file_format": "",
5     "key": "",
6     "model_from": "",
7     "skip_lines": 0,
8     "subfolder": ""
9   }
10 }

Target

Google Cloud Storage connections may be used as the destination in a model sync.

All targets

Configuration

Name	Type	Description	Required
`format`	`string`	Output format Output file encoding. Accepted values: `csv`, `json-doc`, `json`, `parquet`	`false`

Example

1 {
2   ...
3   "target": {
4       "configuration": {
5         "format": "csv"
6     }
7   }
8 }

Bulk Sync

Destination

Configuration

Name	Type	Description	Required
`format`	`string`	Output file encoding	`false`
`subfolder`	`string`	Subfolder to write to (optional)	`false`

Example

1 {
2   ...
3   "destination_configuration": {
4     "format": "csv",
5     "subfolder": "reports"
6   }
7 }