Create Bulk Sync | Polytomic Docs

Create a new Bulk Sync from a source to a destination (data warehouse, database, or cloud storage bucket like S3).

Bulk Syncs are used for the ELT pattern (Extract, Load, and Transform), where you want to sync un-transformed data to your data warehouses, databases, or cloud storage buckets like S3.

All of the functionality described in the product documentation is configurable via the API.

Sample code examples:

Connection specific configuration

The destination_configuration is integration-specific configuration for the selected bulk sync destination. This includes settings such as the output schema and is required when creating a new sync.

The source_configuration is optional. It allows configuration for how Polytomic reads data from the source connection. This will not be available for integrations that do not support additional configuration.

Consult the connection configurations to see configurations for particular integrations (for example, here is the available source configuration for the PostgreSQL bulk sync source).

Create a new Bulk Sync from a source to a destination (data warehouse, database, or cloud storage bucket like S3). Bulk Syncs are used for the ELT pattern (Extract, Load, and Transform), where you want to sync un-transformed data to your data warehouses, databases, or cloud storage buckets like S3. All of the functionality described in [the product documentation](https://docs.polytomic.com/docs/bulk-syncs) is configurable via the API. Sample code examples: - [Bulk sync (ELT) from Salesforce to S3](https://apidocs.polytomic.com/guides/code-examples/bulk-sync-elt-from-salesforce-to-s-3) - [Bulk sync (ELT) from Salesforce to Snowflake](https://apidocs.polytomic.com/guides/code-examples/bulk-sync-elt-from-salesforce-to-snowflake) - [Bulk sync (ELT) from HubSpot to PostgreSQL](https://apidocs.polytomic.com/guides/code-examples/bulk-sync-elt-from-hub-spot-to-postgre-sql) ## Connection specific configuration The `destination_configuration` is integration-specific configuration for the selected bulk sync destination. This includes settings such as the output schema and is required when creating a new sync. The `source_configuration` is optional. It allows configuration for how Polytomic reads data from the source connection. This will not be available for integrations that do not support additional configuration. Consult the [connection configurations](https://apidocs.polytomic.com/2024-02-08/guides/configuring-your-connections/overview) to see configurations for particular integrations (for example, [here](https://apidocs.polytomic.com/2024-02-08/guides/configuring-your-connections/connections/postgre-sql#source-1) is the available source configuration for the PostgreSQL bulk sync source).

Authentication

AuthorizationBearer

Bearer authentication of the form Bearer <token>, where token is your auth token.

Request

This endpoint expects an object.

destination_configurationmap from strings to anyRequired

destination_connection_idstringRequiredformat: "uuid"

namestringRequired

scheduleobjectRequired

source_connection_idstringRequiredformat: "uuid"

activebooleanOptional

automatically_add_new_fieldsenumOptional

Allowed values:

automatically_add_new_objectsenumOptional

Allowed values:

data_cutoff_timestampdatetimeOptional

disable_record_timestampsbooleanOptionalDefaults to false

discoverbooleanOptional

DEPRECATED: Use automatically_add_new_objects/automatically_add_new_fields instead

modeenumOptional

Allowed values:

normalize_namesenumOptional

Allowed values:

organization_idstringOptionalformat: "uuid"

policieslist of stringsOptional

schemaslist of strings or objectsOptional

List of schemas to sync; if omitted, all schemas will be selected for syncing.

source_configurationmap from strings to anyOptional

Response

dataobject or null

Errors

Create a new Bulk Sync from a source to a destination (data warehouse, database, or cloud storage bucket like S3).

Bulk Syncs are used for the ELT pattern (Extract, Load, and Transform), where you want to sync un-transformed data to your data warehouses, databases, or cloud storage buckets like S3.

All of the functionality described in the product documentation is configurable via the API.

Sample code examples:

Connection specific configuration

Consult the connection configurations to see configurations for particular integrations (for example, here is the available source configuration for the PostgreSQL bulk sync source).

1	from polytomic import BulkSchedule, Polytomic
2
3	client = Polytomic(
4	version="YOUR_VERSION",
5	token="YOUR_TOKEN",
6	)
7	client.bulk_sync.create(
8	destination_connection_id="248df4b7-aa70-47b8-a036-33ac447e668d",
9	name="My Bulk Sync",
10	schedule=BulkSchedule(
11	frequency="manual",
12	),
13	source_connection_id="248df4b7-aa70-47b8-a036-33ac447e668d",
14	)

1	{
2	"data": {
3	"active": true,
4	"automatically_add_new_fields": "all",
5	"automatically_add_new_objects": "all",
6	"created_at": "2024-01-15T09:30:00Z",
7	"created_by": {
8	"id": "248df4b7-aa70-47b8-a036-33ac447e668d",
9	"name": "name",
10	"type": "type"
11	},
12	"data_cutoff_timestamp": "2023-04-25T12:00:00Z",
13	"destination_configuration": {
14	"schema": "my_schema"
15	},
16	"destination_connection_id": "248df4b7-aa70-47b8-a036-33ac447e668d",
17	"disable_record_timestamps": false,
18	"id": "248df4b7-aa70-47b8-a036-33ac447e668d",
19	"mode": "snapshot",
20	"name": "My Bulk Sync",
21	"normalize_names": "enabled",
22	"organization_id": "248df4b7-aa70-47b8-a036-33ac447e668d",
23	"policies": [
24	"248df4b7-aa70-47b8-a036-33ac447e668d"
25	],
26	"schedule": {
27	"frequency": "manual",
28	"day_of_month": "1",
29	"day_of_week": "monday",
30	"hour": "0",
31	"minute": "0",
32	"month": "1"
33	},
34	"source_configuration": {
35	"replication_slot": "slot"
36	},
37	"source_connection_id": "248df4b7-aa70-47b8-a036-33ac447e668d",
38	"updated_at": "2024-01-15T09:30:00Z",
39	"updated_by": {
40	"id": "248df4b7-aa70-47b8-a036-33ac447e668d",
41	"name": "name",
42	"type": "type"
43	},
44	"discover": true
45	}
46	}

Connection specific configuration

Authentication

Headers

Request

Response

Errors

Connection specific configuration