Skip to navigation
API ReferenceBulk Sync

Create Bulk Sync

Creates a new bulk sync.

Bulk syncs are used for the ELT pattern (Extract, Load, and Transform), where you want to sync un-transformed data to your data warehouses, databases, or cloud storage buckets like S3.

All of the functionality described in the product documentation is configurable via the API.

Sample code examples:

Connection specific configuration

The destination_configuration is integration-specific configuration for the selected bulk sync destination. This includes settings such as the output schema and is required when creating a new sync.

The source_configuration is optional. It allows configuration for how Polytomic reads data from the source connection. This will not be available for integrations that do not support additional configuration.

Consult the connection configurations to see configurations for particular integrations (for example, here is the available source configuration for the PostgreSQL bulk sync source).

Defaults and selection behavior

If schemas is omitted, the sync is created with all available source schemas selected. Pass schemas explicitly if you want the initial sync to include only a subset of tables or objects.

Schedule times are interpreted in UTC.

When omitted, automatic discovery defaults are conservative:

  • automatically_add_new_objects defaults to not enabling newly discovered source objects automatically.
  • automatically_add_new_fields defaults to enabling newly discovered fields on already selected objects.
  • normalize_names defaults to enabled.

Scheduling

To sync the two selective groups of objects at different frequencies, add one entry to additional_schedules with selective_mode set to incrementalFields and another with nonincrementalFields. Set default_schedule.frequency to manual to avoid an additional automatic run covering all selected objects. The default schedule does not accept a selective mode. Check capabilities.supports_selective_modes on GET /api/connections/{id}/bulksync/source for the source connection first.

Authentication

AuthorizationBearer
Bearer user API key
OR
AuthorizationBasic

Basic organization-scoped API key

Headers

X-Polytomic-VersionstringOptional

Request

This endpoint expects an object.
default_scheduleobjectRequired
The default bulk sync schedule. This schedule applies to all fields and schemas. Schedules are interpreted in UTC.
destination_configurationmap from strings to anyRequired

Destination-specific bulk sync configuration (e.g. output schema name, file format). The accepted keys depend on the destination connection type.

destination_connection_idstringRequiredformat: "uuid"
Unique identifier of the connection rows are written to
modeenumRequired

How source data is written to the destination. replicate keeps the destination in sync with the source; snapshot writes a timestamped copy per execution; append adds every delivered record, including full reads, without updating existing rows.

Allowed values:
namestringRequired

Human-readable name for the bulk sync

source_connection_idstringRequiredformat: "uuid"
Unique identifier of the connection rows are read from
activeboolean or nullOptional
Whether the sync is active. Inactive syncs do not run on their schedule but can still be triggered manually.
additional_scheduleslist of objects or nullOptional
Additional bulk sync schedules. Schedule times are interpreted in UTC.
automatically_add_new_fieldsenumOptional
Controls whether Polytomic automatically enables newly discovered fields on existing objects. Valid values depend on the source backend.
Allowed values:
automatically_add_new_objectsenumOptional

Controls whether Polytomic automatically enables newly discovered source objects (tables). Valid values depend on the source backend.

Allowed values:
concurrency_limitinteger or nullOptional
Override the default concurrency limit for this sync.
data_cutoff_timestampdatetime or nullOptional
Global cutoff applied across schemas. Source records older than this timestamp are excluded from sync runs.
disable_record_timestampsboolean or nullOptionalDefaults to false
When true, Polytomic will not add its own timestamp columns to destination rows.
normalize_namesenumOptional

Controls whether destination object and field names are normalized (e.g. lowercased, special characters replaced). Valid values depend on the destination backend.

Allowed values:
organization_idstring or nullOptionalformat: "uuid"

Ignored for user and organization-scoped API keys, which always act in their own organization

policieslist of strings or nullOptional

Permissions policy IDs applied to the bulk sync. On update, a non-empty list replaces the bulk sync’s policies and an empty or omitted list leaves them unchanged.

resync_concurrency_limitinteger or nullOptional
Override the default resync concurrency limit for this sync.
schemaslist of strings or objects or nullOptional

List of schemas to sync; if omitted, all schemas will be selected for syncing.

source_configurationmap from strings to any or nullOptional

Source-specific bulk sync configuration (e.g. replication slot name, sync lookback). The accepted keys depend on the source connection type.

Response

OK
dataobjectOptional
The bulk sync

Errors

400
Bad Request Error
403
Forbidden Error
422
Unprocessable Entity Error
500
Internal Server Error