Code examples

Bulk sync (ELT) from Salesforce to Snowflake

UI demo video

Watch the same steps in the Polytomic admin console in this demo video.

Code

Set your Polytomic API key as an environment variable:

$export POLYTOMIC_API_KEY=YOUR-API-TOKEN

This example covers three steps:

  1. Create a Salesforce Connection.
  2. Create a Snowflake Connection.
  3. Bulk Sync data from Salesforce into Snowflake.

1. Create a Salesforce connection

The following request creates a Salesforce Connection. See the Salesforce connection configuration for the required fields.

$curl --request POST  \
>     --url https://app.polytomic.com/api/connections \
>     --header "accept: application/json"  \
>     --header "content-type: application/json"  \
>     --header "X-Polytomic-Version: 2024-02-08"   \
>     --header "Authorization: Bearer ${POLYTOMIC_API_KEY}"  \
>     -d '{"name": "Salesforce Connection","type": "salesforce", "configuration": {"domain": "https://example.my.salesforce.com"}}'

Salesforce Connections authenticate with OAuth. Open the URL returned in the auth_url field of the response to complete the flow.

OAuth redirection

By default, the API expects auth_url to open in a new browser window. Set the optional redirect_url parameter in the request body to change the redirect target.

2. Create a Snowflake connection

The Snowflake instance must be reachable from Polytomic over the network. For details, see the Snowflake connection guide.

$     curl --request POST \
>     --url https://app.polytomic.com/api/connections \
>     --header "accept: application/json" \
>     --header "content-type: application/json" \
>     --header "X-Polytomic-Version: 2024-02-08"  \
>     --header "Authorization: Bearer ${POLYTOMIC_API_KEY}" \
>     -d '{
>     "name": "Snowflake Connection",
>     "type": "snowflake",
>     "configuration": {
>        "account": "uc193736182",
>        "username": "user",
>        "password": "secret",
>        "dbname": "database"
>        }
>    }'

3. Create a Bulk Sync

Before creating the sync, poll the Get Schema Status endpoint for your Salesforce Connection ID until cache_status is cached. This confirms Polytomic has read the full Salesforce schema. Caching takes a few seconds after Step 1, and you should wait for it after connecting Polytomic to any SaaS platform.

ℹ️ Retry while schemas cache

Until the cache is populated, Bulk Sync creation returns:

1{"status":400,"message":"No schemas found. This can mean that the Polytomic has not cached any schemas yet. Please try again.","metadata":null}

This request is safe to retry until it returns a 2xx response.

Once the schema is cached, the following request creates a Bulk Sync that replicates the Salesforce Account and Contact objects into Snowflake. Omit schemas to sync every object on the source:

$curl --request POST \
>     --url https://app.polytomic.com/api/bulk/syncs \
>     --header "accept: application/json" \
>     --header "content-type: application/json" \
>     --header "X-Polytomic-Version: 2024-02-08" \
>     --header "Authorization: Bearer ${POLYTOMIC_API_KEY}" \
>     -d '{
>          "name": "Salesforce Bulk Sync",
>          "source_connection_id": "YOUR-SALESFORCE-CONNECTION-ID",
>          "destination_connection_id": "YOUR-SNOWFLAKE-CONNECTION-ID",
>          "mode": "replicate",
>          "schedule": {
>              "frequency": "continuous"
>          },
>          "destination_configuration": {
>               "schema": "SALESFORCE_CONNECTION"
>          },
>          "schemas": ["Account", "Contact"]
>    	}'