Connection

Configuration

NAMETYPEDESCRIPTIONREQUIREDREADONLY
project_idstringfalsetrue
client_emailstringfalsetrue
service_accountstringtruefalse
locationstringRegion or multi-region for query operationsfalsefalse
override_project_idstringOverride service key’s project ID for cross-account accessfalsefalse
structured_values_as_jsonbooleanfalsefalse

Example

1{
2 "name": "bigquery connection",
3 "type": "bigquery",
4 "configuration": {
5 "location": "us-east1",
6 "override_project_id": "",
7 "service_account": "",
8 "structured_values_as_json": false
9 }
10}

Model Sync

Source

Configuration

NAMETYPEDESCRIPTIONREQUIREDREADONLY
querystringfalsefalse
datasetstringDatasetfalsefalse
tablestringTablefalsefalse

Example

1{
2 ...
3 "configuration": {
4 "dataset": "nyctaxi",
5 "query": "SELECT * from sampledata.nyctaxi.taxis",
6 "table": "taxis"
7 }
8}

Bulk Sync

Destination

Configuration

NAMETYPEDESCRIPTIONREQUIREDREADONLY
advancedobjectfalsefalse
mirror_schemasbooleanMirror schemasfalsefalse
datasetstringOutput datasetfalsefalse

Example

1{
2 ...
3 "destination_configuration": {
4 "advanced": {
5 "expire_tables_after_inactivity": false,
6 "hard_deletes": false,
7 "table_expiry_days": 0,
8 "table_prefix": "",
9 "truncate_existing": false
10 },
11 "dataset": "dataset",
12 "mirror_schemas": false
13 }
14}

Type handling

Destination types

POLYTOMIC TYPEGOOGLE BIGQUERY TYPE
array<>JSON
bigintINTEGER
booleanBOOLEAN
dateDATE
datetimeTIMESTAMP
decimal(precision, scale)BIGNUMERIC(precision,scale)
doubleFLOAT
intINTEGER
jsonJSON
jsonarrayJSON
numberBIGNUMERIC
object{}JSON
singleFLOAT
smallintINTEGER
stringSTRING
timeTIME

Source types

GOOGLE BIGQUERY TYPEPOLYTOMIC TYPE
ARRAY<>array<>
BIGNUMERICnumber
BIGNUMERIC(precision, scale)decimal(precision, scale)
DATEdate
FLOATdouble
INTEGERint
NUMERICnumber
NUMERIC(precision, scale)decimal(precision, scale)
RECORDjson
RECORD<>object{}
TIMEtime
TIMESTAMPdatetime