Google BigQuery (2022-12-12)

Connection

Configuration

NAME	TYPE	DESCRIPTION	REQUIRED	READONLY
project_id	string		false	true
client_email	string		false	true
service_account	string		true	false
location	string	Region or multi-region for query operations	false	false
override_project_id	string	Override service key’s project ID for cross-account access	false	false
structured_values_as_json	boolean		false	false

Example

1 {
2   "name": "bigquery connection",
3   "type": "bigquery",
4   "configuration": {
5     "location": "us-east1",
6     "override_project_id": "",
7     "service_account": "",
8     "structured_values_as_json": false
9   }
10 }

Model Sync

Source

Configuration

NAME	TYPE	DESCRIPTION	REQUIRED	READONLY
query	string		false	false
dataset	string	Dataset	false	false
table	string	Table	false	false

Example

1 {
2   ...
3   "configuration": {
4     "dataset": "nyctaxi",
5     "query": "SELECT * from sampledata.nyctaxi.taxis",
6     "table": "taxis"
7   }
8 }

Bulk Sync

Destination

Configuration

NAME	TYPE	DESCRIPTION	REQUIRED	READONLY
advanced	object		false	false
dataset	string	Output dataset	false	false
mirror_schemas	boolean	Mirror schemas	false	false

Example

1 {
2   ...
3   "destination_configuration": {
4     "advanced": {
5         "expire_tables_after_inactivity": false,
6         "hard_deletes": false,
7         "table_expiry_days": 0,
8         "table_prefix": "",
9         "truncate_existing": false
10     },
11     "dataset": "dataset",
12     "mirror_schemas": false
13   }
14 }

Type handling

Destination types

POLYTOMIC TYPE	GOOGLE BIGQUERY TYPE
`array<>`	`JSON`
`bigint`	`INTEGER`
`boolean`	`BOOLEAN`
`date`	`DATE`
`datetime`	`TIMESTAMP`
`decimal(precision, scale)`	`BIGNUMERIC(precision,scale)`
`double`	`FLOAT`
`int`	`INTEGER`
`json`	`JSON`
`jsonarray`	`JSON`
`number`	`BIGNUMERIC`
`object{}`	`JSON`
`single`	`FLOAT`
`smallint`	`INTEGER`
`string`	`STRING`
`time`	`TIME`

Source types

GOOGLE BIGQUERY TYPE	POLYTOMIC TYPE
`ARRAY<>`	`array<>`
`BIGNUMERIC`	`number`
`BIGNUMERIC(precision, scale)`	`decimal(precision, scale)`
`DATE`	`date`
`FLOAT`	`double`
`INTEGER`	`int`
`NUMERIC`	`number`
`NUMERIC(precision, scale)`	`decimal(precision, scale)`
`RECORD`	`json`
`RECORD<>`	`object{}`
`TIME`	`time`
`TIMESTAMP`	`datetime`

1	{
2	"name": "bigquery connection",
3	"type": "bigquery",
4	"configuration": {
5	"location": "us-east1",
6	"override_project_id": "",
7	"service_account": "",
8	"structured_values_as_json": false
9	}
10	}

1	{
2	...
3	"configuration": {
4	"dataset": "nyctaxi",
5	"query": "SELECT * from sampledata.nyctaxi.taxis",
6	"table": "taxis"
7	}
8	}

1	{
2	...
3	"destination_configuration": {
4	"advanced": {
5	"expire_tables_after_inactivity": false,
6	"hard_deletes": false,
7	"table_expiry_days": 0,
8	"table_prefix": "",
9	"truncate_existing": false
10	},
11	"dataset": "dataset",
12	"mirror_schemas": false
13	}
14	}