bq_extract>: Exporting Data from Google BigQuery

bq_extract> operator can be used to export data from Google BigQuery tables.

Copy

Copied

_export:
  bq:
    dataset: my_dataset

+process:
  bq>: queries/analyze.sql
  destination_table: result

+export:
  bq_extract>: result
  destination: gs://my_bucket/result.csv.gz
  compression: GZIP

Secrets

When you don't know how to set secrets, please refer to Managing Workflow Secret

gcp.credential : CREDENTIAL
See gcp.credential.

Options

bq_extract> : TABLE A reference to the table that should be exported.
Examples:
Copy

Copied
```
bq_extract>: my_table
```
Copy

Copied
```
bq_extract>: my_dataset.my_table
```
Copy

Copied
```
bq_extract>: my_project:my_dataset.my_table
```
destination : URI | LIST A URI or list of URIs with the location of the destination export files. These must be Google Cloud Storage URIs.
Examples:
Copy

Copied
```
destination: gs://my_bucket/my_export.csv
```
Copy

Copied
```
destination:
- gs://my_bucket/my_export_1.csv
- gs://my_bucket/my_export_2.csv
```
location : LOCATION The location where the job should run. The table and the destination must be in this location. See BigQuery locations for a list of available locations.
Examples:
Copy

Copied
```
location: asia-northeast1
```
print_header : BOOLEAN Whether to print out a header row in the results. Default : true .
field_delimiter : CHARACTER A delimiter to use between fields in the output. Default : , .
Examples:
Copy

Copied
```
field_delimiter: "\t"
```
destination_format : CSV | NEWLINE DELIMITED JSON | AVRO The format of the destination export file. Default : CSV .
Examples:
Copy

Copied
```
destination_format: CSV
```
Copy

Copied
```
destination_format: NEWLINE_DELIMITED_JSON
```
Copy

Copied
```
destination_format: AVRO
```
compression : GZIP | NONE The compression to use for the export file. Default : NONE .
Examples:
Copy

Copied
```
compression: NONE
```
Copy

Copied
```
compression: GZIP
```

Output parameters

bq.last_job_id
The id of the BigQuery job that performed this export.

Note: bq.last_jobid parameter is kept only for backward compatibility but you must not use it because it will be removed in a near future release.