The Google BigQuery Loader job entry enables you to load data into Google BigQuery from a Google Cloud Storage account.
The Google BigQuery Loader supports the following formats:
- Comma-separated values (CSV)
- JSON (newline-delimited)
Before you begin
Perform the following steps to set up your system to use Google BigQuery:
Download the service account credentials file that you have created using the Google API Console to your local machine.
Create a new system environmental variable on your operating system named GOOGLE_APPLICATION_CREDENTIALS.
Set the path to the downloaded JSON service account credentials file as the value of the GOOGLE_APPLICATION_CREDENTIALS variable.
Reboot your local machine.
Enter the following information in the job entry field:
- Entry name: Specifies the unique name of the Google BigQuery Loader entry on the canvas. You can customize the name or leave it as the default.
The Google BigQuery Loader job entry features two tabs with fields. Each tab is described below.
This tab includes the following fields:
|Storage source URL
|Specify the Google Cloud Storage URL of the data to import. The URL can point to a file or a folder in Google Cloud Storage. The URL must begin with gs:// and must specify the bucket and object you want to load. You must specify the file type in the File tab.
|Specify the dataset where you want to import a table. The dataset drop-down is automatically populated when you select the Storage source URL, but you can enter a new dataset name in the field. If the dataset does not exist it will be created at runtime.
|Specify the table name in the dataset where you want to import data.
|Specify the column name in the datatset table where you want to import data.
|Specify the column type in the datatset table where you want to import data.
|Overwrite existing table
|Select to overwrite existing data with imported data.
|Specify the file type. The values are CSV (default), JSON, and Avro. You must specify the correct File type that is associated with the Storage source URL field in the Setup tab.
|Leading rows to skip
|Specify how many rows of the CSV file to skip.
|Specify the delimiter character used by the CSV file.
|Specify the escape (quote) character used for values that have the delimiter character in them. For example, when the delimiter character is a comma, and a field contains a comma, and the quote character is a backslash, inserting a backslash before the comma in the field will prevent that field from being evaluated as the beginning of a new field value.