com.google.cloud.spring.bigquery.core

Class BigQueryTemplate

java.lang.Object

com.google.cloud.spring.bigquery.core.BigQueryTemplate

All Implemented Interfaces:

BigQueryOperations

public class BigQueryTemplate
extends Object
implements BigQueryOperations

Helper class which simplifies common operations done in BigQuery.

Since:

1.2

Author:

Daniel Zou

Constructor Summary

Constructors

Constructor and Description
BigQueryTemplate(BigQuery bigQuery, String datasetName) Creates the BigQuery template.
BigQueryTemplate(BigQuery bigQuery, String datasetName, org.springframework.scheduling.TaskScheduler taskScheduler) Creates the BigQuery template.

Method Summary

Modifier and Type	Method and Description
String	getDatasetName()
void	setAutoDetectSchema(boolean autoDetectSchema) Sets whether BigQuery should attempt to autodetect the schema of the data when loading data into an empty table for the first time.
void	setJobPollInterval(Duration jobPollInterval) Sets the Duration amount of time to wait between successive polls on the status of a BigQuery job.
void	setWriteDisposition(JobInfo.WriteDisposition writeDisposition) Sets the JobInfo.WriteDisposition which specifies how data should be inserted into BigQuery tables.
org.springframework.util.concurrent.ListenableFuture<Job>	writeDataToTable(String tableName, InputStream inputStream, FormatOptions dataFormatOptions) Writes data to a specified BigQuery table.
org.springframework.util.concurrent.ListenableFuture<Job>	writeDataToTable(String tableName, InputStream inputStream, FormatOptions dataFormatOptions, Schema schema) Writes data to a specified BigQuery table with a manually-specified table Schema.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BigQueryTemplate

public BigQueryTemplate(BigQuery bigQuery,
                        String datasetName)

Creates the BigQuery template.

Parameters:

bigQuery - the underlying client object used to interface with BigQuery

datasetName - the name of the dataset in which all operations will take place

BigQueryTemplate

public BigQueryTemplate(BigQuery bigQuery,
                        String datasetName,
                        org.springframework.scheduling.TaskScheduler taskScheduler)

Creates the BigQuery template.

Parameters:

bigQuery - the underlying client object used to interface with BigQuery

datasetName - the name of the dataset in which all operations will take place

taskScheduler - the TaskScheduler used to poll for the status of long-running BigQuery operations

Method Detail

setAutoDetectSchema

public void setAutoDetectSchema(boolean autoDetectSchema)

Sets whether BigQuery should attempt to autodetect the schema of the data when loading data into an empty table for the first time. If set to false, the schema must be defined explicitly for the table before load.

Parameters:

autoDetectSchema - whether data schema should be autodetected from the structure of the data. Default is true.

setWriteDisposition

public void setWriteDisposition(JobInfo.WriteDisposition writeDisposition)

Sets the JobInfo.WriteDisposition which specifies how data should be inserted into BigQuery tables.

Parameters:

writeDisposition - whether to append to or truncate (overwrite) data in the BigQuery table. Default is WriteDisposition.WRITE_APPEND to append data to a table.

setJobPollInterval

public void setJobPollInterval(Duration jobPollInterval)

Sets the Duration amount of time to wait between successive polls on the status of a BigQuery job.

Parameters:

jobPollInterval - the Duration poll interval for BigQuery job status polling

writeDataToTable

public org.springframework.util.concurrent.ListenableFuture<Job> writeDataToTable(String tableName,
                                                                                  InputStream inputStream,
                                                                                  FormatOptions dataFormatOptions)

Description copied from interface: BigQueryOperations

Writes data to a specified BigQuery table.

Specified by:

writeDataToTable in interface BigQueryOperations

Parameters:

tableName - name of the table to write to

inputStream - input stream of the table data to write

dataFormatOptions - the format of the data to write

Returns:

ListenableFuture containing the BigQuery Job indicating completion of operation

writeDataToTable

public org.springframework.util.concurrent.ListenableFuture<Job> writeDataToTable(String tableName,
                                                                                  InputStream inputStream,
                                                                                  FormatOptions dataFormatOptions,
                                                                                  Schema schema)

Description copied from interface: BigQueryOperations

Writes data to a specified BigQuery table with a manually-specified table Schema.

Example:

 Schema schema = Schema.of(
    Field.of("CountyId", StandardSQLTypeName.INT64),
    Field.of("State", StandardSQLTypeName.STRING),
    Field.of("County", StandardSQLTypeName.STRING)
 );

 ListenableFuture<Job> bigQueryJobFuture =
     bigQueryTemplate.writeDataToTable(
 	       TABLE_NAME, dataFile.getInputStream(), FormatOptions.csv(), schema);
 

Specified by:

writeDataToTable in interface BigQueryOperations

Parameters:

tableName - name of the table to write to

inputStream - input stream of the table data to write

dataFormatOptions - the format of the data to write

schema - the schema of the table being loaded

Returns:

ListenableFuture containing the BigQuery Job indicating completion of operation

getDatasetName

public String getDatasetName()

Returns:

the name of the BigQuery dataset that the template is operating in.