com.google.cloud.spring.bigquery.core

Interface BigQueryOperations

All Known Implementing Classes:

BigQueryTemplate

public interface BigQueryOperations

Defines operations for use with BigQuery.

Since:

1.2

Method Summary

Modifier and Type	Method and Description
org.springframework.util.concurrent.ListenableFuture<com.google.cloud.bigquery.Job>	writeDataToTable(String tableName, InputStream inputStream, com.google.cloud.bigquery.FormatOptions dataFormatOptions) Writes data to a specified BigQuery table.
org.springframework.util.concurrent.ListenableFuture<com.google.cloud.bigquery.Job>	writeDataToTable(String tableName, InputStream inputStream, com.google.cloud.bigquery.FormatOptions dataFormatOptions, com.google.cloud.bigquery.Schema schema) Writes data to a specified BigQuery table with a manually-specified table Schema.
org.springframework.util.concurrent.ListenableFuture<WriteApiResponse>	writeJsonStream(String tableName, InputStream jsonInputStream) This method uses BigQuery Storage Write API to write new line delimited JSON file to the specified table.
org.springframework.util.concurrent.ListenableFuture<WriteApiResponse>	writeJsonStream(String tableName, InputStream jsonInputStream, com.google.cloud.bigquery.Schema schema) This method uses BigQuery Storage Write API to write new line delimited JSON file to the specified table.

Method Detail

writeDataToTable

org.springframework.util.concurrent.ListenableFuture<com.google.cloud.bigquery.Job> writeDataToTable(String tableName,
                                                                                                     InputStream inputStream,
                                                                                                     com.google.cloud.bigquery.FormatOptions dataFormatOptions)

Writes data to a specified BigQuery table.

Parameters:

tableName - name of the table to write to

inputStream - input stream of the table data to write

dataFormatOptions - the format of the data to write

Returns:

ListenableFuture containing the BigQuery Job indicating completion of operation

Throws:

BigQueryException - if errors occur when loading data to the BigQuery table

writeDataToTable

org.springframework.util.concurrent.ListenableFuture<com.google.cloud.bigquery.Job> writeDataToTable(String tableName,
                                                                                                     InputStream inputStream,
                                                                                                     com.google.cloud.bigquery.FormatOptions dataFormatOptions,
                                                                                                     com.google.cloud.bigquery.Schema schema)

Writes data to a specified BigQuery table with a manually-specified table Schema.

Example:

 Schema schema = Schema.of(
    Field.of("CountyId", StandardSQLTypeName.INT64),
    Field.of("State", StandardSQLTypeName.STRING),
    Field.of("County", StandardSQLTypeName.STRING)
 );

 ListenableFuture<Job> bigQueryJobFuture =
     bigQueryTemplate.writeDataToTable(
          TABLE_NAME, dataFile.getInputStream(), FormatOptions.csv(), schema);
 

Parameters:

tableName - name of the table to write to

inputStream - input stream of the table data to write

dataFormatOptions - the format of the data to write

schema - the schema of the table being loaded

Returns:

ListenableFuture containing the BigQuery Job indicating completion of operation

Throws:

BigQueryException - if errors occur when loading data to the BigQuery table

writeJsonStream

org.springframework.util.concurrent.ListenableFuture<WriteApiResponse> writeJsonStream(String tableName,
                                                                                       InputStream jsonInputStream)

This method uses BigQuery Storage Write API to write new line delimited JSON file to the specified table. The Table should already be created as BigQuery Storage Write API doesn't create it automatically.

Parameters:

tableName - name of the table to write to

jsonInputStream - input stream of the json file to be written

Returns:

ListenableFuture containing the WriteApiResponse indicating completion of operation

writeJsonStream

org.springframework.util.concurrent.ListenableFuture<WriteApiResponse> writeJsonStream(String tableName,
                                                                                       InputStream jsonInputStream,
                                                                                       com.google.cloud.bigquery.Schema schema)

This method uses BigQuery Storage Write API to write new line delimited JSON file to the specified table. This method creates a table with the specified schema.

Parameters:

tableName - name of the table to write to

jsonInputStream - input stream of the json file to be written

Returns:

ListenableFuture containing the WriteApiResponse indicating completion of operation