public interface BigQuery extends Service<BigQueryOptions>
Modifier and Type | Interface and Description |
---|---|
static class |
BigQuery.DatasetDeleteOption
Class for specifying dataset delete options.
|
static class |
BigQuery.DatasetField
Fields of a BigQuery Dataset resource.
|
static class |
BigQuery.DatasetListOption
Class for specifying dataset list options.
|
static class |
BigQuery.DatasetOption
Class for specifying dataset get, create and update options.
|
static class |
BigQuery.JobField
Fields of a BigQuery Job resource.
|
static class |
BigQuery.JobListOption
Class for specifying job list options.
|
static class |
BigQuery.JobOption
Class for specifying table get and create options.
|
static class |
BigQuery.QueryOption |
static class |
BigQuery.QueryResultsOption
Class for specifying query results options.
|
static class |
BigQuery.TableDataListOption
Class for specifying table data list options.
|
static class |
BigQuery.TableField
Fields of a BigQuery Table resource.
|
static class |
BigQuery.TableListOption
Class for specifying table list options.
|
static class |
BigQuery.TableOption
Class for specifying table get, create and update options.
|
Modifier and Type | Method and Description |
---|---|
boolean |
cancel(JobId jobId)
Sends a job cancel request.
|
boolean |
cancel(String jobId)
Sends a job cancel request.
|
Dataset |
create(DatasetInfo datasetInfo,
BigQuery.DatasetOption... options)
Creates a new dataset.
|
Job |
create(JobInfo jobInfo,
BigQuery.JobOption... options)
Creates a new job.
|
Table |
create(TableInfo tableInfo,
BigQuery.TableOption... options)
Creates a new table.
|
boolean |
delete(DatasetId datasetId,
BigQuery.DatasetDeleteOption... options)
Deletes the requested dataset.
|
boolean |
delete(String datasetId,
BigQuery.DatasetDeleteOption... options)
Deletes the requested dataset.
|
boolean |
delete(String datasetId,
String tableId)
Deletes the requested table.
|
boolean |
delete(TableId tableId)
Deletes the requested table.
|
Dataset |
getDataset(DatasetId datasetId,
BigQuery.DatasetOption... options)
Returns the requested dataset or
null if not found. |
Dataset |
getDataset(String datasetId,
BigQuery.DatasetOption... options)
Returns the requested dataset or
null if not found. |
Job |
getJob(JobId jobId,
BigQuery.JobOption... options)
Returns the requested job or
null if not found. |
Job |
getJob(String jobId,
BigQuery.JobOption... options)
Returns the requested job or
null if not found. |
QueryResponse |
getQueryResults(JobId jobId,
BigQuery.QueryResultsOption... options)
Returns results of the query associated with the provided job.
|
Table |
getTable(String datasetId,
String tableId,
BigQuery.TableOption... options)
Returns the requested table or
null if not found. |
Table |
getTable(TableId tableId,
BigQuery.TableOption... options)
Returns the requested table or
null if not found. |
InsertAllResponse |
insertAll(InsertAllRequest request)
Sends an insert all request.
|
Page<Dataset> |
listDatasets(BigQuery.DatasetListOption... options)
Lists the project's datasets.
|
Page<Dataset> |
listDatasets(String projectId,
BigQuery.DatasetListOption... options)
Lists the datasets in the provided project.
|
Page<Job> |
listJobs(BigQuery.JobListOption... options)
Lists the jobs.
|
List<String> |
listPartitions(TableId tableId) |
TableResult |
listTableData(String datasetId,
String tableId,
BigQuery.TableDataListOption... options)
Lists the table's rows.
|
TableResult |
listTableData(String datasetId,
String tableId,
Schema schema,
BigQuery.TableDataListOption... options)
Lists the table's rows.
|
TableResult |
listTableData(TableId tableId,
BigQuery.TableDataListOption... options)
Lists the table's rows.
|
TableResult |
listTableData(TableId tableId,
Schema schema,
BigQuery.TableDataListOption... options)
Lists the table's rows.
|
Page<Table> |
listTables(DatasetId datasetId,
BigQuery.TableListOption... options)
Lists the tables in the dataset.
|
Page<Table> |
listTables(String datasetId,
BigQuery.TableListOption... options)
Lists the tables in the dataset.
|
TableResult |
query(QueryJobConfiguration configuration,
BigQuery.JobOption... options)
Runs the query associated with the request, using an internally-generated random JobId.
|
TableResult |
query(QueryJobConfiguration configuration,
JobId jobId,
BigQuery.JobOption... options)
Runs the query associated with the request, using the given JobId.
|
Dataset |
update(DatasetInfo datasetInfo,
BigQuery.DatasetOption... options)
Updates dataset information.
|
Table |
update(TableInfo tableInfo,
BigQuery.TableOption... options)
Updates table information.
|
TableDataWriteChannel |
writer(JobId jobId,
WriteChannelConfiguration writeChannelConfiguration)
Returns a channel to write data to be inserted into a BigQuery table.
|
TableDataWriteChannel |
writer(WriteChannelConfiguration writeChannelConfiguration)
Returns a channel to write data to be inserted into a BigQuery table.
|
getOptions
Dataset create(DatasetInfo datasetInfo, BigQuery.DatasetOption... options)
Example of creating a dataset.
String datasetName = "my_dataset_name";
Dataset dataset = null;
DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();
try {
// the dataset was created
dataset = bigquery.create(datasetInfo);
} catch (BigQueryException e) {
// the dataset was not created
}
BigQueryException
- upon failureTable create(TableInfo tableInfo, BigQuery.TableOption... options)
Example of creating a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String fieldName = "string_field";
TableId tableId = TableId.of(datasetName, tableName);
// Table field definition
Field field = Field.of(fieldName, LegacySQLTypeName.STRING);
// Table schema definition
Schema schema = Schema.of(field);
TableDefinition tableDefinition = StandardTableDefinition.of(schema);
TableInfo tableInfo = TableInfo.newBuilder(tableId, tableDefinition).build();
Table table = bigquery.create(tableInfo);
BigQueryException
- upon failureJob create(JobInfo jobInfo, BigQuery.JobOption... options)
Example of loading a newline-delimited-json file with textual fields from GCS to a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String sourceUri = "gs://cloud-samples-data/bigquery/us-states/us-states.json";
TableId tableId = TableId.of(datasetName, tableName);
// Table field definition
Field[] fields =
new Field[] {
Field.of("name", LegacySQLTypeName.STRING),
Field.of("post_abbr", LegacySQLTypeName.STRING)
};
// Table schema definition
Schema schema = Schema.of(fields);
LoadJobConfiguration configuration =
LoadJobConfiguration.builder(tableId, sourceUri)
.setFormatOptions(FormatOptions.json())
.setCreateDisposition(CreateDisposition.CREATE_IF_NEEDED)
.setSchema(schema)
.build();
// Load the table
Job loadJob = bigquery.create(JobInfo.of(configuration));
loadJob = loadJob.waitFor();
// Check the table
System.out.println("State: " + loadJob.getStatus().getState());
return ((StandardTableDefinition) bigquery.getTable(tableId).getDefinition()).getNumRows();
Example of creating a query job.
String query = "SELECT field FROM my_dataset_name.my_table_name";
Job job = null;
JobConfiguration jobConfiguration = QueryJobConfiguration.of(query);
JobInfo jobInfo = JobInfo.of(jobConfiguration);
try {
job = bigquery.create(jobInfo);
} catch (BigQueryException e) {
// the job was not created
}
BigQueryException
- upon failureDataset getDataset(String datasetId, BigQuery.DatasetOption... options)
null
if not found.
Example of getting a dataset.
String datasetName = "my_dataset";
Dataset dataset = bigquery.getDataset(datasetName);
BigQueryException
- upon failureDataset getDataset(DatasetId datasetId, BigQuery.DatasetOption... options)
null
if not found.
Example of getting a dataset.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
DatasetId datasetId = DatasetId.of(projectId, datasetName);
Dataset dataset = bigquery.getDataset(datasetId);
BigQueryException
- upon failurePage<Dataset> listDatasets(BigQuery.DatasetListOption... options)
DatasetInfo.getDatasetId()
, DatasetInfo.getFriendlyName()
and DatasetInfo.getGeneratedId()
). To get complete information use either getDataset(String,
DatasetOption...)
or getDataset(DatasetId, DatasetOption...)
.
Example of listing datasets, specifying the page size.
// List datasets in the default project
Page<Dataset> datasets = bigquery.listDatasets(DatasetListOption.pageSize(100));
for (Dataset dataset : datasets.iterateAll()) {
// do something with the dataset
}
BigQueryException
- upon failurePage<Dataset> listDatasets(String projectId, BigQuery.DatasetListOption... options)
DatasetInfo.getDatasetId()
, DatasetInfo.getFriendlyName()
and DatasetInfo.getGeneratedId()
). To get complete information use either getDataset(String,
DatasetOption...)
or getDataset(DatasetId, DatasetOption...)
.
Example of listing datasets in a project, specifying the page size.
String projectId = "my_project_id";
// List datasets in a specified project
Page<Dataset> datasets = bigquery.listDatasets(projectId, DatasetListOption.pageSize(100));
for (Dataset dataset : datasets.iterateAll()) {
// do something with the dataset
}
BigQueryException
- upon failureboolean delete(String datasetId, BigQuery.DatasetDeleteOption... options)
Example of deleting a dataset from its id, even if non-empty.
String datasetName = "my_dataset_name";
boolean deleted = bigquery.delete(datasetName, DatasetDeleteOption.deleteContents());
if (deleted) {
// the dataset was deleted
} else {
// the dataset was not found
}
true
if dataset was deleted, false
if it was not foundBigQueryException
- upon failureboolean delete(DatasetId datasetId, BigQuery.DatasetDeleteOption... options)
Example of deleting a dataset, even if non-empty.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
DatasetId datasetId = DatasetId.of(projectId, datasetName);
boolean deleted = bigquery.delete(datasetId, DatasetDeleteOption.deleteContents());
if (deleted) {
// the dataset was deleted
} else {
// the dataset was not found
}
true
if dataset was deleted, false
if it was not foundBigQueryException
- upon failureboolean delete(String datasetId, String tableId)
Example of deleting a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
boolean deleted = bigquery.delete(datasetName, tableName);
if (deleted) {
// the table was deleted
} else {
// the table was not found
}
true
if table was deleted, false
if it was not foundBigQueryException
- upon failureboolean delete(TableId tableId)
Example of deleting a table.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableId = TableId.of(projectId, datasetName, tableName);
boolean deleted = bigquery.delete(tableId);
if (deleted) {
// the table was deleted
} else {
// the table was not found
}
true
if table was deleted, false
if it was not foundBigQueryException
- upon failureDataset update(DatasetInfo datasetInfo, BigQuery.DatasetOption... options)
Example of updating a dataset by changing its description.
// String datasetName = "my_dataset_name";
// String tableName = "my_table_name";
// String newDescription = "new_description";
Table beforeTable = bigquery.getTable(datasetName, tableName);
TableInfo tableInfo = beforeTable.toBuilder()
.setDescription(newDescription)
.build();
Table afterTable = bigquery.update(tableInfo);
BigQueryException
- upon failureTable update(TableInfo tableInfo, BigQuery.TableOption... options)
Example of updating a table by changing its description.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String newDescription = "new_description";
Table beforeTable = bigquery.getTable(datasetName, tableName);
TableInfo tableInfo = beforeTable.toBuilder()
.setDescription(newDescription)
.build();
Table afterTable = bigquery.update(tableInfo);
Example of updating a table by changing its expiration.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Table beforeTable = bigquery.getTable(datasetName, tableName);
// Set table to expire 5 days from now.
long expirationMillis = DateTime.now().plusDays(5).getMillis();
TableInfo tableInfo = beforeTable.toBuilder()
.setExpirationTime(expirationMillis)
.build();
Table afterTable = bigquery.update(tableInfo);
BigQueryException
- upon failureTable getTable(String datasetId, String tableId, BigQuery.TableOption... options)
null
if not found.
Example of getting a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Table table = bigquery.getTable(datasetName, tableName);
BigQueryException
- upon failureTable getTable(TableId tableId, BigQuery.TableOption... options)
null
if not found.
Example of getting a table.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableId = TableId.of(projectId, datasetName, tableName);
Table table = bigquery.getTable(tableId);
BigQueryException
- upon failurePage<Table> listTables(String datasetId, BigQuery.TableListOption... options)
TableInfo.getTableId()
, TableInfo.getFriendlyName()
, TableInfo.getGeneratedId()
and type,
which is part of TableInfo.getDefinition()
). To get complete information use either getTable(TableId, TableOption...)
or getTable(String, String, TableOption...)
.
Example of listing the tables in a dataset, specifying the page size.
String datasetName = "my_dataset_name";
Page<Table> tables = bigquery.listTables(datasetName, TableListOption.pageSize(100));
for (Table table : tables.iterateAll()) {
// do something with the table
}
BigQueryException
- upon failurePage<Table> listTables(DatasetId datasetId, BigQuery.TableListOption... options)
TableInfo.getTableId()
, TableInfo.getFriendlyName()
, TableInfo.getGeneratedId()
and type,
which is part of TableInfo.getDefinition()
). To get complete information use either getTable(TableId, TableOption...)
or getTable(String, String, TableOption...)
.
Example of listing the tables in a dataset.
String projectId = "my_project_id";
String datasetName = "my_dataset_name";
DatasetId datasetId = DatasetId.of(projectId, datasetName);
Page<Table> tables = bigquery.listTables(datasetId, TableListOption.pageSize(100));
for (Table table : tables.iterateAll()) {
// do something with the table
}
BigQueryException
- upon failureList<String> listPartitions(TableId tableId)
tableId
- InsertAllResponse insertAll(InsertAllRequest request)
Example of inserting rows into a table without running a load job.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableId = TableId.of(datasetName, tableName);
// Values of the row to insert
Map<String, Object> rowContent = new HashMap<>();
rowContent.put("booleanField", true);
// Bytes are passed in base64
rowContent.put("bytesField", "Cg0NDg0="); // 0xA, 0xD, 0xD, 0xE, 0xD in base64
// Records are passed as a map
Map<String, Object> recordsContent = new HashMap<>();
recordsContent.put("stringField", "Hello, World!");
rowContent.put("recordField", recordsContent);
InsertAllResponse response =
bigquery.insertAll(
InsertAllRequest.newBuilder(tableId)
.addRow("rowId", rowContent)
// More rows can be added in the same RPC by invoking .addRow() on the builder
.build());
if (response.hasErrors()) {
// If any of the insertions failed, this lets you inspect the errors
for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
// inspect row error
}
}
BigQueryException
- upon failureTableResult listTableData(String datasetId, String tableId, BigQuery.TableDataListOption... options)
Example of listing table rows, specifying the page size.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
// This example reads the result 100 rows per RPC call. If there's no need to limit the number,
// simply omit the option.
TableResult tableData =
bigquery.listTableData(datasetName, tableName, TableDataListOption.pageSize(100));
for (FieldValueList row : tableData.iterateAll()) {
// do something with the row
}
BigQueryException
- upon failureTableResult listTableData(TableId tableId, BigQuery.TableDataListOption... options)
Example of listing table rows, specifying the page size.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
TableId tableIdObject = TableId.of(datasetName, tableName);
// This example reads the result 100 rows per RPC call. If there's no need to limit the number,
// simply omit the option.
TableResult tableData =
bigquery.listTableData(tableIdObject, TableDataListOption.pageSize(100));
for (FieldValueList row : tableData.iterateAll()) {
// do something with the row
}
BigQueryException
- upon failureTableResult listTableData(String datasetId, String tableId, Schema schema, BigQuery.TableDataListOption... options)
schema
is not null
, it is available to the
FieldValueList
iterated over.
Example of listing table rows with schema.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Schema schema = ...;
String field = "field";
TableResult tableData = bigquery.listTableData(datasetName, tableName, schema);
for (FieldValueList row : tableData.iterateAll()) {
row.get(field);
}
BigQueryException
- upon failureTableResult listTableData(TableId tableId, Schema schema, BigQuery.TableDataListOption... options)
schema
is not null
, it is available to the
FieldValueList
iterated over.
Example of listing table rows with schema.
Schema schema =
Schema.of(
Field.of("word", LegacySQLTypeName.STRING),
Field.of("word_count", LegacySQLTypeName.STRING),
Field.of("corpus", LegacySQLTypeName.STRING),
Field.of("corpus_date", LegacySQLTypeName.STRING));
TableResult tableData =
bigquery.listTableData(
TableId.of("bigquery-public-data", "samples", "shakespeare"), schema);
FieldValueList row = tableData.getValues().iterator().next();
System.out.println(row.get("word").getStringValue());
BigQueryException
- upon failureJob getJob(String jobId, BigQuery.JobOption... options)
null
if not found. If the location of the job is not "US"
or "EU", getJob(JobId, JobOption...)
must be used instead.
Example of getting a job.
String jobName = "my_job_name";
Job job = bigquery.getJob(jobName);
if (job == null) {
// job was not found
}
BigQueryException
- upon failureJob getJob(JobId jobId, BigQuery.JobOption... options)
null
if not found. If the location of the job is not "US"
or "EU", the jobId
must specify the job location.
Example of getting a job.
String jobName = "my_job_name";
JobId jobIdObject = JobId.of(jobName);
Job job = bigquery.getJob(jobIdObject);
if (job == null) {
// job was not found
}
BigQueryException
- upon failurePage<Job> listJobs(BigQuery.JobListOption... options)
Example of listing jobs, specifying the page size.
Page<Job> jobs = bigquery.listJobs(JobListOption.pageSize(100));
for (Job job : jobs.iterateAll()) {
// do something with the job
}
BigQueryException
- upon failureboolean cancel(String jobId)
getJob(JobId, JobOption...)
or getJob(String,
JobOption...)
).
If the location of the job is not "US" or "EU", cancel(JobId)
must be used instead.
Example of cancelling a job.
String jobName = "my_job_name";
boolean success = bigquery.cancel(jobName);
if (success) {
// job was cancelled
} else {
// job was not found
}
true
if cancel was requested successfully, false
if the job was not
foundBigQueryException
- upon failureboolean cancel(JobId jobId)
getJob(JobId, JobOption...)
or getJob(String,
JobOption...)
).
If the location of the job is not "US" or "EU", the jobId
must specify the job
location.
Example of cancelling a job.
String jobName = "my_job_name";
JobId jobId = JobId.of(jobName);
boolean success = bigquery.cancel(jobId);
if (success) {
// job was cancelled
} else {
// job was not found
}
true
if cancel was requested successfully, false
if the job was not
foundBigQueryException
- upon failureTableResult query(QueryJobConfiguration configuration, BigQuery.JobOption... options) throws InterruptedException, JobException
If the location of the job is not "US" or "EU", query(QueryJobConfiguration, JobId,
JobOption...)
must be used instead.
This method cannot be used in conjuction with QueryJobConfiguration.dryRun()
queries. Since dry-run queries are not actually executed, there's no way to retrieve results.
Example of running a query.
// BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();
String query = "SELECT corpus FROM `bigquery-public-data.samples.shakespeare` GROUP BY corpus;";
QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(query).build();
// Print the results.
for (FieldValueList row : bigquery.query(queryConfig).iterateAll()) {
for (FieldValue val : row) {
System.out.printf("%s,", val.toString());
}
System.out.printf("\n");
}
BigQueryException
- upon failureInterruptedException
- if the current thread gets interrupted while waiting for the query
to completeJobException
- if the job completes unsuccessfullyTableResult query(QueryJobConfiguration configuration, JobId jobId, BigQuery.JobOption... options) throws InterruptedException, JobException
If the location of the job is not "US" or "EU", the jobId
must specify the job
location.
This method cannot be used in conjuction with QueryJobConfiguration.dryRun()
queries. Since dry-run queries are not actually executed, there's no way to retrieve results.
See query(QueryJobConfiguration, JobOption...)
for examples on populating a QueryJobConfiguration
.
BigQueryException
- upon failureInterruptedException
- if the current thread gets interrupted while waiting for the query
to completeJobException
- if the job completes unsuccessfully@InternalApi QueryResponse getQueryResults(JobId jobId, BigQuery.QueryResultsOption... options)
Users are encouraged to use Job#getQueryResults(QueryResultsOption...)
instead.
TableDataWriteChannel writer(WriteChannelConfiguration writeChannelConfiguration)
WriteChannelConfiguration
parameter. If the job is
not in "US" or "EU", writer(JobId, WriteChannelConfiguration)
must be used instead.
Example of creating a channel with which to write to a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String csvData = "StringValue1\nStringValue2\n";
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId).setFormatOptions(FormatOptions.csv()).build();
TableDataWriteChannel writer = bigquery.writer(writeChannelConfiguration);
// Write data to writer
try {
writer.write(ByteBuffer.wrap(csvData.getBytes(Charsets.UTF_8)));
} finally {
writer.close();
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
Example of writing a local file to a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
Path csvPath = FileSystems.getDefault().getPath(".", "my-data.csv");
String location = "us";
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId).setFormatOptions(FormatOptions.csv()).build();
// The location must be specified; other fields can be auto-detected.
JobId jobId = JobId.newBuilder().setLocation(location).build();
TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
// Write data to writer
try (OutputStream stream = Channels.newOutputStream(writer)) {
Files.copy(csvPath, stream);
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
BigQueryException
- upon failureTableDataWriteChannel writer(JobId jobId, WriteChannelConfiguration writeChannelConfiguration)
WriteChannelConfiguration
parameter. If the job is
not in "US" or "EU", the jobId
must contain the location of the job.
Example of creating a channel with which to write to a table.
String datasetName = "my_dataset_name";
String tableName = "my_table_name";
String csvData = "StringValue1\nStringValue2\n";
String location = "us";
TableId tableId = TableId.of(datasetName, tableName);
WriteChannelConfiguration writeChannelConfiguration =
WriteChannelConfiguration.newBuilder(tableId).setFormatOptions(FormatOptions.csv()).build();
// The location must be specified; other fields can be auto-detected.
JobId jobId = JobId.newBuilder().setLocation(location).build();
TableDataWriteChannel writer = bigquery.writer(jobId, writeChannelConfiguration);
// Write data to writer
try {
writer.write(ByteBuffer.wrap(csvData.getBytes(Charsets.UTF_8)));
} finally {
writer.close();
}
// Get load job
Job job = writer.getJob();
job = job.waitFor();
LoadStatistics stats = job.getStatistics();
return stats.getOutputRows();
Copyright © 2019 Google LLC. All rights reserved.