BigQuery
Source: index.
In the following examples from this page and the other modules (Dataset,
                Table, etc.), we are going to be using a dataset from
                data.gov of higher education institutions.
We will create a table with the correct schema, import the public CSV file into that table, and query it for data.
Properties
Methods
new BigQuery(options)
Examples
<caption>Install the client library with <a
href="https://www.npmjs.com/">npm</a>:</caption> npm install --saveImport the client library
const {BigQuery} = require('@google-cloud/bigquery');<caption>Create a client that uses <a
href="https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application">Application
Default Credentials (ADC)</a>:</caption> const bigquery = new BigQuery();<caption>Create a client with <a
href="https://cloud.google.com/docs/authentication/production#obtaining_and_providing_service_account_credentials_manually">explicit
credentials</a>:</caption> const bigquery = new BigQuery({ projectId:
'your-project-id', keyFilename: '/path/to/keyfile.json'
});include:samples/quickstart.js
region_tag:bigquery_quickstart
Full quickstart example:Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| options | 
 | 
 | Constructor options. | 
- See also
Properties
createQueryStream
Run a query scoped to your project as a readable object stream.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const query = 'SELECT url FROM `publicdata.samples.github_nested` LIMIT
100';
bigquery.createQueryStream(query)
  .on('error', console.error)
  .on('data', function(row) {
    // row is a result from your query.
  })
  .on('end', function() {
    // All rows retrieved.
  });
//-
// If you anticipate many results, you can end a stream early to prevent
// unnecessary processing and API requests.
//-
bigquery.createQueryStream(query)
  .on('data', function(row) {
    this.end();
  });Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| query | object | 
 | Configuration object. See Query for a complete list of options. | 
- Returns
- 
                  stream
getDatasetsStream
List all or some of the Dataset objects in your project as a readable object stream.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
bigquery.getDatasetsStream()
  .on('error', console.error)
  .on('data', function(dataset) {
    // dataset is a Dataset object.
  })
  .on('end', function() {
    // All datasets retrieved.
  });
//-
// If you anticipate many results, you can end a stream early to prevent
// unnecessary processing and API requests.
//-
bigquery.getDatasetsStream()
  .on('data', function(dataset) {
    this.end();
  });Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| options | object | Yes | Configuration object. See BigQuery#getDatasets for a complete list of options. | 
- Returns
- 
                  stream
getJobsStream
List all or some of the Job objects in your project as a readable object stream.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
bigquery.getJobsStream()
  .on('error', console.error)
  .on('data', function(job) {
    // job is a Job object.
  })
  .on('end', function() {
    // All jobs retrieved.
  });
//-
// If you anticipate many results, you can end a stream early to prevent
// unnecessary processing and API requests.
//-
bigquery.getJobsStream()
  .on('data', function(job) {
    this.end();
  });Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| options | object | Yes | Configuration object. See BigQuery#getJobs for a complete list of options. | 
- Returns
- 
                  stream
location string
Methods
date(value)
The DATE type represents a logical calendar date, independent of time
                zone. It does not represent a specific 24-hour time period. Rather, a given
                DATE value represents a different 24-hour period when interpreted in
                different time zones, and may represent a shorter or longer day during
                Daylight Savings Time transitions.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const date = bigquery.date('2017-01-01');
//-
// Alternatively, provide an object.
//-
const date2 = bigquery.date({
  year: 2017,
  month: 1,
  day: 1
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| value | (object or string) | 
 | The date. If a string, this should be in the
                          format the API describes:  Values in  
 | 
datetime(value)
A DATETIME data type represents a point in time. Unlike a TIMESTAMP,
                this does not refer to an absolute instance in time. Instead, it is the
                civil time, or the time that a user would see on a watch or calendar.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const datetime = BigQuery.datetime('2017-01-01 13:00:00');
//-
// Alternatively, provide an object.
//-
const datetime = BigQuery.datetime({
  year: 2017,
  month: 1,
  day: 1,
  hours: 14,
  minutes: 0,
  seconds: 0
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| value | (object or string) | 
 | The time. If a string, this should be in the
                          format the API describes:  Values in  
 | 
geography(value)
A geography value represents a surface area on the Earth in Well-known Text (WKT) format.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const geography = BigQuery.geography('POINT(1, 2)');Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| value | string | 
 | The geospatial data. | 
time(value)
A TIME data type represents a time, independent of a specific date.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const time = BigQuery.time('14:00:00'); // 2:00 PM
//-
// Alternatively, provide an object.
//-
const time = BigQuery.time({
  hours: 14,
  minutes: 0,
  seconds: 0
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| value | (object or string) | 
 | The time. If a string, this should be in the
                          format the API describes:  Values in  
 | 
timestamp(value)
A timestamp represents an absolute point in time, independent of any time zone or convention such as Daylight Savings Time.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const timestamp = BigQuery.timestamp(new Date());Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| value | (Date or string) | 
 | The time. | 
createDataset(id[, options][, callback]) → Promise
Create a dataset.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
bigquery.createDataset('my-dataset', function(err, dataset, apiResponse)
{});
//-
// If the callback is omitted, we'll return a Promise.
//-
bigquery.createDataset('my-dataset').then(function(data) {
  const dataset = data[0];
  const apiResponse = data[1];
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string | 
 | ID of the dataset to create. | ||||||||||||||||
| options | object | Yes | See a Dataset resource. | ||||||||||||||||
| callback | function() | Yes | The callback function. Values in  
 | 
- See also
- Returns
- 
                  Promise
createJob(options[, callback]) → Promise
Creates a job. Typically when creating a job you'll have a very specific task in mind. For this we recommend one of the following methods:
- BigQuery#createQueryJob
- Table#createCopyJob
- Table#createCopyFromJob
- Table#createExtractJob
- Table#createLoadJob
However in the event you need a finer level of control over the job creation, you can use this method to pass in a raw Job resource object.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const options = {
  configuration: {
    query: {
      query: 'SELECT url FROM `publicdata.samples.github_nested` LIMIT 100'
    }
  }
};
bigquery.createJob(options, function(err, job) {
  if (err) {
    // Error handling omitted.
  }
  job.getQueryResults(function(err, rows) {});
});
//-
// If the callback is omitted, we'll return a Promise.
//-
bigquery.createJob(options).then(function(data) {
  const job = data[0];
  return job.getQueryResults();
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| options | object | 
 | Object in the form of a Job resource; Values in  
 | ||||||||||||||||
| callback | function() | Yes | The callback function. Values in  
 | 
- See also
- Returns
- 
                  Promise
createQueryJob(options[, callback]) → Promise
Run a query as a job. No results are immediately returned. Instead, your callback will be executed with a Job object that you must ping for the results. See the Job documentation for explanations of how to check on the status of the job.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const query = 'SELECT url FROM `publicdata.samples.github_nested` LIMIT
100';
//-
// You may pass only a query string, having a new table created to store
the
// results of the query.
//-
bigquery.createQueryJob(query, function(err, job) {});
//-
// You can also control the destination table by providing a
// {@link Table} object.
//-
bigquery.createQueryJob({
  destination: bigquery.dataset('higher_education').table('institutions'),
  query: query
}, function(err, job) {});
//-
// After you have run `createQueryJob`, your query will execute in a job.
Your
// callback is executed with a {@link Job} object so that you may
// check for the results.
//-
bigquery.createQueryJob(query, function(err, job) {
  if (!err) {
    job.getQueryResults(function(err, rows, apiResponse) {});
  }
});
//-
// If the callback is omitted, we'll return a Promise.
//-
bigquery.createQueryJob(query).then(function(data) {
  const job = data[0];
  const apiResponse = data[1];
  return job.getQueryResults();
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| options | (object or string) | 
 | The configuration object. This must be in
                          the format of the  Values in  
 | ||||||||||||||||||||||||||||||||
| callback | function() | Yes | The callback function. Values in  
 | 
- See also
- Throws
- 
                  ErrorIf a query is not specified. 
- 
                  ErrorIf a Table is not provided as a destination. 
- Returns
- 
                  Promise
dataset(id[, options]) → Dataset
Create a reference to a dataset.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const dataset = bigquery.dataset('higher_education');Parameters
| Name | Type | Optional | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string | 
 | ID of the dataset. | ||||||||
| options | object | Yes | Dataset options. Values in  
 | 
- Returns
date(value)
Example
const {BigQuery} = require('@google-cloud/bigquery');
const date = BigQuery.date('2017-01-01');
//-
// Alternatively, provide an object.
//-
const date2 = BigQuery.date({
  year: 2017,
  month: 1,
  day: 1
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| value | (object or string) | 
 | The date. If a string, this should be in the
                          format the API describes:  Values in  
 | 
datetime(value)
A DATETIME data type represents a point in time. Unlike a TIMESTAMP,
                this does not refer to an absolute instance in time. Instead, it is the
                civil time, or the time that a user would see on a watch or calendar.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const datetime = bigquery.datetime('2017-01-01 13:00:00');
//-
// Alternatively, provide an object.
//-
const datetime = bigquery.datetime({
  year: 2017,
  month: 1,
  day: 1,
  hours: 14,
  minutes: 0,
  seconds: 0
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| value | (object or string) | 
 | The time. If a string, this should be in the
                          format the API describes:  Values in  
 | 
geography(value)
A geography value represents a surface area on the Earth in Well-known Text (WKT) format.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const geography = bigquery.geography('POINT(1, 2)');Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| value | string | 
 | The geospatial data. | 
getDatasets([options][, callback]) → Promise
List all or some of the datasets in your project.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
bigquery.getDatasets(function(err, datasets) {
  if (!err) {
    // datasets is an array of Dataset objects.
  }
});
//-
// To control how many API requests are made and page through the results
// manually, set `autoPaginate` to `false`.
//-
function manualPaginationCallback(err, datasets, nextQuery, apiResponse) {
  if (nextQuery) {
    // More results exist.
    bigquery.getDatasets(nextQuery, manualPaginationCallback);
  }
}
bigquery.getDatasets({
  autoPaginate: false
}, manualPaginationCallback);
//-
// If the callback is omitted, we'll return a Promise.
//-
bigquery.getDatasets().then(function(datasets) {});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| options | object | Yes | Configuration object. Values in  
 | ||||||||||||||||||||||||
| callback | function() | Yes | The callback function. Values in  
 | 
- See also
- Returns
- 
                  Promise
getJobs([options][, callback]) → Promise
Get all of the jobs from your project.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
bigquery.getJobs(function(err, jobs) {
  if (!err) {
    // jobs is an array of Job objects.
  }
});
//-
// To control how many API requests are made and page through the results
// manually, set `autoPaginate` to `false`.
//-
function manualPaginationCallback(err, jobs, nextQuery, apiRespose) {
  if (nextQuery) {
    // More results exist.
    bigquery.getJobs(nextQuery, manualPaginationCallback);
  }
}
bigquery.getJobs({
  autoPaginate: false
}, manualPaginationCallback);
//-
// If the callback is omitted, we'll return a Promise.
//-
bigquery.getJobs().then(function(data) {
  const jobs = data[0];
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| options | object | Yes | Configuration object. Values in  
 | ||||||||||||||||||||||||||||||||
| callback | function() | Yes | The callback function. Values in  
 | 
- See also
- Returns
- 
                  Promise
job(id[, options]) → Job
Create a reference to an existing job.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const myExistingJob = bigquery.job('job-id');Parameters
| Name | Type | Optional | Description | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| id | string | 
 | ID of the job. | ||||||||
| options | object | Yes | Configuration object. Values in  
 | 
- Returns
query(query[, options][, callback]) → Promise
Run a query scoped to your project. For manual pagination please refer to BigQuery#createQueryJob.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const query = 'SELECT url FROM `publicdata.samples.github_nested` LIMIT
100';
bigquery.query(query, function(err, rows) {
  if (!err) {
    // rows is an array of results.
  }
});
//-
// Positional SQL parameters are supported.
//-
bigquery.query({
  query: [
    'SELECT url',
    'FROM `publicdata.samples.github_nested`',
    'WHERE repository.owner = ?'
  ].join(' '),
  params: [
    'google'
  ]
}, function(err, rows) {});
//-
// Or if you prefer to name them, that's also supported.
//-
bigquery.query({
  query: [
    'SELECT url',
    'FROM `publicdata.samples.github_nested`',
    'WHERE repository.owner = @owner'
  ].join(' '),
  params: {
    owner: 'google'
  }
}, function(err, rows) {});
//-
// If you need to use a `DATE`, `DATETIME`, `TIME`, or `TIMESTAMP` type in
// your query, see {@link BigQuery#date}, {@link BigQuery#datetime},
// {@link BigQuery#time}, and {@link BigQuery#timestamp}.
//-
//-
// If the callback is omitted, we'll return a Promise.
//-
bigquery.query(query).then(function(data) {
  const rows = data[0];
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| query | (string or object) | 
 | A string SQL query or configuration object. For all available options, see Jobs: query request body. Values in  
 | ||||||||||||||||||||||||||||
| options | object | Yes | Configuration object for query results. Values in  
 | ||||||||||||||||||||||||||||
| callback | function() | Yes | The callback function. Values in  
 | 
- See also
- Returns
- 
                  Promise
time(value)
A TIME data type represents a time, independent of a specific date.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const time = bigquery.time('14:00:00'); // 2:00 PM
//-
// Alternatively, provide an object.
//-
const time = bigquery.time({
  hours: 14,
  minutes: 0,
  seconds: 0
});Parameters
| Name | Type | Optional | Description | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| value | (object or string) | 
 | The time. If a string, this should be in the
                          format the API describes:  Values in  
 | 
timestamp(value)
A timestamp represents an absolute point in time, independent of any time zone or convention such as Daylight Savings Time.
Example
const {BigQuery} = require('@google-cloud/bigquery');
const bigquery = new BigQuery();
const timestamp = bigquery.timestamp(new Date());Parameter
| Name | Type | Optional | Description | 
|---|---|---|---|
| value | (Date or string) | 
 | The time. |