Class: Google::Cloud::Bigquery::Project
- Inherits:
-
Object
- Object
- Google::Cloud::Bigquery::Project
- Defined in:
- lib/google/cloud/bigquery/project.rb,
lib/google/cloud/bigquery/project/list.rb
Overview
Project
Projects are top-level containers in Google Cloud Platform. They store information about billing and authorized users, and they contain BigQuery data. Each project has a friendly name and a unique ID.
Google::Cloud::Bigquery::Project is the main object for interacting with Google BigQuery. Dataset objects are created, accessed, and deleted by Google::Cloud::Bigquery::Project.
Defined Under Namespace
Classes: List
Instance Attribute Summary collapse
-
#name ⇒ String?
readonly
The descriptive name of the project.
-
#numeric_id ⇒ Integer?
readonly
The numeric ID of the project.
Data collapse
-
#copy(source_table, destination_table, create: nil, write: nil) {|job| ... } ⇒ Boolean
Copies the data from the source table to the destination table using a synchronous method that blocks for a response.
-
#copy_job(source_table, destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob
Copies the data from the source table to the destination table using an asynchronous method.
-
#create_dataset(dataset_id, name: nil, description: nil, expiration: nil, location: nil) {|access| ... } ⇒ Google::Cloud::Bigquery::Dataset
Creates a new dataset.
-
#dataset(dataset_id, skip_lookup: nil) ⇒ Google::Cloud::Bigquery::Dataset?
Retrieves an existing dataset by ID.
-
#datasets(all: nil, filter: nil, token: nil, max: nil) ⇒ Array<Google::Cloud::Bigquery::Dataset>
Retrieves the list of datasets belonging to the project.
-
#encryption(kms_key: nil) ⇒ Google::Cloud::Bigquery::EncryptionConfiguration
Creates a new Bigquery::EncryptionConfiguration instance.
-
#external(url, format: nil) {|ext| ... } ⇒ External::DataSource
Creates a new External::DataSource (or subclass) object that represents the external data source that can be queried from directly, even though the data is not stored in BigQuery.
-
#extract(source, extract_url, format: nil, compression: nil, delimiter: nil, header: nil) {|job| ... } ⇒ Boolean
Extracts the data from a table or exports a model to Google Cloud Storage using a synchronous method that blocks for a response.
-
#extract_job(source, extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob
Extracts the data from a table or exports a model to Google Cloud Storage asynchronously, immediately returning an ExtractJob that can be used to track the progress of the export job.
-
#job(job_id, location: nil) ⇒ Google::Cloud::Bigquery::Job?
Retrieves an existing job by ID.
-
#jobs(all: nil, token: nil, max: nil, filter: nil, min_created_at: nil, max_created_at: nil, parent_job: nil) ⇒ Array<Google::Cloud::Bigquery::Job>
Retrieves the list of jobs belonging to the project.
-
#projects(token: nil, max: nil) ⇒ Array<Google::Cloud::Bigquery::Project>
Retrieves the list of all projects for which the currently authorized account has been granted any project role.
-
#query(query, params: nil, types: nil, external: nil, max: nil, cache: true, dataset: nil, project: nil, standard_sql: nil, legacy_sql: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::Data
Queries data and waits for the results.
-
#query_job(query, params: nil, types: nil, external: nil, priority: "INTERACTIVE", cache: true, table: nil, create: nil, write: nil, dryrun: nil, dataset: nil, project: nil, standard_sql: nil, legacy_sql: nil, large_results: nil, flatten: nil, maximum_billing_tier: nil, maximum_bytes_billed: nil, job_id: nil, prefix: nil, labels: nil, udfs: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::QueryJob
Queries data by creating a query job.
-
#schema {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema
Creates a new schema instance.
-
#time(hour, minute, second) ⇒ Bigquery::Time
Creates a Bigquery::Time object to represent a time, independent of a specific date.
Instance Method Summary collapse
-
#initialize(service) ⇒ Project
constructor
Creates a new Service instance.
-
#project_id ⇒ Object
(also: #project)
The BigQuery project connected to.
-
#service_account_email ⇒ String
The email address of the service account for the project used to connect to BigQuery.
Constructor Details
#initialize(service) ⇒ Project
Creates a new Service instance.
65 66 67 |
# File 'lib/google/cloud/bigquery/project.rb', line 65 def initialize service @service = service end |
Instance Attribute Details
#name ⇒ String? (readonly)
The descriptive name of the project. Can only be present if the project was retrieved with #projects.
54 55 56 |
# File 'lib/google/cloud/bigquery/project.rb', line 54 def name @name end |
#numeric_id ⇒ Integer? (readonly)
The numeric ID of the project. Can only be present if the project was retrieved with #projects.
54 55 56 |
# File 'lib/google/cloud/bigquery/project.rb', line 54 def numeric_id @numeric_id end |
Instance Method Details
#copy(source_table, destination_table, create: nil, write: nil) {|job| ... } ⇒ Boolean
Copies the data from the source table to the destination table using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See #copy_job for the asynchronous version. Use this method instead of Table#copy to copy from source tables in other projects.
The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method.
269 270 271 272 273 274 |
# File 'lib/google/cloud/bigquery/project.rb', line 269 def copy source_table, destination_table, create: nil, write: nil, &block job = copy_job source_table, destination_table, create: create, write: write, &block job.wait_until_done! ensure_job_succeeded! job true end |
#copy_job(source_table, destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::CopyJob
Copies the data from the source table to the destination table using an asynchronous method. In this method, a CopyJob is immediately returned. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling #Job#wait_until_done!. See #copy for the synchronous version. Use this method instead of Table#copy_job to copy from source tables in other projects.
The geographic location for the job ("US", "EU", etc.) can be set via CopyJob::Updater#location= in a block passed to this method.
192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 |
# File 'lib/google/cloud/bigquery/project.rb', line 192 def copy_job source_table, destination_table, create: nil, write: nil, job_id: nil, prefix: nil, labels: nil ensure_service! = { create: create, write: write, labels: labels, job_id: job_id, prefix: prefix } updater = CopyJob::Updater.( service, Service.get_table_ref(source_table, default_ref: project_ref), Service.get_table_ref(destination_table, default_ref: project_ref), ) yield updater if block_given? job_gapi = updater.to_gapi gapi = service.copy_table job_gapi Job.from_gapi gapi, service end |
#create_dataset(dataset_id, name: nil, description: nil, expiration: nil, location: nil) {|access| ... } ⇒ Google::Cloud::Bigquery::Dataset
Creates a new dataset.
1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 |
# File 'lib/google/cloud/bigquery/project.rb', line 1020 def create_dataset dataset_id, name: nil, description: nil, expiration: nil, location: nil ensure_service! new_ds = Google::Apis::BigqueryV2::Dataset.new( dataset_reference: Google::Apis::BigqueryV2::DatasetReference.new( project_id: project, dataset_id: dataset_id ) ) # Can set location only on creation, no Dataset#location method new_ds.update! location: location unless location.nil? updater = Dataset::Updater.new(new_ds).tap do |b| b.name = name unless name.nil? b.description = description unless description.nil? b.default_expiration = expiration unless expiration.nil? end if block_given? yield updater updater.check_for_mutated_access! end gapi = service.insert_dataset new_ds Dataset.from_gapi gapi, service end |
#dataset(dataset_id, skip_lookup: nil) ⇒ Google::Cloud::Bigquery::Dataset?
Retrieves an existing dataset by ID.
965 966 967 968 969 970 971 972 |
# File 'lib/google/cloud/bigquery/project.rb', line 965 def dataset dataset_id, skip_lookup: nil ensure_service! return Dataset.new_reference project, dataset_id, service if skip_lookup gapi = service.get_dataset dataset_id Dataset.from_gapi gapi, service rescue Google::Cloud::NotFoundError nil end |
#datasets(all: nil, filter: nil, token: nil, max: nil) ⇒ Array<Google::Cloud::Bigquery::Dataset>
Retrieves the list of datasets belonging to the project.
1092 1093 1094 1095 1096 |
# File 'lib/google/cloud/bigquery/project.rb', line 1092 def datasets all: nil, filter: nil, token: nil, max: nil ensure_service! gapi = service.list_datasets all: all, filter: filter, token: token, max: max Dataset::List.from_gapi gapi, service, all, filter, max end |
#encryption(kms_key: nil) ⇒ Google::Cloud::Bigquery::EncryptionConfiguration
Creates a new Bigquery::EncryptionConfiguration instance.
This method does not execute an API call. Use the encryption configuration to encrypt a table when creating one via Bigquery::Dataset#create_table, Bigquery::Dataset#load, Bigquery::Table#copy, or Bigquery::Project#query.
1457 1458 1459 1460 1461 |
# File 'lib/google/cloud/bigquery/project.rb', line 1457 def encryption kms_key: nil encrypt_config = Bigquery::EncryptionConfiguration.new encrypt_config.kms_key = kms_key unless kms_key.nil? encrypt_config end |
#external(url, format: nil) {|ext| ... } ⇒ External::DataSource
Creates a new External::DataSource (or subclass) object that represents the external data source that can be queried from directly, even though the data is not stored in BigQuery. Instead of loading or streaming the data, this object references the external data source.
932 933 934 935 936 |
# File 'lib/google/cloud/bigquery/project.rb', line 932 def external url, format: nil ext = External.from_urls url, format yield ext if block_given? ext end |
#extract(source, extract_url, format: nil, compression: nil, delimiter: nil, header: nil) {|job| ... } ⇒ Boolean
Extracts the data from a table or exports a model to Google Cloud Storage using a synchronous method that blocks for a response. Timeouts and transient errors are generally handled as needed to complete the job. See #extract_job for the asynchronous version.
Use this method instead of Table#extract or Model#extract to extract data from source tables or models in other projects.
The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method.
1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 |
# File 'lib/google/cloud/bigquery/project.rb', line 1670 def extract source, extract_url, format: nil, compression: nil, delimiter: nil, header: nil, &block job = extract_job source, extract_url, format: format, compression: compression, delimiter: delimiter, header: header, &block job.wait_until_done! ensure_job_succeeded! job true end |
#extract_job(source, extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::ExtractJob
Extracts the data from a table or exports a model to Google Cloud Storage asynchronously, immediately returning an ExtractJob that can be used to track the progress of the export job. The caller may poll the service by repeatedly calling Job#reload! and Job#done? to detect when the job is done, or simply block until the job is done by calling
Job#wait_until_done!. See #extract for the synchronous version.
Use this method instead of Table#extract_job or Model#extract_job to extract data from source tables or models in other projects.
The geographic location for the job ("US", "EU", etc.) can be set via ExtractJob::Updater#location= in a block passed to this method.
1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 |
# File 'lib/google/cloud/bigquery/project.rb', line 1574 def extract_job source, extract_url, format: nil, compression: nil, delimiter: nil, header: nil, job_id: nil, prefix: nil, labels: nil ensure_service! = { format: format, compression: compression, delimiter: delimiter, header: header, job_id: job_id, prefix: prefix, labels: labels } source_ref = if source.respond_to? :model_ref source.model_ref else Service.get_table_ref source, default_ref: project_ref end updater = ExtractJob::Updater. service, source_ref, extract_url, yield updater if block_given? job_gapi = updater.to_gapi gapi = service.extract_table job_gapi Job.from_gapi gapi, service end |
#job(job_id, location: nil) ⇒ Google::Cloud::Bigquery::Job?
Retrieves an existing job by ID.
1115 1116 1117 1118 1119 1120 1121 |
# File 'lib/google/cloud/bigquery/project.rb', line 1115 def job job_id, location: nil ensure_service! gapi = service.get_job job_id, location: location Job.from_gapi gapi, service rescue Google::Cloud::NotFoundError nil end |
#jobs(all: nil, token: nil, max: nil, filter: nil, min_created_at: nil, max_created_at: nil, parent_job: nil) ⇒ Array<Google::Cloud::Bigquery::Job>
Retrieves the list of jobs belonging to the project.
1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 |
# File 'lib/google/cloud/bigquery/project.rb', line 1234 def jobs all: nil, token: nil, max: nil, filter: nil, min_created_at: nil, max_created_at: nil, parent_job: nil ensure_service! parent_job = parent_job.job_id if parent_job.is_a? Job = { parent_job_id: parent_job, all: all, token: token, max: max, filter: filter, min_created_at: min_created_at, max_created_at: max_created_at } gapi = service.list_jobs(**) Job::List.from_gapi gapi, service, ** end |
#project_id ⇒ Object Also known as: project
The BigQuery project connected to.
82 83 84 |
# File 'lib/google/cloud/bigquery/project.rb', line 82 def project_id service.project end |
#projects(token: nil, max: nil) ⇒ Array<Google::Cloud::Bigquery::Project>
Retrieves the list of all projects for which the currently authorized account has been granted any project role. The returned project instances share the same credentials as the project used to retrieve them, but lazily create a new API connection for interactions with the BigQuery service.
1296 1297 1298 1299 1300 |
# File 'lib/google/cloud/bigquery/project.rb', line 1296 def projects token: nil, max: nil ensure_service! gapi = service.list_projects token: token, max: max Project::List.from_gapi gapi, service, max end |
#query(query, params: nil, types: nil, external: nil, max: nil, cache: true, dataset: nil, project: nil, standard_sql: nil, legacy_sql: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::Data
Queries data and waits for the results. In this method, a QueryJob is created and its results are saved to a temporary table, then read from the table. Timeouts and transient errors are generally handled as needed to complete the query. When used for executing DDL/DML statements, this method does not return row data.
The geographic location for the job ("US", "EU", etc.) can be set via QueryJob::Updater#location= in a block passed to this method.
865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 |
# File 'lib/google/cloud/bigquery/project.rb', line 865 def query query, params: nil, types: nil, external: nil, max: nil, cache: true, dataset: nil, project: nil, standard_sql: nil, legacy_sql: nil, &block job = query_job query, params: params, types: types, external: external, cache: cache, dataset: dataset, project: project, standard_sql: standard_sql, legacy_sql: legacy_sql, &block job.wait_until_done! if job.failed? begin # raise to activate ruby exception cause handling raise job.gapi_error rescue StandardError => e # wrap Google::Apis::Error with Google::Cloud::Error raise Google::Cloud::Error.from_error(e) end end job.data max: max end |
#query_job(query, params: nil, types: nil, external: nil, priority: "INTERACTIVE", cache: true, table: nil, create: nil, write: nil, dryrun: nil, dataset: nil, project: nil, standard_sql: nil, legacy_sql: nil, large_results: nil, flatten: nil, maximum_billing_tier: nil, maximum_bytes_billed: nil, job_id: nil, prefix: nil, labels: nil, udfs: nil) {|job| ... } ⇒ Google::Cloud::Bigquery::QueryJob
Queries data by creating a query job.
The geographic location for the job ("US", "EU", etc.) can be set via QueryJob::Updater#location= in a block passed to this method.
596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 |
# File 'lib/google/cloud/bigquery/project.rb', line 596 def query_job query, params: nil, types: nil, external: nil, priority: "INTERACTIVE", cache: true, table: nil, create: nil, write: nil, dryrun: nil, dataset: nil, project: nil, standard_sql: nil, legacy_sql: nil, large_results: nil, flatten: nil, maximum_billing_tier: nil, maximum_bytes_billed: nil, job_id: nil, prefix: nil, labels: nil, udfs: nil ensure_service! = { params: params, types: types, external: external, priority: priority, cache: cache, table: table, create: create, write: write, dryrun: dryrun, dataset: dataset, project: (project || self.project), standard_sql: standard_sql, legacy_sql: legacy_sql, large_results: large_results, flatten: flatten, maximum_billing_tier: maximum_billing_tier, maximum_bytes_billed: maximum_bytes_billed, job_id: job_id, prefix: prefix, labels: labels, udfs: udfs } updater = QueryJob::Updater. service, query, yield updater if block_given? gapi = service.query_job updater.to_gapi Job.from_gapi gapi, service end |
#schema {|schema| ... } ⇒ Google::Cloud::Bigquery::Schema
Creates a new schema instance. An optional block may be given to configure the schema, otherwise the schema is returned empty and may be configured directly.
The returned schema can be passed to Dataset#load using the
schema
option. However, for most use cases, the block yielded by
Dataset#load is a more convenient way to configure the schema
for the destination table.
1386 1387 1388 1389 1390 |
# File 'lib/google/cloud/bigquery/project.rb', line 1386 def schema s = Schema.from_gapi yield s if block_given? s end |
#service_account_email ⇒ String
The email address of the service account for the project used to connect to BigQuery. (See also #project_id.)
93 94 95 |
# File 'lib/google/cloud/bigquery/project.rb', line 93 def service_account_email @service_account_email ||= service.project_service_account.email end |
#time(hour, minute, second) ⇒ Bigquery::Time
Creates a Bigquery::Time object to represent a time, independent of a specific date.
1349 1350 1351 |
# File 'lib/google/cloud/bigquery/project.rb', line 1349 def time hour, minute, second Bigquery::Time.new "#{hour}:#{minute}:#{second}" end |