Class: Google::Apis::BigqueryV2::ExternalDataConfiguration
- Inherits:
-
Object
- Object
- Google::Apis::BigqueryV2::ExternalDataConfiguration
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- lib/google/apis/bigquery_v2/classes.rb,
lib/google/apis/bigquery_v2/representations.rb,
lib/google/apis/bigquery_v2/representations.rb
Instance Attribute Summary collapse
-
#autodetect ⇒ Boolean
(also: #autodetect?)
Try to detect schema and format options automatically.
-
#avro_options ⇒ Google::Apis::BigqueryV2::AvroOptions
Options for external data sources.
-
#bigtable_options ⇒ Google::Apis::BigqueryV2::BigtableOptions
Options specific to Google Cloud Bigtable data sources.
-
#compression ⇒ String
Optional.
-
#connection_id ⇒ String
Optional.
-
#csv_options ⇒ Google::Apis::BigqueryV2::CsvOptions
Information related to a CSV data source.
-
#decimal_target_types ⇒ Array<String>
Defines the list of possible SQL data types to which the source decimal values are converted.
-
#file_set_spec_type ⇒ String
Optional.
-
#google_sheets_options ⇒ Google::Apis::BigqueryV2::GoogleSheetsOptions
Options specific to Google Sheets data sources.
-
#hive_partitioning_options ⇒ Google::Apis::BigqueryV2::HivePartitioningOptions
Options for configuring hive partitioning detect.
-
#ignore_unknown_values ⇒ Boolean
(also: #ignore_unknown_values?)
Optional.
-
#json_extension ⇒ String
Optional.
-
#json_options ⇒ Google::Apis::BigqueryV2::JsonOptions
Json Options for load and make external tables.
-
#max_bad_records ⇒ Fixnum
Optional.
-
#metadata_cache_mode ⇒ String
Optional.
-
#object_metadata ⇒ String
Optional.
-
#parquet_options ⇒ Google::Apis::BigqueryV2::ParquetOptions
Parquet Options for load and make external tables.
-
#reference_file_schema_uri ⇒ String
Optional.
-
#schema ⇒ Google::Apis::BigqueryV2::TableSchema
Schema of a table Corresponds to the JSON property
schema
. -
#source_format ⇒ String
[Required] The data format.
-
#source_uris ⇒ Array<String>
[Required] The fully-qualified URIs that point to your data in Google Cloud.
Instance Method Summary collapse
-
#initialize(**args) ⇒ ExternalDataConfiguration
constructor
A new instance of ExternalDataConfiguration.
-
#update!(**args) ⇒ Object
Update properties of this object.
Constructor Details
#initialize(**args) ⇒ ExternalDataConfiguration
Returns a new instance of ExternalDataConfiguration.
3264 3265 3266 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3264 def initialize(**args) update!(**args) end |
Instance Attribute Details
#autodetect ⇒ Boolean Also known as: autodetect?
Try to detect schema and format options automatically. Any option specified
explicitly will be honored.
Corresponds to the JSON property autodetect
3103 3104 3105 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3103 def autodetect @autodetect end |
#avro_options ⇒ Google::Apis::BigqueryV2::AvroOptions
Options for external data sources.
Corresponds to the JSON property avroOptions
3109 3110 3111 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3109 def @avro_options end |
#bigtable_options ⇒ Google::Apis::BigqueryV2::BigtableOptions
Options specific to Google Cloud Bigtable data sources.
Corresponds to the JSON property bigtableOptions
3114 3115 3116 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3114 def @bigtable_options end |
#compression ⇒ String
Optional. The compression type of the data source. Possible values include
GZIP and NONE. The default value is NONE. This setting is ignored for Google
Cloud Bigtable, Google Cloud Datastore backups, Avro, ORC and Parquet formats.
An empty string is an invalid value.
Corresponds to the JSON property compression
3122 3123 3124 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3122 def compression @compression end |
#connection_id ⇒ String
Optional. The connection specifying the credentials to be used to read
external storage, such as Azure Blob, Cloud Storage, or S3. The connection_id
can have the form project_id`.`location_id`;`connection_id
or projects/
project_id/locations/
location_id/connections/
connection_id`.
Corresponds to the JSON property
connectionId`
3130 3131 3132 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3130 def connection_id @connection_id end |
#csv_options ⇒ Google::Apis::BigqueryV2::CsvOptions
Information related to a CSV data source.
Corresponds to the JSON property csvOptions
3135 3136 3137 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3135 def @csv_options end |
#decimal_target_types ⇒ Array<String>
Defines the list of possible SQL data types to which the source decimal values
are converted. This list and the precision and the scale parameters of the
decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC,
and STRING, a type is picked if it is in the specified list and if it supports
the precision and the scale. STRING supports all precision and scale values.
If none of the listed types supports the precision and the scale, the type
supporting the widest range in the specified list is picked, and if a value
exceeds the supported range when reading the data, an error will be thrown.
Example: Suppose the value of this field is ["NUMERIC", "BIGNUMERIC"]. If (
precision,scale) is: * (38,9) -> NUMERIC; * (39,9) -> BIGNUMERIC (NUMERIC
cannot hold 30 integer digits); * (38,10) -> BIGNUMERIC (NUMERIC cannot hold
10 fractional digits); * (76,38) -> BIGNUMERIC; * (77,38) -> BIGNUMERIC (error
if value exeeds supported range). This field cannot contain duplicate types.
The order of the types in this field is ignored. For example, ["BIGNUMERIC", "
NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes
precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["
NUMERIC"] for the other file formats.
Corresponds to the JSON property decimalTargetTypes
3156 3157 3158 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3156 def decimal_target_types @decimal_target_types end |
#file_set_spec_type ⇒ String
Optional. Specifies how source URIs are interpreted for constructing the file
set to load. By default source URIs are expanded against the underlying
storage. Other options include specifying manifest files. Only applicable to
object storage systems.
Corresponds to the JSON property fileSetSpecType
3164 3165 3166 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3164 def file_set_spec_type @file_set_spec_type end |
#google_sheets_options ⇒ Google::Apis::BigqueryV2::GoogleSheetsOptions
Options specific to Google Sheets data sources.
Corresponds to the JSON property googleSheetsOptions
3169 3170 3171 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3169 def @google_sheets_options end |
#hive_partitioning_options ⇒ Google::Apis::BigqueryV2::HivePartitioningOptions
Options for configuring hive partitioning detect.
Corresponds to the JSON property hivePartitioningOptions
3174 3175 3176 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3174 def @hive_partitioning_options end |
#ignore_unknown_values ⇒ Boolean Also known as: ignore_unknown_values?
Optional. Indicates if BigQuery should allow extra values that are not
represented in the table schema. If true, the extra values are ignored. If
false, records with extra columns are treated as bad records, and if there are
too many bad records, an invalid error is returned in the job result. The
default value is false. The sourceFormat property determines what BigQuery
treats as an extra value: CSV: Trailing columns JSON: Named values that don't
match any column names Google Cloud Bigtable: This setting is ignored. Google
Cloud Datastore backups: This setting is ignored. Avro: This setting is
ignored. ORC: This setting is ignored. Parquet: This setting is ignored.
Corresponds to the JSON property ignoreUnknownValues
3187 3188 3189 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3187 def ignore_unknown_values @ignore_unknown_values end |
#json_extension ⇒ String
Optional. Load option to be used together with source_format newline-delimited
JSON to indicate that a variant of JSON is being loaded. To load newline-
delimited GeoJSON, specify GEOJSON (and source_format must be set to
NEWLINE_DELIMITED_JSON).
Corresponds to the JSON property jsonExtension
3196 3197 3198 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3196 def json_extension @json_extension end |
#json_options ⇒ Google::Apis::BigqueryV2::JsonOptions
Json Options for load and make external tables.
Corresponds to the JSON property jsonOptions
3201 3202 3203 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3201 def @json_options end |
#max_bad_records ⇒ Fixnum
Optional. The maximum number of bad records that BigQuery can ignore when
reading data. If the number of bad records exceeds this value, an invalid
error is returned in the job result. The default value is 0, which requires
that all records are valid. This setting is ignored for Google Cloud Bigtable,
Google Cloud Datastore backups, Avro, ORC and Parquet formats.
Corresponds to the JSON property maxBadRecords
3210 3211 3212 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3210 def max_bad_records @max_bad_records end |
#metadata_cache_mode ⇒ String
Optional. Metadata Cache Mode for the table. Set this to enable caching of
metadata from external data source.
Corresponds to the JSON property metadataCacheMode
3216 3217 3218 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3216 def @metadata_cache_mode end |
#object_metadata ⇒ String
Optional. ObjectMetadata is used to create Object Tables. Object Tables
contain a listing of objects (with their metadata) found at the source_uris.
If ObjectMetadata is set, source_format should be omitted. Currently SIMPLE is
the only supported Object Metadata type.
Corresponds to the JSON property objectMetadata
3224 3225 3226 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3224 def @object_metadata end |
#parquet_options ⇒ Google::Apis::BigqueryV2::ParquetOptions
Parquet Options for load and make external tables.
Corresponds to the JSON property parquetOptions
3229 3230 3231 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3229 def @parquet_options end |
#reference_file_schema_uri ⇒ String
Optional. When creating an external table, the user can provide a reference
file with the table schema. This is enabled for the following formats: AVRO,
PARQUET, ORC.
Corresponds to the JSON property referenceFileSchemaUri
3236 3237 3238 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3236 def reference_file_schema_uri @reference_file_schema_uri end |
#schema ⇒ Google::Apis::BigqueryV2::TableSchema
Schema of a table
Corresponds to the JSON property schema
3241 3242 3243 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3241 def schema @schema end |
#source_format ⇒ String
[Required] The data format. For CSV files, specify "CSV". For Google sheets,
specify "GOOGLE_SHEETS". For newline-delimited JSON, specify "
NEWLINE_DELIMITED_JSON". For Avro files, specify "AVRO". For Google Cloud
Datastore backups, specify "DATASTORE_BACKUP". For Apache Iceberg tables,
specify "ICEBERG". For ORC files, specify "ORC". For Parquet files, specify "
PARQUET". [Beta] For Google Cloud Bigtable, specify "BIGTABLE".
Corresponds to the JSON property sourceFormat
3251 3252 3253 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3251 def source_format @source_format end |
#source_uris ⇒ Array<String>
[Required] The fully-qualified URIs that point to your data in Google Cloud.
For Google Cloud Storage URIs: Each URI can contain one '' wildcard character
and it must come after the 'bucket' name. Size limits related to load jobs
apply to external data sources. For Google Cloud Bigtable URIs: Exactly one
URI can be specified and it has be a fully specified and valid HTTPS URL for a
Google Cloud Bigtable table. For Google Cloud Datastore backups, exactly one
URI can be specified. Also, the '' wildcard character is not allowed.
Corresponds to the JSON property sourceUris
3262 3263 3264 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3262 def source_uris @source_uris end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 |
# File 'lib/google/apis/bigquery_v2/classes.rb', line 3269 def update!(**args) @autodetect = args[:autodetect] if args.key?(:autodetect) @avro_options = args[:avro_options] if args.key?(:avro_options) @bigtable_options = args[:bigtable_options] if args.key?(:bigtable_options) @compression = args[:compression] if args.key?(:compression) @connection_id = args[:connection_id] if args.key?(:connection_id) @csv_options = args[:csv_options] if args.key?(:csv_options) @decimal_target_types = args[:decimal_target_types] if args.key?(:decimal_target_types) @file_set_spec_type = args[:file_set_spec_type] if args.key?(:file_set_spec_type) @google_sheets_options = args[:google_sheets_options] if args.key?(:google_sheets_options) @hive_partitioning_options = args[:hive_partitioning_options] if args.key?(:hive_partitioning_options) @ignore_unknown_values = args[:ignore_unknown_values] if args.key?(:ignore_unknown_values) @json_extension = args[:json_extension] if args.key?(:json_extension) @json_options = args[:json_options] if args.key?(:json_options) @max_bad_records = args[:max_bad_records] if args.key?(:max_bad_records) @metadata_cache_mode = args[:metadata_cache_mode] if args.key?(:metadata_cache_mode) @object_metadata = args[:object_metadata] if args.key?(:object_metadata) @parquet_options = args[:parquet_options] if args.key?(:parquet_options) @reference_file_schema_uri = args[:reference_file_schema_uri] if args.key?(:reference_file_schema_uri) @schema = args[:schema] if args.key?(:schema) @source_format = args[:source_format] if args.key?(:source_format) @source_uris = args[:source_uris] if args.key?(:source_uris) end |