Class: Google::Apis::BigqueryV2::JobConfigurationLoad
- Inherits:
-
Object
- Object
- Google::Apis::BigqueryV2::JobConfigurationLoad
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/bigquery_v2/classes.rb,
generated/google/apis/bigquery_v2/representations.rb,
generated/google/apis/bigquery_v2/representations.rb
Instance Attribute Summary collapse
-
#allow_jagged_rows ⇒ Boolean
(also: #allow_jagged_rows?)
[Optional] Accept rows that are missing trailing optional columns.
-
#allow_quoted_newlines ⇒ Boolean
(also: #allow_quoted_newlines?)
Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file.
-
#autodetect ⇒ Boolean
(also: #autodetect?)
[Optional] Indicates if we should automatically infer the options and schema for CSV and JSON sources.
-
#clustering ⇒ Google::Apis::BigqueryV2::Clustering
[Beta] Clustering specification for the destination table.
-
#create_disposition ⇒ String
[Optional] Specifies whether the job is allowed to create new tables.
-
#destination_encryption_configuration ⇒ Google::Apis::BigqueryV2::EncryptionConfiguration
Custom encryption configuration (e.g., Cloud KMS keys).
-
#destination_table ⇒ Google::Apis::BigqueryV2::TableReference
[Required] The destination table to load the data into.
-
#destination_table_properties ⇒ Google::Apis::BigqueryV2::DestinationTableProperties
[Beta] [Optional] Properties with which to create the destination table if it is new.
-
#encoding ⇒ String
[Optional] The character encoding of the data.
-
#field_delimiter ⇒ String
[Optional] The separator for fields in a CSV file.
-
#hive_partitioning_mode ⇒ String
[Optional, Trusted Tester] If hive partitioning is enabled, which mode to use.
-
#hive_partitioning_options ⇒ Google::Apis::BigqueryV2::HivePartitioningOptions
[Optional, Trusted Tester] Options to configure hive partitioning support.
-
#ignore_unknown_values ⇒ Boolean
(also: #ignore_unknown_values?)
[Optional] Indicates if BigQuery should allow extra values that are not represented in the table schema.
-
#max_bad_records ⇒ Fixnum
[Optional] The maximum number of bad records that BigQuery can ignore when running the job.
-
#null_marker ⇒ String
[Optional] Specifies a string that represents a null value in a CSV file.
-
#projection_fields ⇒ Array<String>
If sourceFormat is set to "DATASTORE_BACKUP", indicates which entity properties to load into BigQuery from a Cloud Datastore backup.
-
#quote ⇒ String
[Optional] The value that is used to quote data sections in a CSV file.
-
#range_partitioning ⇒ Google::Apis::BigqueryV2::RangePartitioning
[TrustedTester] Range partitioning specification for this table.
-
#schema ⇒ Google::Apis::BigqueryV2::TableSchema
[Optional] The schema for the destination table.
-
#schema_inline ⇒ String
[Deprecated] The inline schema.
-
#schema_inline_format ⇒ String
[Deprecated] The format of the schemaInline property.
-
#schema_update_options ⇒ Array<String>
Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration.
-
#skip_leading_rows ⇒ Fixnum
[Optional] The number of rows at the top of a CSV file that BigQuery will skip when loading the data.
-
#source_format ⇒ String
[Optional] The format of the data files.
-
#source_uris ⇒ Array<String>
[Required] The fully-qualified URIs that point to your data in Google Cloud.
-
#time_partitioning ⇒ Google::Apis::BigqueryV2::TimePartitioning
Time-based partitioning specification for the destination table.
-
#use_avro_logical_types ⇒ Boolean
(also: #use_avro_logical_types?)
[Optional] If sourceFormat is set to "AVRO", indicates whether to enable interpreting logical types into their corresponding types (ie. TIMESTAMP), instead of only using their raw types (ie. INTEGER).
-
#write_disposition ⇒ String
[Optional] Specifies the action that occurs if the destination table already exists.
Instance Method Summary collapse
-
#initialize(**args) ⇒ JobConfigurationLoad
constructor
A new instance of JobConfigurationLoad.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ JobConfigurationLoad
Returns a new instance of JobConfigurationLoad
2453 2454 2455 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2453 def initialize(**args) update!(**args) end |
Instance Attribute Details
#allow_jagged_rows ⇒ Boolean Also known as: allow_jagged_rows?
[Optional] Accept rows that are missing trailing optional columns. The missing
values are treated as nulls. If false, records with missing trailing columns
are treated as bad records, and if there are too many bad records, an invalid
error is returned in the job result. The default value is false. Only
applicable to CSV, ignored for other formats.
Corresponds to the JSON property allowJaggedRows
2232 2233 2234 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2232 def allow_jagged_rows @allow_jagged_rows end |
#allow_quoted_newlines ⇒ Boolean Also known as: allow_quoted_newlines?
Indicates if BigQuery should allow quoted data sections that contain newline
characters in a CSV file. The default value is false.
Corresponds to the JSON property allowQuotedNewlines
2239 2240 2241 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2239 def allow_quoted_newlines @allow_quoted_newlines end |
#autodetect ⇒ Boolean Also known as: autodetect?
[Optional] Indicates if we should automatically infer the options and schema
for CSV and JSON sources.
Corresponds to the JSON property autodetect
2246 2247 2248 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2246 def autodetect @autodetect end |
#clustering ⇒ Google::Apis::BigqueryV2::Clustering
[Beta] Clustering specification for the destination table. Must be specified
with time-based partitioning, data in the table will be first partitioned and
subsequently clustered.
Corresponds to the JSON property clustering
2254 2255 2256 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2254 def clustering @clustering end |
#create_disposition ⇒ String
[Optional] Specifies whether the job is allowed to create new tables. The
following values are supported: CREATE_IF_NEEDED: If the table does not exist,
BigQuery creates the table. CREATE_NEVER: The table must already exist. If it
does not, a 'notFound' error is returned in the job result. The default value
is CREATE_IF_NEEDED. Creation, truncation and append actions occur as one
atomic update upon job completion.
Corresponds to the JSON property createDisposition
2264 2265 2266 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2264 def create_disposition @create_disposition end |
#destination_encryption_configuration ⇒ Google::Apis::BigqueryV2::EncryptionConfiguration
Custom encryption configuration (e.g., Cloud KMS keys).
Corresponds to the JSON property destinationEncryptionConfiguration
2269 2270 2271 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2269 def destination_encryption_configuration @destination_encryption_configuration end |
#destination_table ⇒ Google::Apis::BigqueryV2::TableReference
[Required] The destination table to load the data into.
Corresponds to the JSON property destinationTable
2274 2275 2276 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2274 def destination_table @destination_table end |
#destination_table_properties ⇒ Google::Apis::BigqueryV2::DestinationTableProperties
[Beta] [Optional] Properties with which to create the destination table if it
is new.
Corresponds to the JSON property destinationTableProperties
2280 2281 2282 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2280 def destination_table_properties @destination_table_properties end |
#encoding ⇒ String
[Optional] The character encoding of the data. The supported values are UTF-8
or ISO-8859-1. The default value is UTF-8. BigQuery decodes the data after the
raw, binary data has been split using the values of the quote and
fieldDelimiter properties.
Corresponds to the JSON property encoding
2288 2289 2290 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2288 def encoding @encoding end |
#field_delimiter ⇒ String
[Optional] The separator for fields in a CSV file. The separator can be any
ISO-8859-1 single-byte character. To use a character in the range 128-255, you
must encode the character as UTF8. BigQuery converts the string to ISO-8859-1
encoding, and then uses the first byte of the encoded string to split the data
in its raw, binary state. BigQuery also supports the escape sequence "\t" to
specify a tab separator. The default value is a comma (',').
Corresponds to the JSON property fieldDelimiter
2298 2299 2300 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2298 def field_delimiter @field_delimiter end |
#hive_partitioning_mode ⇒ String
[Optional, Trusted Tester] If hive partitioning is enabled, which mode to use.
Two modes are supported: - AUTO: automatically infer partition key name(s) and
type(s). - STRINGS: automatic infer partition key name(s). All types are
strings. Not all storage formats support hive partitioning -- requesting hive
partitioning on an unsupported format will lead to an error.
Corresponds to the JSON property hivePartitioningMode
2307 2308 2309 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2307 def hive_partitioning_mode @hive_partitioning_mode end |
#hive_partitioning_options ⇒ Google::Apis::BigqueryV2::HivePartitioningOptions
[Optional, Trusted Tester] Options to configure hive partitioning support.
Corresponds to the JSON property hivePartitioningOptions
2312 2313 2314 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2312 def @hive_partitioning_options end |
#ignore_unknown_values ⇒ Boolean Also known as: ignore_unknown_values?
[Optional] Indicates if BigQuery should allow extra values that are not
represented in the table schema. If true, the extra values are ignored. If
false, records with extra columns are treated as bad records, and if there are
too many bad records, an invalid error is returned in the job result. The
default value is false. The sourceFormat property determines what BigQuery
treats as an extra value: CSV: Trailing columns JSON: Named values that don't
match any column names
Corresponds to the JSON property ignoreUnknownValues
2323 2324 2325 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2323 def ignore_unknown_values @ignore_unknown_values end |
#max_bad_records ⇒ Fixnum
[Optional] The maximum number of bad records that BigQuery can ignore when
running the job. If the number of bad records exceeds this value, an invalid
error is returned in the job result. This is only valid for CSV and JSON. The
default value is 0, which requires that all records are valid.
Corresponds to the JSON property maxBadRecords
2332 2333 2334 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2332 def max_bad_records @max_bad_records end |
#null_marker ⇒ String
[Optional] Specifies a string that represents a null value in a CSV file. For
example, if you specify "\N", BigQuery interprets "\N" as a null value when
loading a CSV file. The default value is the empty string. If you set this
property to a custom value, BigQuery throws an error if an empty string is
present for all data types except for STRING and BYTE. For STRING and BYTE
columns, BigQuery interprets the empty string as an empty value.
Corresponds to the JSON property nullMarker
2342 2343 2344 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2342 def null_marker @null_marker end |
#projection_fields ⇒ Array<String>
If sourceFormat is set to "DATASTORE_BACKUP", indicates which entity
properties to load into BigQuery from a Cloud Datastore backup. Property names
are case sensitive and must be top-level properties. If no properties are
specified, BigQuery loads all properties. If any named property isn't found in
the Cloud Datastore backup, an invalid error is returned in the job result.
Corresponds to the JSON property projectionFields
2351 2352 2353 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2351 def projection_fields @projection_fields end |
#quote ⇒ String
[Optional] The value that is used to quote data sections in a CSV file.
BigQuery converts the string to ISO-8859-1 encoding, and then uses the first
byte of the encoded string to split the data in its raw, binary state. The
default value is a double-quote ('"'). If your data does not contain quoted
sections, set the property value to an empty string. If your data contains
quoted newline characters, you must also set the allowQuotedNewlines property
to true.
Corresponds to the JSON property quote
2362 2363 2364 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2362 def quote @quote end |
#range_partitioning ⇒ Google::Apis::BigqueryV2::RangePartitioning
[TrustedTester] Range partitioning specification for this table. Only one of
timePartitioning and rangePartitioning should be specified.
Corresponds to the JSON property rangePartitioning
2368 2369 2370 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2368 def range_partitioning @range_partitioning end |
#schema ⇒ Google::Apis::BigqueryV2::TableSchema
[Optional] The schema for the destination table. The schema can be omitted if
the destination table already exists, or if you're loading data from Google
Cloud Datastore.
Corresponds to the JSON property schema
2375 2376 2377 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2375 def schema @schema end |
#schema_inline ⇒ String
[Deprecated] The inline schema. For CSV schemas, specify as "Field1:Type1[,
Field2:Type2]*". For example, "foo:STRING, bar:INTEGER, baz:FLOAT".
Corresponds to the JSON property schemaInline
2381 2382 2383 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2381 def schema_inline @schema_inline end |
#schema_inline_format ⇒ String
[Deprecated] The format of the schemaInline property.
Corresponds to the JSON property schemaInlineFormat
2386 2387 2388 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2386 def schema_inline_format @schema_inline_format end |
#schema_update_options ⇒ Array<String>
Allows the schema of the destination table to be updated as a side effect of
the load job if a schema is autodetected or supplied in the job configuration.
Schema update options are supported in two cases: when writeDisposition is
WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination
table is a partition of a table, specified by partition decorators. For normal
tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the
following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable
field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field
in the original schema to nullable.
Corresponds to the JSON property schemaUpdateOptions
2399 2400 2401 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2399 def @schema_update_options end |
#skip_leading_rows ⇒ Fixnum
[Optional] The number of rows at the top of a CSV file that BigQuery will skip
when loading the data. The default value is 0. This property is useful if you
have header rows in the file that should be skipped.
Corresponds to the JSON property skipLeadingRows
2406 2407 2408 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2406 def skip_leading_rows @skip_leading_rows end |
#source_format ⇒ String
[Optional] The format of the data files. For CSV files, specify "CSV". For
datastore backups, specify "DATASTORE_BACKUP". For newline-delimited JSON,
specify "NEWLINE_DELIMITED_JSON". For Avro, specify "AVRO". For parquet,
specify "PARQUET". For orc, specify "ORC". The default value is CSV.
Corresponds to the JSON property sourceFormat
2414 2415 2416 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2414 def source_format @source_format end |
#source_uris ⇒ Array<String>
[Required] The fully-qualified URIs that point to your data in Google Cloud.
For Google Cloud Storage URIs: Each URI can contain one '' wildcard character
and it must come after the 'bucket' name. Size limits related to load jobs
apply to external data sources. For Google Cloud Bigtable URIs: Exactly one
URI can be specified and it has be a fully specified and valid HTTPS URL for a
Google Cloud Bigtable table. For Google Cloud Datastore backups: Exactly one
URI can be specified. Also, the '' wildcard character is not allowed.
Corresponds to the JSON property sourceUris
2425 2426 2427 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2425 def source_uris @source_uris end |
#time_partitioning ⇒ Google::Apis::BigqueryV2::TimePartitioning
Time-based partitioning specification for the destination table. Only one of
timePartitioning and rangePartitioning should be specified.
Corresponds to the JSON property timePartitioning
2431 2432 2433 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2431 def time_partitioning @time_partitioning end |
#use_avro_logical_types ⇒ Boolean Also known as: use_avro_logical_types?
[Optional] If sourceFormat is set to "AVRO", indicates whether to enable
interpreting logical types into their corresponding types (ie. TIMESTAMP),
instead of only using their raw types (ie. INTEGER).
Corresponds to the JSON property useAvroLogicalTypes
2438 2439 2440 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2438 def use_avro_logical_types @use_avro_logical_types end |
#write_disposition ⇒ String
[Optional] Specifies the action that occurs if the destination table already
exists. The following values are supported: WRITE_TRUNCATE: If the table
already exists, BigQuery overwrites the table data. WRITE_APPEND: If the table
already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the
table already exists and contains data, a 'duplicate' error is returned in the
job result. The default value is WRITE_APPEND. Each action is atomic and only
occurs if BigQuery is able to complete the job successfully. Creation,
truncation and append actions occur as one atomic update upon job completion.
Corresponds to the JSON property writeDisposition
2451 2452 2453 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2451 def write_disposition @write_disposition end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 |
# File 'generated/google/apis/bigquery_v2/classes.rb', line 2458 def update!(**args) @allow_jagged_rows = args[:allow_jagged_rows] if args.key?(:allow_jagged_rows) @allow_quoted_newlines = args[:allow_quoted_newlines] if args.key?(:allow_quoted_newlines) @autodetect = args[:autodetect] if args.key?(:autodetect) @clustering = args[:clustering] if args.key?(:clustering) @create_disposition = args[:create_disposition] if args.key?(:create_disposition) @destination_encryption_configuration = args[:destination_encryption_configuration] if args.key?(:destination_encryption_configuration) @destination_table = args[:destination_table] if args.key?(:destination_table) @destination_table_properties = args[:destination_table_properties] if args.key?(:destination_table_properties) @encoding = args[:encoding] if args.key?(:encoding) @field_delimiter = args[:field_delimiter] if args.key?(:field_delimiter) @hive_partitioning_mode = args[:hive_partitioning_mode] if args.key?(:hive_partitioning_mode) @hive_partitioning_options = args[:hive_partitioning_options] if args.key?(:hive_partitioning_options) @ignore_unknown_values = args[:ignore_unknown_values] if args.key?(:ignore_unknown_values) @max_bad_records = args[:max_bad_records] if args.key?(:max_bad_records) @null_marker = args[:null_marker] if args.key?(:null_marker) @projection_fields = args[:projection_fields] if args.key?(:projection_fields) @quote = args[:quote] if args.key?(:quote) @range_partitioning = args[:range_partitioning] if args.key?(:range_partitioning) @schema = args[:schema] if args.key?(:schema) @schema_inline = args[:schema_inline] if args.key?(:schema_inline) @schema_inline_format = args[:schema_inline_format] if args.key?(:schema_inline_format) @schema_update_options = args[:schema_update_options] if args.key?(:schema_update_options) @skip_leading_rows = args[:skip_leading_rows] if args.key?(:skip_leading_rows) @source_format = args[:source_format] if args.key?(:source_format) @source_uris = args[:source_uris] if args.key?(:source_uris) @time_partitioning = args[:time_partitioning] if args.key?(:time_partitioning) @use_avro_logical_types = args[:use_avro_logical_types] if args.key?(:use_avro_logical_types) @write_disposition = args[:write_disposition] if args.key?(:write_disposition) end |