As of January 1, 2020 this library no longer supports Python 2 on the latest released version. Library versions released prior to that date will continue to be available. For more information please visit Python 2 support on Google Cloud.

google.cloud.bigquery.job.LoadJobConfig

class google.cloud.bigquery.job.LoadJobConfig(**kwargs)[source]

Configuration options for load jobs.

All properties in this class are optional. Values which are None -> server defaults. Set properties on the constructed configuration by using the property name as the name of a keyword argument.

__init__(**kwargs)[source]

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__(**kwargs)

Initialize self.

from_api_repr(resource)

Factory: construct a job configuration given its API representation

to_api_repr()

Build an API representation of the job config.

Attributes

allow_jagged_rows

Allow missing trailing optional columns (CSV only).

allow_quoted_newlines

Allow quoted data containing newline characters (CSV only).

autodetect

Automatically infer the schema from a sample of the data.

clustering_fields

Fields defining clustering for the table

create_disposition

Specifies behavior for creating tables.

destination_encryption_configuration

Custom encryption configuration for the destination table.

destination_table_description

Name given to destination table.

destination_table_friendly_name

Name given to destination table.

encoding

The character encoding of the data.

field_delimiter

The separator for fields in a CSV file.

hive_partitioning

[Beta] When set, it configures hive partitioning support.

ignore_unknown_values

Ignore extra values not represented in the table schema.

labels

Labels for the job.

max_bad_records

Number of invalid rows to ignore.

null_marker

Represents a null value (CSV only).

parquet_options

Additional

quote_character

Character used to quote data sections (CSV only).

range_partitioning

Optional[google.cloud.bigquery.table.RangePartitioning]: Configures range-based partitioning for destination table.

schema

Schema of the destination table.

schema_update_options

Specifies updates to the destination table schema to allow as a side effect of the load job.

skip_leading_rows

Number of rows to skip when reading data (CSV only).

source_format

File format of the data.

time_partitioning

Specifies time-based partitioning for the destination table.

use_avro_logical_types

For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types (e.g.

write_disposition

Action that occurs if the destination table already exists.

__setattr__(name, value)

Override to be able to raise error if an unknown property is being set

property allow_jagged_rows

Allow missing trailing optional columns (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.allow_jagged_rows

Type

Optional[bool]

property allow_quoted_newlines

Allow quoted data containing newline characters (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.allow_quoted_newlines

Type

Optional[bool]

property autodetect

Automatically infer the schema from a sample of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.autodetect

Type

Optional[bool]

property clustering_fields

Fields defining clustering for the table

(Defaults to None).

Clustering fields are immutable after table creation.

Note

BigQuery supports clustering for both partitioned and non-partitioned tables.

Type

Optional[List[str]]

property create_disposition

Specifies behavior for creating tables.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.create_disposition

Type

Optional[google.cloud.bigquery.job.CreateDisposition]

property destination_encryption_configuration

Custom encryption configuration for the destination table.

Custom encryption configuration (e.g., Cloud KMS keys) or None if using default encryption.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.destination_encryption_configuration

Type

Optional[google.cloud.bigquery.encryption_configuration.EncryptionConfiguration]

property destination_table_description

Name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#DestinationTableProperties.FIELDS.description

Type

Optional[str]

property destination_table_friendly_name

Name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#DestinationTableProperties.FIELDS.friendly_name

Type

Optional[str]

property encoding

The character encoding of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.encoding

Type

Optional[google.cloud.bigquery.job.Encoding]

property field_delimiter

The separator for fields in a CSV file.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.field_delimiter

Type

Optional[str]

classmethod from_api_repr(resource: dict)google.cloud.bigquery.job.base._JobConfig

Factory: construct a job configuration given its API representation

Parameters

resource (Dict) – A job configuration in the same representation as is returned from the API.

Returns

Configuration parsed from resource.

Return type

google.cloud.bigquery.job._JobConfig

property hive_partitioning

[Beta] When set, it configures hive partitioning support.

Note

Experimental. This feature is experimental and might change or have limited support.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.hive_partitioning_options

Type

Optional[HivePartitioningOptions]

property ignore_unknown_values

Ignore extra values not represented in the table schema.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.ignore_unknown_values

Type

Optional[bool]

property labels

Labels for the job.

This method always returns a dict. Once a job has been created on the server, its labels cannot be modified anymore.

Raises

ValueError – If value type is invalid.

Type

Dict[str, str]

property max_bad_records

Number of invalid rows to ignore.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.max_bad_records

Type

Optional[int]

property null_marker

Represents a null value (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.null_marker

Type

Optional[str]

property parquet_options
Additional

properties to set if sourceFormat is set to PARQUET.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.parquet_options

Type

Optional[google.cloud.bigquery.format_options.ParquetOptions]

property quote_character

Character used to quote data sections (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.quote

Type

Optional[str]

property range_partitioning

Optional[google.cloud.bigquery.table.RangePartitioning]: Configures range-based partitioning for destination table.

Note

Beta. The integer range partitioning feature is in a pre-release state and might change or have limited support.

Only specify at most one of time_partitioning or range_partitioning.

Raises

ValueError – If the value is not RangePartitioning or None.

property schema

Schema of the destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.schema

Type

Optional[Sequence[Union[ SchemaField, Mapping[str, Any] ]]]

property schema_update_options

Specifies updates to the destination table schema to allow as a side effect of the load job.

Type

Optional[List[google.cloud.bigquery.job.SchemaUpdateOption]]

property skip_leading_rows

Number of rows to skip when reading data (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.skip_leading_rows

Type

Optional[int]

property source_format

File format of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.source_format

Type

Optional[google.cloud.bigquery.job.SourceFormat]

property time_partitioning

Specifies time-based partitioning for the destination table.

Only specify at most one of time_partitioning or range_partitioning.

Type

Optional[google.cloud.bigquery.table.TimePartitioning]

to_api_repr()dict

Build an API representation of the job config.

Returns

A dictionary in the format used by the BigQuery API.

Return type

Dict

property use_avro_logical_types

For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types (e.g. TIMESTAMP) rather than raw types (e.g. INTEGER).

Type

Optional[bool]

property write_disposition

Action that occurs if the destination table already exists.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.write_disposition

Type

Optional[google.cloud.bigquery.job.WriteDisposition]