As of January 1, 2020 this library no longer supports Python 2 on the latest released version. Library versions released prior to that date will continue to be available. For more information please visit Python 2 support on Google Cloud.

google.cloud.bigquery.job.LoadJobConfig

class google.cloud.bigquery.job.LoadJobConfig(**kwargs)

Configuration options for load jobs.

All properties in this class are optional. Values which are None -> server defaults. Set properties on the constructed configuration by using the property name as the name of a keyword argument.

__init__(**kwargs)

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__(**kwargs)	Initialize self.
from_api_repr(resource)	Factory: construct a job configuration given its API representation
to_api_repr()	Build an API representation of the job config.

Attributes

allow_jagged_rows	Allow missing trailing optional columns (CSV only).
allow_quoted_newlines	Allow quoted data containing newline characters (CSV only).
autodetect	Automatically infer the schema from a sample of the data.
clustering_fields	Fields defining clustering for the table
create_disposition	Specifies behavior for creating tables.
destination_encryption_configuration	Custom encryption configuration for the destination table.
destination_table_description	Name given to destination table.
destination_table_friendly_name	Name given to destination table.
encoding	The character encoding of the data.
field_delimiter	The separator for fields in a CSV file.
hive_partitioning	[Beta] When set, it configures hive partitioning support.
ignore_unknown_values	Ignore extra values not represented in the table schema.
labels	Labels for the job.
max_bad_records	Number of invalid rows to ignore.
null_marker	Represents a null value (CSV only).
quote_character	Character used to quote data sections (CSV only).
range_partitioning	Optional[google.cloud.bigquery.table.RangePartitioning]: Configures range-based partitioning for destination table.
schema	Schema of the destination table.
schema_update_options	Specifies updates to the destination table schema to allow as a side effect of the load job.
skip_leading_rows	Number of rows to skip when reading data (CSV only).
source_format	File format of the data.
time_partitioning	Specifies time-based partitioning for the destination table.
use_avro_logical_types	For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types (e.g.
write_disposition	Action that occurs if the destination table already exists.

property allow_jagged_rows

Allow missing trailing optional columns (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.allow_jagged_rows

Type

Optional[bool]

property allow_quoted_newlines

Allow quoted data containing newline characters (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.allow_quoted_newlines

Type

Optional[bool]

property autodetect

Automatically infer the schema from a sample of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.autodetect

Type

Optional[bool]

property clustering_fields

Fields defining clustering for the table

(Defaults to None).

Clustering fields are immutable after table creation.

Note

As of 2018-06-29, clustering fields cannot be set on a table which does not also have time partioning defined.

Type

Optional[List[str]]

property create_disposition

Specifies behavior for creating tables.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.create_disposition

Type

Optional[google.cloud.bigquery.job.CreateDisposition]

property destination_encryption_configuration

Custom encryption configuration for the destination table.

Custom encryption configuration (e.g., Cloud KMS keys) or None if using default encryption.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.destination_encryption_configuration

Type

Optional[google.cloud.bigquery.encryption_configuration.EncryptionConfiguration]

property destination_table_description

Name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#DestinationTableProperties.FIELDS.description

Type

Optional[str]

property destination_table_friendly_name

Name given to destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#DestinationTableProperties.FIELDS.friendly_name

Type

Optional[str]

property encoding

The character encoding of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.encoding

Type

Optional[google.cloud.bigquery.job.Encoding]

property field_delimiter

The separator for fields in a CSV file.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.field_delimiter

Type

Optional[str]

classmethod from_api_repr(resource)

Factory: construct a job configuration given its API representation

Parameters

resource (Dict) – An extract job configuration in the same representation as is returned from the API.

Returns

Configuration parsed from resource.

Return type

google.cloud.bigquery.job._JobConfig

property hive_partitioning

[Beta] When set, it configures hive partitioning support.

Note

Experimental. This feature is experimental and might change or have limited support.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.hive_partitioning_options

Type

Optional[HivePartitioningOptions]

property ignore_unknown_values

Ignore extra values not represented in the table schema.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.ignore_unknown_values

Type

Optional[bool]

property labels

Labels for the job.

This method always returns a dict. To change a job’s labels, modify the dict, then call Client.update_job. To delete a label, set its value to None before updating.

Raises

ValueError – If value type is invalid.

Type

Dict[str, str]

property max_bad_records

Number of invalid rows to ignore.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.max_bad_records

Type

Optional[int]

property null_marker

Represents a null value (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.null_marker

Type

Optional[str]

property quote_character

Character used to quote data sections (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.quote

Type

Optional[str]

property range_partitioning

Optional[google.cloud.bigquery.table.RangePartitioning]: Configures range-based partitioning for destination table.

Note

Beta. The integer range partitioning feature is in a pre-release state and might change or have limited support.

Only specify at most one of time_partitioning or range_partitioning.

Raises

ValueError – If the value is not RangePartitioning or None.

property schema

Schema of the destination table.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.schema

Type

Optional[Sequence[Union[ SchemaField, Mapping[str, Any] ]]]

property schema_update_options

Specifies updates to the destination table schema to allow as a side effect of the load job.

Type

Optional[List[google.cloud.bigquery.job.SchemaUpdateOption]]

property skip_leading_rows

Number of rows to skip when reading data (CSV only).

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.skip_leading_rows

Type

Optional[int]

property source_format

File format of the data.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.source_format

Type

Optional[google.cloud.bigquery.job.SourceFormat]

property time_partitioning

Specifies time-based partitioning for the destination table.

Only specify at most one of time_partitioning or range_partitioning.

Type

Optional[google.cloud.bigquery.table.TimePartitioning]

to_api_repr()

Build an API representation of the job config.

Returns

A dictionary in the format used by the BigQuery API.

Return type

Dict

property use_avro_logical_types

For loads of Avro data, governs whether Avro logical types are converted to their corresponding BigQuery types (e.g. TIMESTAMP) rather than raw types (e.g. INTEGER).

Type

Optional[bool]

property write_disposition

Action that occurs if the destination table already exists.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfigurationLoad.FIELDS.write_disposition

Type

Optional[google.cloud.bigquery.job.WriteDisposition]