As of January 1, 2020 this library no longer supports Python 2 on the latest released version. Library versions released prior to that date will continue to be available. For more information please visit Python 2 support on Google Cloud.

google.cloud.bigquery.external_config.ExternalConfig

class google.cloud.bigquery.external_config.ExternalConfig(source_format)

Description of an external data source.

Parameters

source_format (ExternalSourceFormat) – See source_format.

__init__(source_format) → None

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__(source_format)	Initialize self.
from_api_repr(resource)	Factory: construct an ExternalConfig instance given its API representation.
to_api_repr()	Build an API representation of this object.

Attributes

autodetect	If True, try to detect schema and format options automatically.
avro_options	Additional properties to set if sourceFormat is set to AVRO.
bigtable_options	Additional properties to set if sourceFormat is set to BIGTABLE.
compression	The compression type of the data source.
connection_id	[Experimental] ID of a BigQuery Connection API resource.
csv_options	Additional properties to set if sourceFormat is set to CSV.
decimal_target_types	Possible SQL data types to which the source decimal values are converted.
google_sheets_options	Additional properties to set if sourceFormat is set to GOOGLE_SHEETS.
hive_partitioning	[Beta] When set, it configures hive partitioning support.
ignore_unknown_values	If True, extra values that are not represented in the table schema are ignored.
max_bad_records	The maximum number of bad records that BigQuery can ignore when reading data.
options	Source-specific options.
parquet_options	Additional properties to set if sourceFormat is set to PARQUET.
reference_file_schema_uri	Optional[str]: When creating an external table, the user can provide a reference file with the table schema.
schema	The schema for the data.
source_format	ExternalSourceFormat: Format of external source.
source_uris	URIs that point to your data in Google Cloud.

property autodetect

If True, try to detect schema and format options automatically.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.autodetect

Type

bool

property avro_options: Optional[google.cloud.bigquery.format_options.AvroOptions]

Additional properties to set if sourceFormat is set to AVRO.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.avro_options

property bigtable_options: Optional[google.cloud.bigquery.external_config.BigtableOptions]

Additional properties to set if sourceFormat is set to BIGTABLE.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.bigtable_options

property compression

The compression type of the data source.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.compression

Type

str

property connection_id

[Experimental] ID of a BigQuery Connection API resource.

Warning

This feature is experimental. Pre-GA features may have limited support, and changes to pre-GA features may not be compatible with other pre-GA versions.

Type

Optional[str]

property csv_options: Optional[google.cloud.bigquery.external_config.CSVOptions]

Additional properties to set if sourceFormat is set to CSV.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.csv_options

property decimal_target_types: Optional[FrozenSet[str]]

Possible SQL data types to which the source decimal values are converted.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.decimal_target_types

New in version 2.21.0.

classmethod from_api_repr(resource: dict) → google.cloud.bigquery.external_config.ExternalConfig

Factory: construct an ExternalConfig instance given its API representation.

Parameters

resource (Dict[str, Any]) – Definition of an ExternalConfig instance in the same representation as is returned from the API.

Returns

Configuration parsed from resource.

Return type

ExternalConfig

property google_sheets_options: Optional[google.cloud.bigquery.external_config.GoogleSheetsOptions]

Additional properties to set if sourceFormat is set to GOOGLE_SHEETS.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.google_sheets_options

property hive_partitioning

[Beta] When set, it configures hive partitioning support.

Note

Experimental. This feature is experimental and might change or have limited support.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.hive_partitioning_options

Type

Optional[HivePartitioningOptions]

property ignore_unknown_values

If True, extra values that are not represented in the table schema are ignored. Defaults to False.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.ignore_unknown_values

Type

bool

property max_bad_records

The maximum number of bad records that BigQuery can ignore when reading data.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.max_bad_records

Type

int

property options: Optional[Union[google.cloud.bigquery.format_options.AvroOptions, google.cloud.bigquery.external_config.BigtableOptions, google.cloud.bigquery.external_config.CSVOptions, google.cloud.bigquery.external_config.GoogleSheetsOptions, google.cloud.bigquery.format_options.ParquetOptions]]

Source-specific options.

property parquet_options: Optional[google.cloud.bigquery.format_options.ParquetOptions]

Additional properties to set if sourceFormat is set to PARQUET.

See: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.parquet_options

property reference_file_schema_uri

Optional[str]: When creating an external table, the user can provide a reference file with the table schema. This is enabled for the following formats:

AVRO, PARQUET, ORC

property schema

The schema for the data.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.schema

Type

List[SchemaField]

property source_format

ExternalSourceFormat: Format of external source.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.source_format

property source_uris

URIs that point to your data in Google Cloud.

See https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#ExternalDataConfiguration.FIELDS.source_uris

Type

List[str]

to_api_repr() → dict

Build an API representation of this object.

Returns

A dictionary in the format used by the BigQuery API.

Return type

Dict[str, Any]