com.google.cloud.bigquery

Class ExternalTableDefinition

java.lang.Object

com.google.cloud.bigquery.TableDefinition

com.google.cloud.bigquery.ExternalTableDefinition

All Implemented Interfaces:

Serializable

public abstract class ExternalTableDefinition
extends TableDefinition

Google BigQuery external table definition. BigQuery's external tables are tables whose data reside outside of BigQuery but can be queried as normal BigQuery tables. External tables are experimental and might be subject to change or removed.

See Also:

Federated Data Sources , Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ExternalTableDefinition.Builder

Nested classes/interfaces inherited from class com.google.cloud.bigquery.TableDefinition

TableDefinition.Type

Constructor Summary

Constructors

Constructor and Description
ExternalTableDefinition()

Method Summary

Modifier and Type	Method and Description
abstract Boolean	getAutodetect() [Experimental] Returns whether automatic detection of schema and format options should be performed.
abstract String	getCompression() Returns the compression type of the data source.
abstract String	getConnectionId() Returns the connection ID used to connect to external data source.
abstract com.google.common.collect.ImmutableList<String>	getDecimalTargetTypes()
<F extends FormatOptions> F	getFormatOptions() Returns the source format, and possibly some parsing options, of the external data.
HivePartitioningOptions	getHivePartitioningOptions() [Experimental] Returns the HivePartitioningOptions when the data layout follows Hive partitioning convention
abstract Boolean	getIgnoreUnknownValues()
abstract Integer	getMaxBadRecords() Returns the maximum number of bad records that BigQuery can ignore when reading data.
abstract String	getReferenceFileSchemaUri()
List<String>	getSourceUris() Returns the fully-qualified URIs that point to your data in Google Cloud Storage.
abstract com.google.common.collect.ImmutableList<String>	getSourceUrisImmut()
Boolean	ignoreUnknownValues() Returns whether BigQuery should allow extra values that are not represented in the table schema.
static ExternalTableDefinition.Builder	newBuilder(List<String> sourceUris, Schema schema, FormatOptions format) Creates a builder for an ExternalTableDefinition object.
static ExternalTableDefinition.Builder	newBuilder(String sourceUri, FormatOptions format) Creates a builder for an ExternalTableDefinition object.
static ExternalTableDefinition.Builder	newBuilder(String sourceUri, Schema schema, FormatOptions format) Creates a builder for an ExternalTableDefinition object.
static ExternalTableDefinition	of(List<String> sourceUris, Schema schema, FormatOptions format) Creates an ExternalTableDefinition object.
static ExternalTableDefinition	of(String sourceUri, FormatOptions format) Creates a builder for an ExternalTableDefinition object.
static ExternalTableDefinition	of(String sourceUri, Schema schema, FormatOptions format) Creates an ExternalTableDefinition object.
abstract ExternalTableDefinition.Builder	toBuilder() Returns a builder for the ExternalTableDefinition object.

Methods inherited from class com.google.cloud.bigquery.TableDefinition

getSchema, getType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ExternalTableDefinition

public ExternalTableDefinition()

Method Detail

getCompression

@Nullable
public abstract String getCompression()

Returns the compression type of the data source.

See Also:

Compression

getConnectionId

@Nullable
public abstract String getConnectionId()

Returns the connection ID used to connect to external data source.

See Also:

ConnectionId

ignoreUnknownValues

@Nullable
public Boolean ignoreUnknownValues()

Returns whether BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The value of getFormatOptions() determines what BigQuery treats as an extra value.

See Also:

Ignore Unknown Values

getIgnoreUnknownValues

@Nullable
public abstract Boolean getIgnoreUnknownValues()

getMaxBadRecords

@Nullable
public abstract Integer getMaxBadRecords()

Returns the maximum number of bad records that BigQuery can ignore when reading data. If the number of bad records exceeds this value, an invalid error is returned in the job result.

getSourceUris

@Nullable
public List<String> getSourceUris()

Returns the fully-qualified URIs that point to your data in Google Cloud Storage. Each URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

See Also:

Quota

getSourceUrisImmut

@Nullable
public abstract com.google.common.collect.ImmutableList<String> getSourceUrisImmut()

getFormatOptions

@Nullable
public <F extends FormatOptions> F getFormatOptions()

Returns the source format, and possibly some parsing options, of the external data. Supported formats are CSV and NEWLINE_DELIMITED_JSON.

getDecimalTargetTypes

@Nullable
public abstract com.google.common.collect.ImmutableList<String> getDecimalTargetTypes()

getAutodetect

@Nullable
public abstract Boolean getAutodetect()

[Experimental] Returns whether automatic detection of schema and format options should be performed.

getReferenceFileSchemaUri

@Nullable
public abstract String getReferenceFileSchemaUri()

getHivePartitioningOptions

@Nullable
public HivePartitioningOptions getHivePartitioningOptions()

[Experimental] Returns the HivePartitioningOptions when the data layout follows Hive partitioning convention

toBuilder

public abstract ExternalTableDefinition.Builder toBuilder()

Returns a builder for the ExternalTableDefinition object.

Specified by:

toBuilder in class TableDefinition

newBuilder

public static ExternalTableDefinition.Builder newBuilder(List<String> sourceUris,
                                                         Schema schema,
                                                         FormatOptions format)

Creates a builder for an ExternalTableDefinition object.

Parameters:

sourceUris - the fully-qualified URIs that point to your data in Google Cloud Storage. Each URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

schema - the schema for the external data

format - the source format of the external data

Returns:

a builder for an ExternalTableDefinition object given source URIs, schema and format

See Also:

Quota, Source Format

newBuilder

public static ExternalTableDefinition.Builder newBuilder(String sourceUri,
                                                         Schema schema,
                                                         FormatOptions format)

Creates a builder for an ExternalTableDefinition object.

Parameters:

sourceUri - a fully-qualified URI that points to your data in Google Cloud Storage. The URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources.

schema - the schema for the external data

format - the source format of the external data

Returns:

a builder for an ExternalTableDefinition object given source URI, schema and format

See Also:

Quota, Source Format

newBuilder

public static ExternalTableDefinition.Builder newBuilder(String sourceUri,
                                                         FormatOptions format)

Creates a builder for an ExternalTableDefinition object.

Parameters:

sourceUri - the fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

format - the source format of the external data

Returns:

a builder for an ExternalTableDefinition object given source URIs and format

See Also:

Quota, Source Format

of

public static ExternalTableDefinition of(List<String> sourceUris,
                                         Schema schema,
                                         FormatOptions format)

Creates an ExternalTableDefinition object.

Parameters:

sourceUris - the fully-qualified URIs that point to your data in Google Cloud Storage. Each URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

schema - the schema for the external data

format - the source format of the external data

Returns:

an ExternalTableDefinition object given source URIs, schema and format

See Also:

Quota, Source Format

of

public static ExternalTableDefinition of(String sourceUri,
                                         Schema schema,
                                         FormatOptions format)

Creates an ExternalTableDefinition object.

Parameters:

sourceUri - a fully-qualified URI that points to your data in Google Cloud Storage. The URI can contain one '*' wildcard character that must come after the bucket's name. Size limits related to load jobs apply to external data sources.

schema - the schema for the external data

format - the source format of the external data

Returns:

an ExternalTableDefinition object given source URIs, schema and format

See Also:

Quota, Source Format

of

public static ExternalTableDefinition of(String sourceUri,
                                         FormatOptions format)

Creates a builder for an ExternalTableDefinition object.

Parameters:

sourceUri - the fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. Size limits related to load jobs apply to external data sources, plus an additional limit of 10 GB maximum size across all URIs.

format - the source format of the external data

Returns:

a builder for an ExternalTableDefinition object given source URIs and format

See Also:

Quota, Source Format