Members
(constant) gapicConfig
Client JSON configuration object, loaded from
src/v1/big_query_read_client_config.json.
This file defines retry strategy and timeouts for all API methods in this library.
(constant) gapicConfig
Client JSON configuration object, loaded from
src/v1/big_query_write_client_config.json.
This file defines retry strategy and timeouts for all API methods in this library.
(constant) gapicConfig
Client JSON configuration object, loaded from
src/v1alpha/metastore_partition_service_client_config.json.
This file defines retry strategy and timeouts for all API methods in this library.
(constant) gapicConfig
Client JSON configuration object, loaded from
src/v1beta1/big_query_storage_client_config.json.
This file defines retry strategy and timeouts for all API methods in this library.
Methods
convertBigQuerySchemaToStorageTableSchema(schema)
Converts a bigquery Schema into the protobuf-based TableSchema used by the BigQuery Storage WriteClient.
Parameters:
| Name | Type | Description |
|---|---|---|
schema |
a BigQuery TableSchema |
Returns:
| Type | Description |
|---|---|
|
StorageTableSchema |
convertStorageSchemaToProto2Descriptor(schema, scope)
Builds a DescriptorProto for a given table schema using proto2 syntax.
If a column name is not compatible as a protobuf field name, it will be replaced
with a placeholder name. The placeholder name is generated by removing all invalid
characters from the column name and replacing all dashes with underscores. If the
column name doesn't have any valid characters, we generate a placeholder name using
the field number field{fieldNumber}.
Parameters:
| Name | Type | Description |
|---|---|---|
schema |
a BigQuery Storage TableSchema. |
|
scope |
scope to namespace protobuf structs. |
Returns:
| Type | Description |
|---|---|
|
DescriptorProto |
convertStorageSchemaToProto3Descriptor(schema, scope)
Builds a DescriptorProto for a given table schema using proto3 syntax.
If a column name is not compatible as a protobuf field name, it will be replaced
with a placeholder name. The placeholder name is generated by removing all invalid
characters from the column name and replacing all dashes with underscores. If the
column name doesn't have any valid characters, we generate a placeholder name using
the field number field{fieldNumber}.
Parameters:
| Name | Type | Description |
|---|---|---|
schema |
a Bigquery TableSchema. |
|
scope |
scope to namespace protobuf structs. |
Returns:
| Type | Description |
|---|---|
|
DescriptorProto |
generatePlaceholderFieldName(fieldName, fNumber)
Generates a placeholder name that consists of a sanitized field name with only valid characters. If the field doesn't have any valid characters, we generate a placeholder name using the field number. We replace all dashes with underscores as they are not allowed for proto field names.
Parameters:
| Name | Type | Description |
|---|---|---|
fieldName |
table field name |
|
fNumber |
proto field number |
Returns:
| Type | Description |
|---|---|
|
the generated placeholder field name |
isProtoCompatible(fieldName)
Checks if the field name is compatible with proto field naming convention.
Parameters:
| Name | Type | Description |
|---|---|---|
fieldName |
name for the field |
Returns:
| Type | Description |
|---|---|
|
true if the field name is comptaible with proto naming convention, otherwise, returns false. |
normalizeDescriptor(dp)
Builds a self-contained DescriptorProto suitable for communicating schema information with the BigQuery Storage write API. It's primarily used for cases where users are interested in sending data using a predefined protocol buffer message.
Parameters:
| Name | Type | Description |
|---|---|---|
dp |
DescriptorProto to be bundled. |
Returns:
| Type | Description |
|---|---|
|
DescriptorProto |
parseStorageErrors(err) → {google.cloud.bigquery.storage.v1.StorageError}
The BigQuery Storage API service augments applicable errors with service-specific details in the form of a StorageError message.
Parameters:
| Name | Type | Description |
|---|---|---|
err |
gax.GoogleError |
Returns:
| Type | Description |
|---|---|
| google.cloud.bigquery.storage.v1.StorageError |
setLogFunction(logger)
Sets or disables the log function for all active Firestore instances.
Parameters:
| Name | Type | Description |
|---|---|---|
logger |
A log function that takes a message (such as |
withChangeSequenceNumber()
Add pseudocolumn _CHANGE_SEQUENCE_NUMBER for BigQuery Change Data Capture.
Used to change behavior of ordering records with same primary key.
See more: https://cloud.google.com/bigquery/docs/change-data-capture#manage_custom_ordering
withChangeType()
Add pseudocolumn _CHANGE_TYPE for BigQuery Change Data Capture.
Used to define the type of change to be professed for each row.
The pseudocolumn _CHANGE_TYPE only accepts the values UPSERT and DELETE.
See more: https://cloud.google.com/bigquery/docs/change-data-capture#specify_changes_to_existing_records