Module: Google::Cloud::Dataflow

Defined in:
lib/google/cloud/dataflow.rb,
lib/google/cloud/dataflow/version.rb

Constant Summary collapse

VERSION = 
"0.5.1"

Class Method Summary collapse

Class Method Details

.configure {|::Google::Cloud.configure.dataflow| ... } ⇒ ::Google::Cloud::Config

Configure the google-cloud-dataflow library.

The following configuration parameters are supported:

  • credentials (type: String, Hash, Google::Auth::Credentials) - The path to the keyfile as a String, the contents of the keyfile as a Hash, or a Google::Auth::Credentials object.
  • lib_name (type: String) - The library name as recorded in instrumentation and logging.
  • lib_version (type: String) - The library version as recorded in instrumentation and logging.
  • interceptors (type: Array<GRPC::ClientInterceptor>) - An array of interceptors that are run before calls are executed.
  • timeout (type: Numeric) - Default timeout in seconds.
  • metadata (type: Hash{Symbol=>String}) - Additional headers to be sent with the call.
  • retry_policy (type: Hash) - The retry policy. The value is a hash with the following keys:
    • :initial_delay (type: Numeric) - The initial delay in seconds.
    • :max_delay (type: Numeric) - The max delay in seconds.
    • :multiplier (type: Numeric) - The incremental backoff multiplier.
    • :retry_codes (type: Array<String>) - The error codes that should trigger a retry.

Yields:

Returns:

  • (::Google::Cloud::Config) —

    The default configuration used by this library



283
284
285
286
287
 
# File 'lib/google/cloud/dataflow.rb', line 283

def self.configure
  yield ::Google::Cloud.configure.dataflow if block_given?

  ::Google::Cloud.configure.dataflow
end

.flex_templates_service(version: :v1beta3, transport: :grpc, &block) ⇒ ::Object

Create a new client object for FlexTemplatesService.

By default, this returns an instance of Google::Cloud::Dataflow::V1beta3::FlexTemplatesService::Client for a gRPC client for version V1beta3 of the API. However, you can specify a different API version by passing it in the version parameter. If the FlexTemplatesService service is supported by that API version, and the corresponding gem is available, the appropriate versioned client will be returned. You can also specify a different transport by passing :rest or :grpc in the transport parameter.

About FlexTemplatesService

Provides a service for Flex templates. This feature is not ready yet.

Parameters:

  • version (::String, ::Symbol) (defaults to: :v1beta3) —

    The API version to connect to. Optional. Defaults to :v1beta3.

  • transport (:grpc, :rest) (defaults to: :grpc) —

    The transport to use. Defaults to :grpc.

Returns:

  • (::Object) —

    A client object for the specified version.



243
244
245
246
247
248
249
250
251
252
253
 
# File 'lib/google/cloud/dataflow.rb', line 243

def self.flex_templates_service version: :v1beta3, transport: :grpc, &block
  require "google/cloud/dataflow/#{version.to_s.downcase}"

  package_name = Google::Cloud::Dataflow
                 .constants
                 .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                 .first
  service_module = Google::Cloud::Dataflow.const_get(package_name).const_get(:FlexTemplatesService)
  service_module = service_module.const_get(:Rest) if transport == :rest
  service_module.const_get(:Client).new(&block)
end

.jobs(version: :v1beta3, transport: :grpc, &block) ⇒ ::Object

Create a new client object for Jobs.

By default, this returns an instance of Google::Cloud::Dataflow::V1beta3::Jobs::Client for a gRPC client for version V1beta3 of the API. However, you can specify a different API version by passing it in the version parameter. If the Jobs service is supported by that API version, and the corresponding gem is available, the appropriate versioned client will be returned. You can also specify a different transport by passing :rest or :grpc in the transport parameter.

About Jobs

Provides a method to create and modify Google Cloud Dataflow jobs. A Job is a multi-stage computation graph run by the Cloud Dataflow service.

Parameters:

  • version (::String, ::Symbol) (defaults to: :v1beta3) —

    The API version to connect to. Optional. Defaults to :v1beta3.

  • transport (:grpc, :rest) (defaults to: :grpc) —

    The transport to use. Defaults to :grpc.

Returns:

  • (::Object) —

    A client object for the specified version.



105
106
107
108
109
110
111
112
113
114
115
 
# File 'lib/google/cloud/dataflow.rb', line 105

def self.jobs version: :v1beta3, transport: :grpc, &block
  require "google/cloud/dataflow/#{version.to_s.downcase}"

  package_name = Google::Cloud::Dataflow
                 .constants
                 .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                 .first
  service_module = Google::Cloud::Dataflow.const_get(package_name).const_get(:Jobs)
  service_module = service_module.const_get(:Rest) if transport == :rest
  service_module.const_get(:Client).new(&block)
end

.messages(version: :v1beta3, transport: :grpc, &block) ⇒ ::Object

Create a new client object for Messages.

By default, this returns an instance of Google::Cloud::Dataflow::V1beta3::Messages::Client for a gRPC client for version V1beta3 of the API. However, you can specify a different API version by passing it in the version parameter. If the Messages service is supported by that API version, and the corresponding gem is available, the appropriate versioned client will be returned. You can also specify a different transport by passing :rest or :grpc in the transport parameter.

About Messages

The Dataflow Messages API is used for monitoring the progress of Dataflow jobs.

Parameters:

  • version (::String, ::Symbol) (defaults to: :v1beta3) —

    The API version to connect to. Optional. Defaults to :v1beta3.

  • transport (:grpc, :rest) (defaults to: :grpc) —

    The transport to use. Defaults to :grpc.

Returns:

  • (::Object) —

    A client object for the specified version.



140
141
142
143
144
145
146
147
148
149
150
 
# File 'lib/google/cloud/dataflow.rb', line 140

def self.messages version: :v1beta3, transport: :grpc, &block
  require "google/cloud/dataflow/#{version.to_s.downcase}"

  package_name = Google::Cloud::Dataflow
                 .constants
                 .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                 .first
  service_module = Google::Cloud::Dataflow.const_get(package_name).const_get(:Messages)
  service_module = service_module.const_get(:Rest) if transport == :rest
  service_module.const_get(:Client).new(&block)
end

.metrics(version: :v1beta3, transport: :grpc, &block) ⇒ ::Object

Create a new client object for Metrics.

By default, this returns an instance of Google::Cloud::Dataflow::V1beta3::Metrics::Client for a gRPC client for version V1beta3 of the API. However, you can specify a different API version by passing it in the version parameter. If the Metrics service is supported by that API version, and the corresponding gem is available, the appropriate versioned client will be returned. You can also specify a different transport by passing :rest or :grpc in the transport parameter.

About Metrics

The Dataflow Metrics API lets you monitor the progress of Dataflow jobs.

Parameters:

  • version (::String, ::Symbol) (defaults to: :v1beta3) —

    The API version to connect to. Optional. Defaults to :v1beta3.

  • transport (:grpc, :rest) (defaults to: :grpc) —

    The transport to use. Defaults to :grpc.

Returns:

  • (::Object) —

    A client object for the specified version.



175
176
177
178
179
180
181
182
183
184
185
 
# File 'lib/google/cloud/dataflow.rb', line 175

def self.metrics version: :v1beta3, transport: :grpc, &block
  require "google/cloud/dataflow/#{version.to_s.downcase}"

  package_name = Google::Cloud::Dataflow
                 .constants
                 .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                 .first
  service_module = Google::Cloud::Dataflow.const_get(package_name).const_get(:Metrics)
  service_module = service_module.const_get(:Rest) if transport == :rest
  service_module.const_get(:Client).new(&block)
end

.snapshots(version: :v1beta3, transport: :grpc, &block) ⇒ ::Object

Create a new client object for Snapshots.

By default, this returns an instance of Google::Cloud::Dataflow::V1beta3::Snapshots::Client for a gRPC client for version V1beta3 of the API. However, you can specify a different API version by passing it in the version parameter. If the Snapshots service is supported by that API version, and the corresponding gem is available, the appropriate versioned client will be returned. You can also specify a different transport by passing :rest or :grpc in the transport parameter.

About Snapshots

Provides methods to manage snapshots of Google Cloud Dataflow jobs.

Parameters:

  • version (::String, ::Symbol) (defaults to: :v1beta3) —

    The API version to connect to. Optional. Defaults to :v1beta3.

  • transport (:grpc, :rest) (defaults to: :grpc) —

    The transport to use. Defaults to :grpc.

Returns:

  • (::Object) —

    A client object for the specified version.



70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/google/cloud/dataflow.rb', line 70

def self.snapshots version: :v1beta3, transport: :grpc, &block
  require "google/cloud/dataflow/#{version.to_s.downcase}"

  package_name = Google::Cloud::Dataflow
                 .constants
                 .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                 .first
  service_module = Google::Cloud::Dataflow.const_get(package_name).const_get(:Snapshots)
  service_module = service_module.const_get(:Rest) if transport == :rest
  service_module.const_get(:Client).new(&block)
end

.templates_service(version: :v1beta3, transport: :grpc, &block) ⇒ ::Object

Create a new client object for TemplatesService.

By default, this returns an instance of Google::Cloud::Dataflow::V1beta3::TemplatesService::Client for a gRPC client for version V1beta3 of the API. However, you can specify a different API version by passing it in the version parameter. If the TemplatesService service is supported by that API version, and the corresponding gem is available, the appropriate versioned client will be returned. You can also specify a different transport by passing :rest or :grpc in the transport parameter.

About TemplatesService

Provides a method to create Cloud Dataflow jobs from templates.

Parameters:

  • version (::String, ::Symbol) (defaults to: :v1beta3) —

    The API version to connect to. Optional. Defaults to :v1beta3.

  • transport (:grpc, :rest) (defaults to: :grpc) —

    The transport to use. Defaults to :grpc.

Returns:

  • (::Object) —

    A client object for the specified version.



209
210
211
212
213
214
215
216
217
218
219
 
# File 'lib/google/cloud/dataflow.rb', line 209

def self.templates_service version: :v1beta3, transport: :grpc, &block
  require "google/cloud/dataflow/#{version.to_s.downcase}"

  package_name = Google::Cloud::Dataflow
                 .constants
                 .select { |sym| sym.to_s.downcase == version.to_s.downcase.tr("_", "") }
                 .first
  service_module = Google::Cloud::Dataflow.const_get(package_name).const_get(:TemplatesService)
  service_module = service_module.const_get(:Rest) if transport == :rest
  service_module.const_get(:Client).new(&block)
end