Class: Google::Apis::DataflowV1b3::WorkerPool
- Inherits:
-
Object
- Object
- Google::Apis::DataflowV1b3::WorkerPool
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataflow_v1b3/classes.rb,
generated/google/apis/dataflow_v1b3/representations.rb,
generated/google/apis/dataflow_v1b3/representations.rb
Overview
Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.
Instance Attribute Summary collapse
-
#autoscaling_settings ⇒ Google::Apis::DataflowV1b3::AutoscalingSettings
Settings for WorkerPool autoscaling.
-
#data_disks ⇒ Array<Google::Apis::DataflowV1b3::Disk>
Data disks that are used by a VM in this workflow.
-
#default_package_set ⇒ String
The default package set to install.
-
#disk_size_gb ⇒ Fixnum
Size of root disk for VMs, in GB.
-
#disk_source_image ⇒ String
Fully qualified source image for disks.
-
#disk_type ⇒ String
Type of root disk for VMs.
-
#ip_configuration ⇒ String
Configuration for VM IPs.
-
#kind ⇒ String
The kind of the worker pool; currently only
harness
andshuffle
are supported. -
#machine_type ⇒ String
Machine type (e.g. "n1-standard-1").
-
#metadata ⇒ Hash<String,String>
Metadata to set on the Google Compute Engine VMs.
-
#network ⇒ String
Network to which VMs will be assigned.
-
#num_threads_per_worker ⇒ Fixnum
The number of threads per worker harness.
-
#num_workers ⇒ Fixnum
Number of Google Compute Engine workers in this pool needed to execute the job.
-
#on_host_maintenance ⇒ String
The action to take on host maintenance, as defined by the Google Compute Engine API.
-
#packages ⇒ Array<Google::Apis::DataflowV1b3::Package>
Packages to be installed on workers.
-
#pool_args ⇒ Hash<String,Object>
Extra arguments for this worker pool.
-
#sdk_harness_container_images ⇒ Array<Google::Apis::DataflowV1b3::SdkHarnessContainerImage>
Set of SDK harness containers needed to execute this pipeline.
-
#subnetwork ⇒ String
Subnetwork to which VMs will be assigned, if desired.
-
#taskrunner_settings ⇒ Google::Apis::DataflowV1b3::TaskRunnerSettings
Taskrunner configuration settings.
-
#teardown_policy ⇒ String
Sets the policy for determining when to turndown worker pool.
-
#worker_harness_container_image ⇒ String
Required.
-
#zone ⇒ String
Zone to run the worker pools in.
Instance Method Summary collapse
-
#initialize(**args) ⇒ WorkerPool
constructor
A new instance of WorkerPool.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ WorkerPool
Returns a new instance of WorkerPool.
5969 5970 5971 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5969 def initialize(**args) update!(**args) end |
Instance Attribute Details
#autoscaling_settings ⇒ Google::Apis::DataflowV1b3::AutoscalingSettings
Settings for WorkerPool autoscaling.
Corresponds to the JSON property autoscalingSettings
5830 5831 5832 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5830 def autoscaling_settings @autoscaling_settings end |
#data_disks ⇒ Array<Google::Apis::DataflowV1b3::Disk>
Data disks that are used by a VM in this workflow.
Corresponds to the JSON property dataDisks
5835 5836 5837 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5835 def data_disks @data_disks end |
#default_package_set ⇒ String
The default package set to install. This allows the service to
select a default set of packages which are useful to worker
harnesses written in a particular language.
Corresponds to the JSON property defaultPackageSet
5842 5843 5844 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5842 def default_package_set @default_package_set end |
#disk_size_gb ⇒ Fixnum
Size of root disk for VMs, in GB. If zero or unspecified, the service will
attempt to choose a reasonable default.
Corresponds to the JSON property diskSizeGb
5848 5849 5850 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5848 def disk_size_gb @disk_size_gb end |
#disk_source_image ⇒ String
Fully qualified source image for disks.
Corresponds to the JSON property diskSourceImage
5853 5854 5855 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5853 def disk_source_image @disk_source_image end |
#disk_type ⇒ String
Type of root disk for VMs. If empty or unspecified, the service will
attempt to choose a reasonable default.
Corresponds to the JSON property diskType
5859 5860 5861 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5859 def disk_type @disk_type end |
#ip_configuration ⇒ String
Configuration for VM IPs.
Corresponds to the JSON property ipConfiguration
5864 5865 5866 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5864 def ip_configuration @ip_configuration end |
#kind ⇒ String
The kind of the worker pool; currently only harness
and shuffle
are supported.
Corresponds to the JSON property kind
5870 5871 5872 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5870 def kind @kind end |
#machine_type ⇒ String
Machine type (e.g. "n1-standard-1"). If empty or unspecified, the
service will attempt to choose a reasonable default.
Corresponds to the JSON property machineType
5876 5877 5878 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5876 def machine_type @machine_type end |
#metadata ⇒ Hash<String,String>
Metadata to set on the Google Compute Engine VMs.
Corresponds to the JSON property metadata
5881 5882 5883 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5881 def @metadata end |
#network ⇒ String
Network to which VMs will be assigned. If empty or unspecified,
the service will use the network "default".
Corresponds to the JSON property network
5887 5888 5889 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5887 def network @network end |
#num_threads_per_worker ⇒ Fixnum
The number of threads per worker harness. If empty or unspecified, the
service will choose a number of threads (according to the number of cores
on the selected machine type for batch, or 1 by convention for streaming).
Corresponds to the JSON property numThreadsPerWorker
5894 5895 5896 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5894 def num_threads_per_worker @num_threads_per_worker end |
#num_workers ⇒ Fixnum
Number of Google Compute Engine workers in this pool needed to
execute the job. If zero or unspecified, the service will
attempt to choose a reasonable default.
Corresponds to the JSON property numWorkers
5901 5902 5903 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5901 def num_workers @num_workers end |
#on_host_maintenance ⇒ String
The action to take on host maintenance, as defined by the Google
Compute Engine API.
Corresponds to the JSON property onHostMaintenance
5907 5908 5909 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5907 def on_host_maintenance @on_host_maintenance end |
#packages ⇒ Array<Google::Apis::DataflowV1b3::Package>
Packages to be installed on workers.
Corresponds to the JSON property packages
5912 5913 5914 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5912 def packages @packages end |
#pool_args ⇒ Hash<String,Object>
Extra arguments for this worker pool.
Corresponds to the JSON property poolArgs
5917 5918 5919 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5917 def pool_args @pool_args end |
#sdk_harness_container_images ⇒ Array<Google::Apis::DataflowV1b3::SdkHarnessContainerImage>
Set of SDK harness containers needed to execute this pipeline. This will
only be set in the Fn API path. For non-cross-language pipelines this
should have only one entry. Cross-language pipelines will have two or more
entries.
Corresponds to the JSON property sdkHarnessContainerImages
5925 5926 5927 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5925 def sdk_harness_container_images @sdk_harness_container_images end |
#subnetwork ⇒ String
Subnetwork to which VMs will be assigned, if desired. Expected to be of
the form "regions/REGION/subnetworks/SUBNETWORK".
Corresponds to the JSON property subnetwork
5931 5932 5933 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5931 def subnetwork @subnetwork end |
#taskrunner_settings ⇒ Google::Apis::DataflowV1b3::TaskRunnerSettings
Taskrunner configuration settings.
Corresponds to the JSON property taskrunnerSettings
5936 5937 5938 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5936 def taskrunner_settings @taskrunner_settings end |
#teardown_policy ⇒ String
Sets the policy for determining when to turndown worker pool.
Allowed values are: TEARDOWN_ALWAYS
, TEARDOWN_ON_SUCCESS
, and
TEARDOWN_NEVER
.
TEARDOWN_ALWAYS
means workers are always torn down regardless of whether
the job succeeds. TEARDOWN_ON_SUCCESS
means workers are torn down
if the job succeeds. TEARDOWN_NEVER
means the workers are never torn
down.
If the workers are not torn down by the service, they will
continue to run and use Google Compute Engine VM resources in the
user's project until they are explicitly terminated by the user.
Because of this, Google recommends using the TEARDOWN_ALWAYS
policy except for small, manually supervised test jobs.
If unknown or unspecified, the service will attempt to choose a reasonable
default.
Corresponds to the JSON property teardownPolicy
5954 5955 5956 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5954 def teardown_policy @teardown_policy end |
#worker_harness_container_image ⇒ String
Required. Docker container image that executes the Cloud Dataflow worker
harness, residing in Google Container Registry.
Deprecated for the Fn API path. Use sdk_harness_container_images instead.
Corresponds to the JSON property workerHarnessContainerImage
5961 5962 5963 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5961 def worker_harness_container_image @worker_harness_container_image end |
#zone ⇒ String
Zone to run the worker pools in. If empty or unspecified, the service
will attempt to choose a reasonable default.
Corresponds to the JSON property zone
5967 5968 5969 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5967 def zone @zone end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
5974 5975 5976 5977 5978 5979 5980 5981 5982 5983 5984 5985 5986 5987 5988 5989 5990 5991 5992 5993 5994 5995 5996 5997 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 5974 def update!(**args) @autoscaling_settings = args[:autoscaling_settings] if args.key?(:autoscaling_settings) @data_disks = args[:data_disks] if args.key?(:data_disks) @default_package_set = args[:default_package_set] if args.key?(:default_package_set) @disk_size_gb = args[:disk_size_gb] if args.key?(:disk_size_gb) @disk_source_image = args[:disk_source_image] if args.key?(:disk_source_image) @disk_type = args[:disk_type] if args.key?(:disk_type) @ip_configuration = args[:ip_configuration] if args.key?(:ip_configuration) @kind = args[:kind] if args.key?(:kind) @machine_type = args[:machine_type] if args.key?(:machine_type) @metadata = args[:metadata] if args.key?(:metadata) @network = args[:network] if args.key?(:network) @num_threads_per_worker = args[:num_threads_per_worker] if args.key?(:num_threads_per_worker) @num_workers = args[:num_workers] if args.key?(:num_workers) @on_host_maintenance = args[:on_host_maintenance] if args.key?(:on_host_maintenance) @packages = args[:packages] if args.key?(:packages) @pool_args = args[:pool_args] if args.key?(:pool_args) @sdk_harness_container_images = args[:sdk_harness_container_images] if args.key?(:sdk_harness_container_images) @subnetwork = args[:subnetwork] if args.key?(:subnetwork) @taskrunner_settings = args[:taskrunner_settings] if args.key?(:taskrunner_settings) @teardown_policy = args[:teardown_policy] if args.key?(:teardown_policy) @worker_harness_container_image = args[:worker_harness_container_image] if args.key?(:worker_harness_container_image) @zone = args[:zone] if args.key?(:zone) end |