Class: Google::Apis::DataflowV1b3::WorkerPool
- Inherits:
-
Object
- Object
- Google::Apis::DataflowV1b3::WorkerPool
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataflow_v1b3/classes.rb,
generated/google/apis/dataflow_v1b3/representations.rb,
generated/google/apis/dataflow_v1b3/representations.rb
Overview
Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.
Instance Attribute Summary collapse
-
#autoscaling_settings ⇒ Google::Apis::DataflowV1b3::AutoscalingSettings
Settings for WorkerPool autoscaling.
-
#data_disks ⇒ Array<Google::Apis::DataflowV1b3::Disk>
Data disks that are used by a VM in this workflow.
-
#default_package_set ⇒ String
The default package set to install.
-
#disk_size_gb ⇒ Fixnum
Size of root disk for VMs, in GB.
-
#disk_source_image ⇒ String
Fully qualified source image for disks.
-
#disk_type ⇒ String
Type of root disk for VMs.
-
#ip_configuration ⇒ String
Configuration for VM IPs.
-
#kind ⇒ String
The kind of the worker pool; currently only
harness
andshuffle
are supported. -
#machine_type ⇒ String
Machine type (e.g. "n1-standard-1").
-
#metadata ⇒ Hash<String,String>
Metadata to set on the Google Compute Engine VMs.
-
#network ⇒ String
Network to which VMs will be assigned.
-
#num_threads_per_worker ⇒ Fixnum
The number of threads per worker harness.
-
#num_workers ⇒ Fixnum
Number of Google Compute Engine workers in this pool needed to execute the job.
-
#on_host_maintenance ⇒ String
The action to take on host maintenance, as defined by the Google Compute Engine API.
-
#packages ⇒ Array<Google::Apis::DataflowV1b3::Package>
Packages to be installed on workers.
-
#pool_args ⇒ Hash<String,Object>
Extra arguments for this worker pool.
-
#subnetwork ⇒ String
Subnetwork to which VMs will be assigned, if desired.
-
#taskrunner_settings ⇒ Google::Apis::DataflowV1b3::TaskRunnerSettings
Taskrunner configuration settings.
-
#teardown_policy ⇒ String
Sets the policy for determining when to turndown worker pool.
-
#worker_harness_container_image ⇒ String
Required.
-
#zone ⇒ String
Zone to run the worker pools in.
Instance Method Summary collapse
-
#initialize(**args) ⇒ WorkerPool
constructor
A new instance of WorkerPool.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ WorkerPool
Returns a new instance of WorkerPool
1198 1199 1200 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1198 def initialize(**args) update!(**args) end |
Instance Attribute Details
#autoscaling_settings ⇒ Google::Apis::DataflowV1b3::AutoscalingSettings
Settings for WorkerPool autoscaling.
Corresponds to the JSON property autoscalingSettings
1153 1154 1155 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1153 def autoscaling_settings @autoscaling_settings end |
#data_disks ⇒ Array<Google::Apis::DataflowV1b3::Disk>
Data disks that are used by a VM in this workflow.
Corresponds to the JSON property dataDisks
1132 1133 1134 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1132 def data_disks @data_disks end |
#default_package_set ⇒ String
The default package set to install. This allows the service to
select a default set of packages which are useful to worker
harnesses written in a particular language.
Corresponds to the JSON property defaultPackageSet
1165 1166 1167 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1165 def default_package_set @default_package_set end |
#disk_size_gb ⇒ Fixnum
Size of root disk for VMs, in GB. If zero or unspecified, the service will
attempt to choose a reasonable default.
Corresponds to the JSON property diskSizeGb
1103 1104 1105 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1103 def disk_size_gb @disk_size_gb end |
#disk_source_image ⇒ String
Fully qualified source image for disks.
Corresponds to the JSON property diskSourceImage
1196 1197 1198 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1196 def disk_source_image @disk_source_image end |
#disk_type ⇒ String
Type of root disk for VMs. If empty or unspecified, the service will
attempt to choose a reasonable default.
Corresponds to the JSON property diskType
1121 1122 1123 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1121 def disk_type @disk_type end |
#ip_configuration ⇒ String
Configuration for VM IPs.
Corresponds to the JSON property ipConfiguration
1143 1144 1145 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1143 def ip_configuration @ip_configuration end |
#kind ⇒ String
The kind of the worker pool; currently only harness
and shuffle
are supported.
Corresponds to the JSON property kind
1127 1128 1129 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1127 def kind @kind end |
#machine_type ⇒ String
Machine type (e.g. "n1-standard-1"). If empty or unspecified, the
service will attempt to choose a reasonable default.
Corresponds to the JSON property machineType
1115 1116 1117 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1115 def machine_type @machine_type end |
#metadata ⇒ Hash<String,String>
Metadata to set on the Google Compute Engine VMs.
Corresponds to the JSON property metadata
1158 1159 1160 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1158 def @metadata end |
#network ⇒ String
Network to which VMs will be assigned. If empty or unspecified,
the service will use the network "default".
Corresponds to the JSON property network
1171 1172 1173 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1171 def network @network end |
#num_threads_per_worker ⇒ Fixnum
The number of threads per worker harness. If empty or unspecified, the
service will choose a number of threads (according to the number of cores
on the selected machine type for batch, or 1 by convention for streaming).
Corresponds to the JSON property numThreadsPerWorker
1191 1192 1193 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1191 def num_threads_per_worker @num_threads_per_worker end |
#num_workers ⇒ Fixnum
Number of Google Compute Engine workers in this pool needed to
execute the job. If zero or unspecified, the service will
attempt to choose a reasonable default.
Corresponds to the JSON property numWorkers
1184 1185 1186 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1184 def num_workers @num_workers end |
#on_host_maintenance ⇒ String
The action to take on host maintenance, as defined by the Google
Compute Engine API.
Corresponds to the JSON property onHostMaintenance
1092 1093 1094 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1092 def on_host_maintenance @on_host_maintenance end |
#packages ⇒ Array<Google::Apis::DataflowV1b3::Package>
Packages to be installed on workers.
Corresponds to the JSON property packages
1068 1069 1070 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1068 def packages @packages end |
#pool_args ⇒ Hash<String,Object>
Extra arguments for this worker pool.
Corresponds to the JSON property poolArgs
1097 1098 1099 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1097 def pool_args @pool_args end |
#subnetwork ⇒ String
Subnetwork to which VMs will be assigned, if desired. Expected to be of
the form "regions/REGION/subnetworks/SUBNETWORK".
Corresponds to the JSON property subnetwork
1138 1139 1140 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1138 def subnetwork @subnetwork end |
#taskrunner_settings ⇒ Google::Apis::DataflowV1b3::TaskRunnerSettings
Taskrunner configuration settings.
Corresponds to the JSON property taskrunnerSettings
1148 1149 1150 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1148 def taskrunner_settings @taskrunner_settings end |
#teardown_policy ⇒ String
Sets the policy for determining when to turndown worker pool.
Allowed values are: TEARDOWN_ALWAYS
, TEARDOWN_ON_SUCCESS
, and
TEARDOWN_NEVER
.
TEARDOWN_ALWAYS
means workers are always torn down regardless of whether
the job succeeds. TEARDOWN_ON_SUCCESS
means workers are torn down
if the job succeeds. TEARDOWN_NEVER
means the workers are never torn
down.
If the workers are not torn down by the service, they will
continue to run and use Google Compute Engine VM resources in the
user's project until they are explicitly terminated by the user.
Because of this, Google recommends using the TEARDOWN_ALWAYS
policy except for small, manually supervised test jobs.
If unknown or unspecified, the service will attempt to choose a reasonable
default.
Corresponds to the JSON property teardownPolicy
1086 1087 1088 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1086 def teardown_policy @teardown_policy end |
#worker_harness_container_image ⇒ String
Required. Docker container image that executes the Cloud Dataflow worker
harness, residing in Google Container Registry.
Corresponds to the JSON property workerHarnessContainerImage
1109 1110 1111 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1109 def worker_harness_container_image @worker_harness_container_image end |
#zone ⇒ String
Zone to run the worker pools in. If empty or unspecified, the service
will attempt to choose a reasonable default.
Corresponds to the JSON property zone
1177 1178 1179 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1177 def zone @zone end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 1203 def update!(**args) @packages = args[:packages] if args.key?(:packages) @teardown_policy = args[:teardown_policy] if args.key?(:teardown_policy) @on_host_maintenance = args[:on_host_maintenance] if args.key?(:on_host_maintenance) @pool_args = args[:pool_args] if args.key?(:pool_args) @disk_size_gb = args[:disk_size_gb] if args.key?(:disk_size_gb) @worker_harness_container_image = args[:worker_harness_container_image] if args.key?(:worker_harness_container_image) @machine_type = args[:machine_type] if args.key?(:machine_type) @disk_type = args[:disk_type] if args.key?(:disk_type) @kind = args[:kind] if args.key?(:kind) @data_disks = args[:data_disks] if args.key?(:data_disks) @subnetwork = args[:subnetwork] if args.key?(:subnetwork) @ip_configuration = args[:ip_configuration] if args.key?(:ip_configuration) @taskrunner_settings = args[:taskrunner_settings] if args.key?(:taskrunner_settings) @autoscaling_settings = args[:autoscaling_settings] if args.key?(:autoscaling_settings) @metadata = args[:metadata] if args.key?(:metadata) @default_package_set = args[:default_package_set] if args.key?(:default_package_set) @network = args[:network] if args.key?(:network) @zone = args[:zone] if args.key?(:zone) @num_workers = args[:num_workers] if args.key?(:num_workers) @num_threads_per_worker = args[:num_threads_per_worker] if args.key?(:num_threads_per_worker) @disk_source_image = args[:disk_source_image] if args.key?(:disk_source_image) end |