Class WorkerPool

Describes one particular pool of Cloud Dataflow workers to be instantiated by the Cloud Dataflow service in order to perform the computations required by a job. Note that a workflow job may use multiple pools, in order to match the various computational requirements of the various stages of the job.

Inheritance

object

WorkerPool

Implements

IDirectResponseSchema

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Google.Apis.Dataflow.v1b3.Data

Assembly: Google.Apis.Dataflow.v1b3.dll

Syntax

public class WorkerPool : IDirectResponseSchema

Properties

AutoscalingSettings

Settings for autoscaling of this WorkerPool.

Declaration

[JsonProperty("autoscalingSettings")]
public virtual AutoscalingSettings AutoscalingSettings { get; set; }

Property Value

Type	Description
AutoscalingSettings

DataDisks

Data disks that are used by a VM in this workflow.

Declaration

[JsonProperty("dataDisks")]
public virtual IList<Disk> DataDisks { get; set; }

Property Value

Type	Description
IList<Disk>

DefaultPackageSet

The default package set to install. This allows the service to select a default set of packages which are useful to worker harnesses written in a particular language.

Declaration

[JsonProperty("defaultPackageSet")]
public virtual string DefaultPackageSet { get; set; }

Property Value

Type	Description
string

DiskSizeGb

Size of root disk for VMs, in GB. If zero or unspecified, the service will attempt to choose a reasonable default.

Declaration

[JsonProperty("diskSizeGb")]
public virtual int? DiskSizeGb { get; set; }

Property Value

Type	Description
int?

DiskSourceImage

Fully qualified source image for disks.

Declaration

[JsonProperty("diskSourceImage")]
public virtual string DiskSourceImage { get; set; }

Property Value

Type	Description
string

DiskType

Type of root disk for VMs. If empty or unspecified, the service will attempt to choose a reasonable default.

Declaration

[JsonProperty("diskType")]
public virtual string DiskType { get; set; }

Property Value

Type	Description
string

ETag

The ETag of the item.

Declaration

public virtual string ETag { get; set; }

Property Value

Type	Description
string

IpConfiguration

Configuration for VM IPs.

Declaration

[JsonProperty("ipConfiguration")]
public virtual string IpConfiguration { get; set; }

Property Value

Type	Description
string

Kind

The kind of the worker pool; currently only harness and shuffle are supported.

Declaration

[JsonProperty("kind")]
public virtual string Kind { get; set; }

Property Value

Type	Description
string

MachineType

Machine type (e.g. "n1-standard-1"). If empty or unspecified, the service will attempt to choose a reasonable default.

Declaration

[JsonProperty("machineType")]
public virtual string MachineType { get; set; }

Property Value

Type	Description
string

Metadata

Metadata to set on the Google Compute Engine VMs.

Declaration

[JsonProperty("metadata")]
public virtual IDictionary<string, string> Metadata { get; set; }

Property Value

Type	Description
IDictionary<string, string>

Network

Network to which VMs will be assigned. If empty or unspecified, the service will use the network "default".

Declaration

[JsonProperty("network")]
public virtual string Network { get; set; }

Property Value

Type	Description
string

NumThreadsPerWorker

The number of threads per worker harness. If empty or unspecified, the service will choose a number of threads (according to the number of cores on the selected machine type for batch, or 1 by convention for streaming).

Declaration

[JsonProperty("numThreadsPerWorker")]
public virtual int? NumThreadsPerWorker { get; set; }

Property Value

Type	Description
int?

NumWorkers

Number of Google Compute Engine workers in this pool needed to execute the job. If zero or unspecified, the service will attempt to choose a reasonable default.

Declaration

[JsonProperty("numWorkers")]
public virtual int? NumWorkers { get; set; }

Property Value

Type	Description
int?

OnHostMaintenance

The action to take on host maintenance, as defined by the Google Compute Engine API.

Declaration

[JsonProperty("onHostMaintenance")]
public virtual string OnHostMaintenance { get; set; }

Property Value

Type	Description
string

Packages

Packages to be installed on workers.

Declaration

[JsonProperty("packages")]
public virtual IList<Package> Packages { get; set; }

Property Value

Type	Description
IList<Package>

PoolArgs

Extra arguments for this worker pool.

Declaration

[JsonProperty("poolArgs")]
public virtual IDictionary<string, object> PoolArgs { get; set; }

Property Value

Type	Description
IDictionary<string, object>

SdkHarnessContainerImages

Set of SDK harness containers needed to execute this pipeline. This will only be set in the Fn API path. For non-cross-language pipelines this should have only one entry. Cross-language pipelines will have two or more entries.

Declaration

[JsonProperty("sdkHarnessContainerImages")]
public virtual IList<SdkHarnessContainerImage> SdkHarnessContainerImages { get; set; }

Property Value

Type	Description
IList<SdkHarnessContainerImage>

Subnetwork

Subnetwork to which VMs will be assigned, if desired. Expected to be of the form "regions/REGION/subnetworks/SUBNETWORK".

Declaration

[JsonProperty("subnetwork")]
public virtual string Subnetwork { get; set; }

Property Value

Type	Description
string

TaskrunnerSettings

Settings passed through to Google Compute Engine workers when using the standard Dataflow task runner. Users should ignore this field.

Declaration

[JsonProperty("taskrunnerSettings")]
public virtual TaskRunnerSettings TaskrunnerSettings { get; set; }

Property Value

Type	Description
TaskRunnerSettings

TeardownPolicy

Sets the policy for determining when to turndown worker pool. Allowed values are: TEARDOWN_ALWAYS, TEARDOWN_ON_SUCCESS, and TEARDOWN_NEVER. TEARDOWN_ALWAYS means workers are always torn down regardless of whether the job succeeds. TEARDOWN_ON_SUCCESS means workers are torn down if the job succeeds. TEARDOWN_NEVER means the workers are never torn down. If the workers are not torn down by the service, they will continue to run and use Google Compute Engine VM resources in the user's project until they are explicitly terminated by the user. Because of this, Google recommends using the TEARDOWN_ALWAYS policy except for small, manually supervised test jobs. If unknown or unspecified, the service will attempt to choose a reasonable default.

Declaration

[JsonProperty("teardownPolicy")]
public virtual string TeardownPolicy { get; set; }

Property Value

Type	Description
string

WorkerHarnessContainerImage

Required. Docker container image that executes the Cloud Dataflow worker harness, residing in Google Container Registry. Deprecated for the Fn API path. Use sdk_harness_container_images instead.

Declaration

[JsonProperty("workerHarnessContainerImage")]
public virtual string WorkerHarnessContainerImage { get; set; }

Property Value

Type	Description
string

Zone

Zone to run the worker pools in. If empty or unspecified, the service will attempt to choose a reasonable default.

Declaration

[JsonProperty("zone")]
public virtual string Zone { get; set; }

Property Value

Type	Description
string

Implements

IDirectResponseSchema