Class Environment

Describes the environment in which a Dataflow Job runs.

Inheritance
object
Environment
Implements
Inherited Members
Namespace: Google.Apis.Dataflow.v1b3.Data
Assembly: Google.Apis.Dataflow.v1b3.dll
Syntax
public class Environment : IDirectResponseSchema

Properties

ClusterManagerApiService

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".

Declaration
[JsonProperty("clusterManagerApiService")]
public virtual string ClusterManagerApiService { get; set; }
Property Value
Type Description
string

Dataset

Optional. The dataset for the current project where various workflow related tables are stored. The supported resource type is: Google BigQuery: bigquery.googleapis.com/{dataset}

Declaration
[JsonProperty("dataset")]
public virtual string Dataset { get; set; }
Property Value
Type Description
string

DebugOptions

Optional. Any debugging options to be supplied to the job.

Declaration
[JsonProperty("debugOptions")]
public virtual DebugOptions DebugOptions { get; set; }
Property Value
Type Description
DebugOptions

ETag

The ETag of the item.

Declaration
public virtual string ETag { get; set; }
Property Value
Type Description
string

Experiments

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options.

Declaration
[JsonProperty("experiments")]
public virtual IList<string> Experiments { get; set; }
Property Value
Type Description
IList<string>

FlexResourceSchedulingGoal

Optional. Which Flexible Resource Scheduling mode to run in.

Declaration
[JsonProperty("flexResourceSchedulingGoal")]
public virtual string FlexResourceSchedulingGoal { get; set; }
Property Value
Type Description
string

InternalExperiments

Experimental settings.

Declaration
[JsonProperty("internalExperiments")]
public virtual IDictionary<string, object> InternalExperiments { get; set; }
Property Value
Type Description
IDictionary<string, object>

SdkPipelineOptions

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.

Declaration
[JsonProperty("sdkPipelineOptions")]
public virtual IDictionary<string, object> SdkPipelineOptions { get; set; }
Property Value
Type Description
IDictionary<string, object>

ServiceAccountEmail

Optional. Identity to run virtual machines as. Defaults to the default account.

Declaration
[JsonProperty("serviceAccountEmail")]
public virtual string ServiceAccountEmail { get; set; }
Property Value
Type Description
string

ServiceKmsKeyName

Optional. If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK). Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY

Declaration
[JsonProperty("serviceKmsKeyName")]
public virtual string ServiceKmsKeyName { get; set; }
Property Value
Type Description
string

ServiceOptions

Optional. The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on).

Declaration
[JsonProperty("serviceOptions")]
public virtual IList<string> ServiceOptions { get; set; }
Property Value
Type Description
IList<string>

ShuffleMode

Output only. The shuffle mode used for the job.

Declaration
[JsonProperty("shuffleMode")]
public virtual string ShuffleMode { get; set; }
Property Value
Type Description
string

StreamingMode

Optional. Specifies the Streaming Engine message processing guarantees. Reduces cost and latency but might result in duplicate messages committed to storage. Designed to run simple mapping streaming ETL jobs at the lowest cost. For example, Change Data Capture (CDC) to BigQuery is a canonical use case. For more information, see Set the pipeline streaming mode.

Declaration
[JsonProperty("streamingMode")]
public virtual string StreamingMode { get; set; }
Property Value
Type Description
string

TempStoragePrefix

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}

Declaration
[JsonProperty("tempStoragePrefix")]
public virtual string TempStoragePrefix { get; set; }
Property Value
Type Description
string

UseStreamingEngineResourceBasedBilling

Output only. Whether the job uses the Streaming Engine resource-based billing model.

Declaration
[JsonProperty("useStreamingEngineResourceBasedBilling")]
public virtual bool? UseStreamingEngineResourceBasedBilling { get; set; }
Property Value
Type Description
bool?

UserAgent

A description of the process that generated the request.

Declaration
[JsonProperty("userAgent")]
public virtual IDictionary<string, object> UserAgent { get; set; }
Property Value
Type Description
IDictionary<string, object>

Version

A structure describing which components and their versions of the service are required in order to run the job.

Declaration
[JsonProperty("version")]
public virtual IDictionary<string, object> Version { get; set; }
Property Value
Type Description
IDictionary<string, object>

WorkerPools

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.

Declaration
[JsonProperty("workerPools")]
public virtual IList<WorkerPool> WorkerPools { get; set; }
Property Value
Type Description
IList<WorkerPool>

WorkerRegion

Optional. The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.

Declaration
[JsonProperty("workerRegion")]
public virtual string WorkerRegion { get; set; }
Property Value
Type Description
string

WorkerZone

Optional. The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.

Declaration
[JsonProperty("workerZone")]
public virtual string WorkerZone { get; set; }
Property Value
Type Description
string

Implements