Class: Google::Apis::DataflowV1b3::Environment

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/dataflow_v1b3/classes.rb,
lib/google/apis/dataflow_v1b3/representations.rb,
lib/google/apis/dataflow_v1b3/representations.rb

Overview

Describes the environment in which a Dataflow Job runs.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ Environment

Returns a new instance of Environment.



1345
1346
1347
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1345

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#cluster_manager_api_service ⇒ String

The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com". Corresponds to the JSON property clusterManagerApiService

Returns:

  • (String) 


1241
1242
1243
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1241

def cluster_manager_api_service
  @cluster_manager_api_service
end

#dataset ⇒ String

The dataset for the current project where various workflow related tables are stored. The supported resource type is: Google BigQuery: bigquery.googleapis. com/dataset Corresponds to the JSON property dataset

Returns:

  • (String) 


1248
1249
1250
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1248

def dataset
  @dataset
end

#experiments ⇒ Array<String>

The list of experiments to enable. This field should be used for SDK related experiments and not for service related experiments. The proper field for service related experiments is service_options. For more details see the rationale at go/user-specified-service-options. Corresponds to the JSON property experiments

Returns:

  • (Array<String>) 


1256
1257
1258
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1256

def experiments
  @experiments
end

#flex_resource_scheduling_goal ⇒ String

Which Flexible Resource Scheduling mode to run in. Corresponds to the JSON property flexResourceSchedulingGoal

Returns:

  • (String) 


1261
1262
1263
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1261

def flex_resource_scheduling_goal
  @flex_resource_scheduling_goal
end

#internal_experiments ⇒ Hash<String,Object>

Experimental settings. Corresponds to the JSON property internalExperiments

Returns:

  • (Hash<String,Object>) 


1266
1267
1268
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1266

def internal_experiments
  @internal_experiments
end

#sdk_pipeline_options ⇒ Hash<String,Object>

The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way. Corresponds to the JSON property sdkPipelineOptions

Returns:

  • (Hash<String,Object>) 


1273
1274
1275
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1273

def sdk_pipeline_options
  @sdk_pipeline_options
end

#service_account_email ⇒ String

Identity to run virtual machines as. Defaults to the default account. Corresponds to the JSON property serviceAccountEmail

Returns:

  • (String) 


1278
1279
1280
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1278

def 
  @service_account_email
end

#service_kms_key_name ⇒ String

If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK). Format: projects/PROJECT_ID/ locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY Corresponds to the JSON property serviceKmsKeyName

Returns:

  • (String) 


1285
1286
1287
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1285

def service_kms_key_name
  @service_kms_key_name
end

#service_options ⇒ Array<String>

The list of service options to enable. This field should be used for service related experiments only. These experiments, when graduating to GA, should be replaced by dedicated fields or become default (i.e. always on). For more details see the rationale at go/user-specified-service-options. Corresponds to the JSON property serviceOptions

Returns:

  • (Array<String>) 


1293
1294
1295
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1293

def service_options
  @service_options
end

#shuffle_mode ⇒ String

Output only. The shuffle mode used for the job. Corresponds to the JSON property shuffleMode

Returns:

  • (String) 


1298
1299
1300
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1298

def shuffle_mode
  @shuffle_mode
end

#temp_storage_prefix ⇒ String

The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-JOBNAME to this resource prefix, where JOBNAME is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is: Google Cloud Storage: storage.googleapis.com/bucket/object bucket.storage.googleapis.com/object Corresponds to the JSON property tempStoragePrefix

Returns:

  • (String) 


1309
1310
1311
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1309

def temp_storage_prefix
  @temp_storage_prefix
end

#user_agent ⇒ Hash<String,Object>

A description of the process that generated the request. Corresponds to the JSON property userAgent

Returns:

  • (Hash<String,Object>) 


1314
1315
1316
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1314

def user_agent
  @user_agent
end

#version ⇒ Hash<String,Object>

A structure describing which components and their versions of the service are required in order to run the job. Corresponds to the JSON property version

Returns:

  • (Hash<String,Object>) 


1320
1321
1322
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1320

def version
  @version
end

#worker_pools ⇒ Array<Google::Apis::DataflowV1b3::WorkerPool>

The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers. Corresponds to the JSON property workerPools

Returns:



1326
1327
1328
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1326

def worker_pools
  @worker_pools
end

#worker_region ⇒ String

The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/ regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region. Corresponds to the JSON property workerRegion

Returns:

  • (String) 


1334
1335
1336
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1334

def worker_region
  @worker_region
end

#worker_zone ⇒ String

The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/ regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity. Corresponds to the JSON property workerZone

Returns:

  • (String) 


1343
1344
1345
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1343

def worker_zone
  @worker_zone
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
 
# File 'lib/google/apis/dataflow_v1b3/classes.rb', line 1350

def update!(**args)
  @cluster_manager_api_service = args[:cluster_manager_api_service] if args.key?(:cluster_manager_api_service)
  @dataset = args[:dataset] if args.key?(:dataset)
  @experiments = args[:experiments] if args.key?(:experiments)
  @flex_resource_scheduling_goal = args[:flex_resource_scheduling_goal] if args.key?(:flex_resource_scheduling_goal)
  @internal_experiments = args[:internal_experiments] if args.key?(:internal_experiments)
  @sdk_pipeline_options = args[:sdk_pipeline_options] if args.key?(:sdk_pipeline_options)
  @service_account_email = args[:service_account_email] if args.key?(:service_account_email)
  @service_kms_key_name = args[:service_kms_key_name] if args.key?(:service_kms_key_name)
  @service_options = args[:service_options] if args.key?(:service_options)
  @shuffle_mode = args[:shuffle_mode] if args.key?(:shuffle_mode)
  @temp_storage_prefix = args[:temp_storage_prefix] if args.key?(:temp_storage_prefix)
  @user_agent = args[:user_agent] if args.key?(:user_agent)
  @version = args[:version] if args.key?(:version)
  @worker_pools = args[:worker_pools] if args.key?(:worker_pools)
  @worker_region = args[:worker_region] if args.key?(:worker_region)
  @worker_zone = args[:worker_zone] if args.key?(:worker_zone)
end