Class: Google::Apis::DataprocV1::Job

Inherits:
Object
  • Object
show all
Includes:
Core::Hashable, Core::JsonObjectSupport
Defined in:
lib/google/apis/dataproc_v1/classes.rb,
lib/google/apis/dataproc_v1/representations.rb,
lib/google/apis/dataproc_v1/representations.rb

Overview

A Dataproc job resource.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(**args) ⇒ Job

Returns a new instance of Job.



2658
2659
2660
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2658

def initialize(**args)
   update!(**args)
end

Instance Attribute Details

#done ⇒ Boolean Also known as: done?

Output only. Indicates whether the job is completed. If the value is false, the job is still in progress. If true, the job is completed, and status.state field will indicate if it was successful, failed, or cancelled. Corresponds to the JSON property done

Returns:

  • (Boolean) 


2524
2525
2526
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2524

def done
  @done
end

#driver_control_files_uri ⇒ String

Output only. If present, the location of miscellaneous control files which can be used as part of job setup and handling. If not present, control files might be placed in the same location as driver_output_uri. Corresponds to the JSON property driverControlFilesUri

Returns:

  • (String) 


2532
2533
2534
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2532

def driver_control_files_uri
  @driver_control_files_uri
end

#driver_output_resource_uri ⇒ String

Output only. A URI pointing to the location of the stdout of the job's driver program. Corresponds to the JSON property driverOutputResourceUri

Returns:

  • (String) 


2538
2539
2540
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2538

def driver_output_resource_uri
  @driver_output_resource_uri
end

#driver_scheduling_config ⇒ Google::Apis::DataprocV1::DriverSchedulingConfig

Driver scheduling configuration. Corresponds to the JSON property driverSchedulingConfig

Returns:



2543
2544
2545
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2543

def driver_scheduling_config
  @driver_scheduling_config
end

#flink_job ⇒ Google::Apis::DataprocV1::FlinkJob

A Dataproc job for running Apache Flink applications on YARN. Corresponds to the JSON property flinkJob

Returns:



2548
2549
2550
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2548

def flink_job
  @flink_job
end

#hadoop_job ⇒ Google::Apis::DataprocV1::HadoopJob

A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/ docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html). Corresponds to the JSON property hadoopJob

Returns:



2556
2557
2558
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2556

def hadoop_job
  @hadoop_job
end

#hive_job ⇒ Google::Apis::DataprocV1::HiveJob

A Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN. Corresponds to the JSON property hiveJob

Returns:



2562
2563
2564
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2562

def hive_job
  @hive_job
end

#job_uuid ⇒ String

Output only. A UUID that uniquely identifies a job within the project over time. This is in contrast to a user-settable reference.job_id that might be reused over time. Corresponds to the JSON property jobUuid

Returns:

  • (String) 


2569
2570
2571
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2569

def job_uuid
  @job_uuid
end

#labels ⇒ Hash<String,String>

Optional. The labels to associate with this job. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035. txt). Label values can be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt) . No more than 32 labels can be associated with a job. Corresponds to the JSON property labels

Returns:

  • (Hash<String,String>) 


2578
2579
2580
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2578

def labels
  @labels
end

#pig_job ⇒ Google::Apis::DataprocV1::PigJob

A Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN. Corresponds to the JSON property pigJob

Returns:



2584
2585
2586
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2584

def pig_job
  @pig_job
end

#placement ⇒ Google::Apis::DataprocV1::JobPlacement

Dataproc job config. Corresponds to the JSON property placement

Returns:



2589
2590
2591
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2589

def placement
  @placement
end

#presto_job ⇒ Google::Apis::DataprocV1::PrestoJob

A Dataproc job for running Presto (https://prestosql.io/) queries. IMPORTANT: The Dataproc Presto Optional Component (https://cloud.google.com/dataproc/docs/ concepts/components/presto) must be enabled when the cluster is created to submit a Presto job to the cluster. Corresponds to the JSON property prestoJob

Returns:



2597
2598
2599
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2597

def presto_job
  @presto_job
end

#pyspark_job ⇒ Google::Apis::DataprocV1::PySparkJob

A Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/ python-programming-guide.html) applications on YARN. Corresponds to the JSON property pysparkJob

Returns:



2603
2604
2605
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2603

def pyspark_job
  @pyspark_job
end

#reference ⇒ Google::Apis::DataprocV1::JobReference

Encapsulates the full scoping used to reference a job. Corresponds to the JSON property reference

Returns:



2608
2609
2610
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2608

def reference
  @reference
end

#scheduling ⇒ Google::Apis::DataprocV1::JobScheduling

Job scheduling options. Corresponds to the JSON property scheduling

Returns:



2613
2614
2615
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2613

def scheduling
  @scheduling
end

#spark_job ⇒ Google::Apis::DataprocV1::SparkJob

A Dataproc job for running Apache Spark (https://spark.apache.org/) applications on YARN. Corresponds to the JSON property sparkJob

Returns:



2619
2620
2621
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2619

def spark_job
  @spark_job
end

#spark_r_job ⇒ Google::Apis::DataprocV1::SparkRJob

A Dataproc job for running Apache SparkR (https://spark.apache.org/docs/latest/ sparkr.html) applications on YARN. Corresponds to the JSON property sparkRJob

Returns:



2625
2626
2627
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2625

def spark_r_job
  @spark_r_job
end

#spark_sql_job ⇒ Google::Apis::DataprocV1::SparkSqlJob

A Dataproc job for running Apache Spark SQL (https://spark.apache.org/sql/) queries. Corresponds to the JSON property sparkSqlJob

Returns:



2631
2632
2633
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2631

def spark_sql_job
  @spark_sql_job
end

#status ⇒ Google::Apis::DataprocV1::JobStatus

Dataproc job status. Corresponds to the JSON property status

Returns:



2636
2637
2638
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2636

def status
  @status
end

#status_history ⇒ Array<Google::Apis::DataprocV1::JobStatus>

Output only. The previous job status. Corresponds to the JSON property statusHistory

Returns:



2641
2642
2643
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2641

def status_history
  @status_history
end

#trino_job ⇒ Google::Apis::DataprocV1::TrinoJob

A Dataproc job for running Trino (https://trino.io/) queries. IMPORTANT: The Dataproc Trino Optional Component (https://cloud.google.com/dataproc/docs/ concepts/components/trino) must be enabled when the cluster is created to submit a Trino job to the cluster. Corresponds to the JSON property trinoJob

Returns:



2649
2650
2651
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2649

def trino_job
  @trino_job
end

#yarn_applications ⇒ Array<Google::Apis::DataprocV1::YarnApplication>

Output only. The collection of YARN applications spun up by this job.Beta Feature: This report is available for testing purposes only. It might be changed before final release. Corresponds to the JSON property yarnApplications

Returns:



2656
2657
2658
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2656

def yarn_applications
  @yarn_applications
end

Instance Method Details

#update!(**args) ⇒ Object

Update properties of this object



2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
 
# File 'lib/google/apis/dataproc_v1/classes.rb', line 2663

def update!(**args)
  @done = args[:done] if args.key?(:done)
  @driver_control_files_uri = args[:driver_control_files_uri] if args.key?(:driver_control_files_uri)
  @driver_output_resource_uri = args[:driver_output_resource_uri] if args.key?(:driver_output_resource_uri)
  @driver_scheduling_config = args[:driver_scheduling_config] if args.key?(:driver_scheduling_config)
  @flink_job = args[:flink_job] if args.key?(:flink_job)
  @hadoop_job = args[:hadoop_job] if args.key?(:hadoop_job)
  @hive_job = args[:hive_job] if args.key?(:hive_job)
  @job_uuid = args[:job_uuid] if args.key?(:job_uuid)
  @labels = args[:labels] if args.key?(:labels)
  @pig_job = args[:pig_job] if args.key?(:pig_job)
  @placement = args[:placement] if args.key?(:placement)
  @presto_job = args[:presto_job] if args.key?(:presto_job)
  @pyspark_job = args[:pyspark_job] if args.key?(:pyspark_job)
  @reference = args[:reference] if args.key?(:reference)
  @scheduling = args[:scheduling] if args.key?(:scheduling)
  @spark_job = args[:spark_job] if args.key?(:spark_job)
  @spark_r_job = args[:spark_r_job] if args.key?(:spark_r_job)
  @spark_sql_job = args[:spark_sql_job] if args.key?(:spark_sql_job)
  @status = args[:status] if args.key?(:status)
  @status_history = args[:status_history] if args.key?(:status_history)
  @trino_job = args[:trino_job] if args.key?(:trino_job)
  @yarn_applications = args[:yarn_applications] if args.key?(:yarn_applications)
end