Class: Google::Apis::DataprocV1beta2::OrderedJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1beta2::OrderedJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1beta2/classes.rb,
generated/google/apis/dataproc_v1beta2/representations.rb,
generated/google/apis/dataproc_v1beta2/representations.rb
Overview
A job executed by the workflow.
Instance Attribute Summary collapse
-
#hadoop_job ⇒ Google::Apis::DataprocV1beta2::HadoopJob
A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/ docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
-
#hive_job ⇒ Google::Apis::DataprocV1beta2::HiveJob
A Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN.
-
#labels ⇒ Hash<String,String>
Optional.
-
#pig_job ⇒ Google::Apis::DataprocV1beta2::PigJob
A Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN.
-
#prerequisite_step_ids ⇒ Array<String>
Optional.
-
#pyspark_job ⇒ Google::Apis::DataprocV1beta2::PySparkJob
A Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/ python-programming-guide.html) applications on YARN.
-
#scheduling ⇒ Google::Apis::DataprocV1beta2::JobScheduling
Job scheduling options.
-
#spark_job ⇒ Google::Apis::DataprocV1beta2::SparkJob
A Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN.
-
#spark_sql_job ⇒ Google::Apis::DataprocV1beta2::SparkSqlJob
A Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries.
-
#step_id ⇒ String
Required.
Instance Method Summary collapse
-
#initialize(**args) ⇒ OrderedJob
constructor
A new instance of OrderedJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ OrderedJob
Returns a new instance of OrderedJob.
2193 2194 2195 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2193 def initialize(**args) update!(**args) end |
Instance Attribute Details
#hadoop_job ⇒ Google::Apis::DataprocV1beta2::HadoopJob
A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/
docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/
MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/
docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
Corresponds to the JSON property hadoopJob
2128 2129 2130 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2128 def hadoop_job @hadoop_job end |
#hive_job ⇒ Google::Apis::DataprocV1beta2::HiveJob
A Dataproc job for running Apache Hive (https://hive.apache.org/) queries on
YARN.
Corresponds to the JSON property hiveJob
2134 2135 2136 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2134 def hive_job @hive_job end |
#labels ⇒ Hash<String,String>
Optional. The labels to associate with this job.Label keys must be between 1
and 63 characters long, and must conform to the following regular expression: \
pLl
\pLo
0,62
Label values must be between 1 and 63 characters long, and
must conform to the following regular expression: \pLl
\pLo
\pN
_-0,63
No
more than 32 labels can be associated with a given job.
Corresponds to the JSON property labels
2143 2144 2145 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2143 def labels @labels end |
#pig_job ⇒ Google::Apis::DataprocV1beta2::PigJob
A Dataproc job for running Apache Pig (https://pig.apache.org/) queries on
YARN.
Corresponds to the JSON property pigJob
2149 2150 2151 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2149 def pig_job @pig_job end |
#prerequisite_step_ids ⇒ Array<String>
Optional. The optional list of prerequisite job step_ids. If not specified,
the job will start at the beginning of workflow.
Corresponds to the JSON property prerequisiteStepIds
2155 2156 2157 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2155 def prerequisite_step_ids @prerequisite_step_ids end |
#pyspark_job ⇒ Google::Apis::DataprocV1beta2::PySparkJob
A Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/
python-programming-guide.html) applications on YARN.
Corresponds to the JSON property pysparkJob
2161 2162 2163 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2161 def pyspark_job @pyspark_job end |
#scheduling ⇒ Google::Apis::DataprocV1beta2::JobScheduling
Job scheduling options.
Corresponds to the JSON property scheduling
2166 2167 2168 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2166 def scheduling @scheduling end |
#spark_job ⇒ Google::Apis::DataprocV1beta2::SparkJob
A Dataproc job for running Apache Spark (http://spark.apache.org/)
applications on YARN. The specification of the main method to call to drive
the job. Specify either the jar file that contains the main class or the main
class name. To pass both a main jar and a main class in that jar, add the jar
to CommonJob.jar_file_uris, and then specify the main class name in main_class.
Corresponds to the JSON property sparkJob
2175 2176 2177 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2175 def spark_job @spark_job end |
#spark_sql_job ⇒ Google::Apis::DataprocV1beta2::SparkSqlJob
A Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/)
queries.
Corresponds to the JSON property sparkSqlJob
2181 2182 2183 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2181 def spark_sql_job @spark_sql_job end |
#step_id ⇒ String
Required. The step id. The id must be unique among all jobs within the
template.The step id is used as prefix for job id, as job goog-dataproc-
workflow-step-id label, and in prerequisiteStepIds field from other steps.The
id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and
hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of
between 3 and 50 characters.
Corresponds to the JSON property stepId
2191 2192 2193 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2191 def step_id @step_id end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2198 def update!(**args) @hadoop_job = args[:hadoop_job] if args.key?(:hadoop_job) @hive_job = args[:hive_job] if args.key?(:hive_job) @labels = args[:labels] if args.key?(:labels) @pig_job = args[:pig_job] if args.key?(:pig_job) @prerequisite_step_ids = args[:prerequisite_step_ids] if args.key?(:prerequisite_step_ids) @pyspark_job = args[:pyspark_job] if args.key?(:pyspark_job) @scheduling = args[:scheduling] if args.key?(:scheduling) @spark_job = args[:spark_job] if args.key?(:spark_job) @spark_sql_job = args[:spark_sql_job] if args.key?(:spark_sql_job) @step_id = args[:step_id] if args.key?(:step_id) end |