Class: Google::Apis::DataprocV1::HadoopJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1::HadoopJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1/classes.rb,
generated/google/apis/dataproc_v1/representations.rb,
generated/google/apis/dataproc_v1/representations.rb
Overview
A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop. apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
Instance Attribute Summary collapse
-
#archive_uris ⇒ Array<String>
Optional.
-
#args ⇒ Array<String>
Optional.
-
#file_uris ⇒ Array<String>
Optional.
-
#jar_file_uris ⇒ Array<String>
Optional.
-
#logging_config ⇒ Google::Apis::DataprocV1::LoggingConfig
The runtime logging config of the job.
-
#main_class ⇒ String
The name of the driver's main class.
-
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class.
-
#properties ⇒ Hash<String,String>
Optional.
Instance Method Summary collapse
-
#initialize(**args) ⇒ HadoopJob
constructor
A new instance of HadoopJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ HadoopJob
Returns a new instance of HadoopJob
666 667 668 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 666 def initialize(**args) update!(**args) end |
Instance Attribute Details
#archive_uris ⇒ Array<String>
Optional. HCFS URIs of archives to be extracted in the working directory of
Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .
zip.
Corresponds to the JSON property archiveUris
617 618 619 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 617 def archive_uris @archive_uris end |
#args ⇒ Array<String>
Optional. The arguments to pass to the driver. Do not include arguments, such
as -libjars or -Dfoo=bar, that can be set as job properties, since a collision
may occur that causes an incorrect job submission.
Corresponds to the JSON property args
624 625 626 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 624 def args @args end |
#file_uris ⇒ Array<String>
Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to
the working directory of Hadoop drivers and distributed tasks. Useful for
naively parallel tasks.
Corresponds to the JSON property fileUris
631 632 633 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 631 def file_uris @file_uris end |
#jar_file_uris ⇒ Array<String>
Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and
tasks.
Corresponds to the JSON property jarFileUris
637 638 639 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 637 def jar_file_uris @jar_file_uris end |
#logging_config ⇒ Google::Apis::DataprocV1::LoggingConfig
The runtime logging config of the job.
Corresponds to the JSON property loggingConfig
642 643 644 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 642 def logging_config @logging_config end |
#main_class ⇒ String
The name of the driver's main class. The jar file containing the class must be
in the default CLASSPATH or specified in jar_file_uris.
Corresponds to the JSON property mainClass
648 649 650 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 648 def main_class @main_class end |
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-
bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-
samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-
mapreduce-examples.jar'
Corresponds to the JSON property mainJarFileUri
656 657 658 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 656 def main_jar_file_uri @main_jar_file_uri end |
#properties ⇒ Hash<String,String>
Optional. A mapping of property names to values, used to configure Hadoop.
Properties that conflict with values set by the Cloud Dataproc API may be
overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes
in user code.
Corresponds to the JSON property properties
664 665 666 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 664 def properties @properties end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
671 672 673 674 675 676 677 678 679 680 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 671 def update!(**args) @archive_uris = args[:archive_uris] if args.key?(:archive_uris) @args = args[:args] if args.key?(:args) @file_uris = args[:file_uris] if args.key?(:file_uris) @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris) @logging_config = args[:logging_config] if args.key?(:logging_config) @main_class = args[:main_class] if args.key?(:main_class) @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri) @properties = args[:properties] if args.key?(:properties) end |