Class: Google::Apis::DataprocV1beta2::HadoopJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1beta2::HadoopJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1beta2/classes.rb,
generated/google/apis/dataproc_v1beta2/representations.rb,
generated/google/apis/dataproc_v1beta2/representations.rb
Overview
A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop. apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
Instance Attribute Summary collapse
-
#archive_uris ⇒ Array<String>
Optional.
-
#args ⇒ Array<String>
Optional.
-
#file_uris ⇒ Array<String>
Optional.
-
#jar_file_uris ⇒ Array<String>
Optional.
-
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
-
#main_class ⇒ String
The name of the driver's main class.
-
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class.
-
#properties ⇒ Hash<String,String>
Optional.
Instance Method Summary collapse
-
#initialize(**args) ⇒ HadoopJob
constructor
A new instance of HadoopJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ HadoopJob
Returns a new instance of HadoopJob
777 778 779 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 777 def initialize(**args) update!(**args) end |
Instance Attribute Details
#archive_uris ⇒ Array<String>
Optional. HCFS URIs of archives to be extracted in the working directory of
Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .
zip.
Corresponds to the JSON property archiveUris
728 729 730 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 728 def archive_uris @archive_uris end |
#args ⇒ Array<String>
Optional. The arguments to pass to the driver. Do not include arguments, such
as -libjars or -Dfoo=bar, that can be set as job properties, since a collision
may occur that causes an incorrect job submission.
Corresponds to the JSON property args
735 736 737 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 735 def args @args end |
#file_uris ⇒ Array<String>
Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to
the working directory of Hadoop drivers and distributed tasks. Useful for
naively parallel tasks.
Corresponds to the JSON property fileUris
742 743 744 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 742 def file_uris @file_uris end |
#jar_file_uris ⇒ Array<String>
Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and
tasks.
Corresponds to the JSON property jarFileUris
748 749 750 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 748 def jar_file_uris @jar_file_uris end |
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
Corresponds to the JSON property loggingConfig
753 754 755 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 753 def logging_config @logging_config end |
#main_class ⇒ String
The name of the driver's main class. The jar file containing the class must be
in the default CLASSPATH or specified in jar_file_uris.
Corresponds to the JSON property mainClass
759 760 761 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 759 def main_class @main_class end |
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-
bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-
samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-
mapreduce-examples.jar'
Corresponds to the JSON property mainJarFileUri
767 768 769 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 767 def main_jar_file_uri @main_jar_file_uri end |
#properties ⇒ Hash<String,String>
Optional. A mapping of property names to values, used to configure Hadoop.
Properties that conflict with values set by the Cloud Dataproc API may be
overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes
in user code.
Corresponds to the JSON property properties
775 776 777 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 775 def properties @properties end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
782 783 784 785 786 787 788 789 790 791 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 782 def update!(**args) @archive_uris = args[:archive_uris] if args.key?(:archive_uris) @args = args[:args] if args.key?(:args) @file_uris = args[:file_uris] if args.key?(:file_uris) @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris) @logging_config = args[:logging_config] if args.key?(:logging_config) @main_class = args[:main_class] if args.key?(:main_class) @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri) @properties = args[:properties] if args.key?(:properties) end |