Class: Google::Apis::DataprocV1beta2::HadoopJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1beta2::HadoopJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1beta2/classes.rb,
generated/google/apis/dataproc_v1beta2/representations.rb,
generated/google/apis/dataproc_v1beta2/representations.rb
Overview
A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop. apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
Instance Attribute Summary collapse
-
#archive_uris ⇒ Array<String>
Optional.
-
#args ⇒ Array<String>
Optional.
-
#file_uris ⇒ Array<String>
Optional.
-
#jar_file_uris ⇒ Array<String>
Optional.
-
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
-
#main_class ⇒ String
The name of the driver's main class.
-
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class.
-
#properties ⇒ Hash<String,String>
Optional.
Instance Method Summary collapse
-
#initialize(**args) ⇒ HadoopJob
constructor
A new instance of HadoopJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ HadoopJob
Returns a new instance of HadoopJob
736 737 738 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 736 def initialize(**args) update!(**args) end |
Instance Attribute Details
#archive_uris ⇒ Array<String>
Optional. HCFS URIs of archives to be extracted in the working directory of
Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .
zip.
Corresponds to the JSON property archiveUris
687 688 689 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 687 def archive_uris @archive_uris end |
#args ⇒ Array<String>
Optional. The arguments to pass to the driver. Do not include arguments, such
as -libjars or -Dfoo=bar, that can be set as job properties, since a collision
may occur that causes an incorrect job submission.
Corresponds to the JSON property args
694 695 696 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 694 def args @args end |
#file_uris ⇒ Array<String>
Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to
the working directory of Hadoop drivers and distributed tasks. Useful for
naively parallel tasks.
Corresponds to the JSON property fileUris
701 702 703 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 701 def file_uris @file_uris end |
#jar_file_uris ⇒ Array<String>
Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and
tasks.
Corresponds to the JSON property jarFileUris
707 708 709 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 707 def jar_file_uris @jar_file_uris end |
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
Corresponds to the JSON property loggingConfig
712 713 714 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 712 def logging_config @logging_config end |
#main_class ⇒ String
The name of the driver's main class. The jar file containing the class must be
in the default CLASSPATH or specified in jar_file_uris.
Corresponds to the JSON property mainClass
718 719 720 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 718 def main_class @main_class end |
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-
bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-
samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-
mapreduce-examples.jar'
Corresponds to the JSON property mainJarFileUri
726 727 728 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 726 def main_jar_file_uri @main_jar_file_uri end |
#properties ⇒ Hash<String,String>
Optional. A mapping of property names to values, used to configure Hadoop.
Properties that conflict with values set by the Cloud Dataproc API may be
overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes
in user code.
Corresponds to the JSON property properties
734 735 736 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 734 def properties @properties end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
741 742 743 744 745 746 747 748 749 750 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 741 def update!(**args) @archive_uris = args[:archive_uris] if args.key?(:archive_uris) @args = args[:args] if args.key?(:args) @file_uris = args[:file_uris] if args.key?(:file_uris) @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris) @logging_config = args[:logging_config] if args.key?(:logging_config) @main_class = args[:main_class] if args.key?(:main_class) @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri) @properties = args[:properties] if args.key?(:properties) end |