Class: Google::Apis::DataprocV1::HadoopJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1::HadoopJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1/classes.rb,
generated/google/apis/dataproc_v1/representations.rb,
generated/google/apis/dataproc_v1/representations.rb
Overview
A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/ docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
Instance Attribute Summary collapse
-
#archive_uris ⇒ Array<String>
Optional.
-
#args ⇒ Array<String>
Optional.
-
#file_uris ⇒ Array<String>
Optional.
-
#jar_file_uris ⇒ Array<String>
Optional.
-
#logging_config ⇒ Google::Apis::DataprocV1::LoggingConfig
The runtime logging config of the job.
-
#main_class ⇒ String
The name of the driver's main class.
-
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class.
-
#properties ⇒ Hash<String,String>
Optional.
Instance Method Summary collapse
-
#initialize(**args) ⇒ HadoopJob
constructor
A new instance of HadoopJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ HadoopJob
Returns a new instance of HadoopJob.
1089 1090 1091 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1089 def initialize(**args) update!(**args) end |
Instance Attribute Details
#archive_uris ⇒ Array<String>
Optional. HCFS URIs of archives to be extracted in the working directory of
Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .
zip.
Corresponds to the JSON property archiveUris
1040 1041 1042 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1040 def archive_uris @archive_uris end |
#args ⇒ Array<String>
Optional. The arguments to pass to the driver. Do not include arguments, such
as -libjars or -Dfoo=bar, that can be set as job properties, since a collision
may occur that causes an incorrect job submission.
Corresponds to the JSON property args
1047 1048 1049 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1047 def args @args end |
#file_uris ⇒ Array<String>
Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to
the working directory of Hadoop drivers and distributed tasks. Useful for
naively parallel tasks.
Corresponds to the JSON property fileUris
1054 1055 1056 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1054 def file_uris @file_uris end |
#jar_file_uris ⇒ Array<String>
Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and
tasks.
Corresponds to the JSON property jarFileUris
1060 1061 1062 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1060 def jar_file_uris @jar_file_uris end |
#logging_config ⇒ Google::Apis::DataprocV1::LoggingConfig
The runtime logging config of the job.
Corresponds to the JSON property loggingConfig
1065 1066 1067 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1065 def logging_config @logging_config end |
#main_class ⇒ String
The name of the driver's main class. The jar file containing the class must be
in the default CLASSPATH or specified in jar_file_uris.
Corresponds to the JSON property mainClass
1071 1072 1073 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1071 def main_class @main_class end |
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-
bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-
samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-
mapreduce-examples.jar'
Corresponds to the JSON property mainJarFileUri
1079 1080 1081 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1079 def main_jar_file_uri @main_jar_file_uri end |
#properties ⇒ Hash<String,String>
Optional. A mapping of property names to values, used to configure Hadoop.
Properties that conflict with values set by the Dataproc API may be
overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes
in user code.
Corresponds to the JSON property properties
1087 1088 1089 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1087 def properties @properties end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 |
# File 'generated/google/apis/dataproc_v1/classes.rb', line 1094 def update!(**args) @archive_uris = args[:archive_uris] if args.key?(:archive_uris) @args = args[:args] if args.key?(:args) @file_uris = args[:file_uris] if args.key?(:file_uris) @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris) @logging_config = args[:logging_config] if args.key?(:logging_config) @main_class = args[:main_class] if args.key?(:main_class) @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri) @properties = args[:properties] if args.key?(:properties) end |