Class: Google::Apis::DataprocV1beta2::HadoopJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1beta2::HadoopJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1beta2/classes.rb,
generated/google/apis/dataproc_v1beta2/representations.rb,
generated/google/apis/dataproc_v1beta2/representations.rb
Overview
A Cloud Dataproc job for running Apache Hadoop MapReduce (https://hadoop. apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/ docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
Instance Attribute Summary collapse
-
#archive_uris ⇒ Array<String>
Optional.
-
#args ⇒ Array<String>
Optional.
-
#file_uris ⇒ Array<String>
Optional.
-
#jar_file_uris ⇒ Array<String>
Optional.
-
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
-
#main_class ⇒ String
The name of the driver's main class.
-
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class.
-
#properties ⇒ Hash<String,String>
Optional.
Instance Method Summary collapse
-
#initialize(**args) ⇒ HadoopJob
constructor
A new instance of HadoopJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ HadoopJob
Returns a new instance of HadoopJob
1077 1078 1079 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1077 def initialize(**args) update!(**args) end |
Instance Attribute Details
#archive_uris ⇒ Array<String>
Optional. HCFS URIs of archives to be extracted in the working directory of
Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .
zip.
Corresponds to the JSON property archiveUris
1028 1029 1030 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1028 def archive_uris @archive_uris end |
#args ⇒ Array<String>
Optional. The arguments to pass to the driver. Do not include arguments, such
as -libjars or -Dfoo=bar, that can be set as job properties, since a collision
may occur that causes an incorrect job submission.
Corresponds to the JSON property args
1035 1036 1037 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1035 def args @args end |
#file_uris ⇒ Array<String>
Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to
the working directory of Hadoop drivers and distributed tasks. Useful for
naively parallel tasks.
Corresponds to the JSON property fileUris
1042 1043 1044 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1042 def file_uris @file_uris end |
#jar_file_uris ⇒ Array<String>
Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and
tasks.
Corresponds to the JSON property jarFileUris
1048 1049 1050 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1048 def jar_file_uris @jar_file_uris end |
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
Corresponds to the JSON property loggingConfig
1053 1054 1055 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1053 def logging_config @logging_config end |
#main_class ⇒ String
The name of the driver's main class. The jar file containing the class must be
in the default CLASSPATH or specified in jar_file_uris.
Corresponds to the JSON property mainClass
1059 1060 1061 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1059 def main_class @main_class end |
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-
bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-
samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-
mapreduce-examples.jar'
Corresponds to the JSON property mainJarFileUri
1067 1068 1069 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1067 def main_jar_file_uri @main_jar_file_uri end |
#properties ⇒ Hash<String,String>
Optional. A mapping of property names to values, used to configure Hadoop.
Properties that conflict with values set by the Cloud Dataproc API may be
overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes
in user code.
Corresponds to the JSON property properties
1075 1076 1077 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1075 def properties @properties end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 1082 def update!(**args) @archive_uris = args[:archive_uris] if args.key?(:archive_uris) @args = args[:args] if args.key?(:args) @file_uris = args[:file_uris] if args.key?(:file_uris) @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris) @logging_config = args[:logging_config] if args.key?(:logging_config) @main_class = args[:main_class] if args.key?(:main_class) @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri) @properties = args[:properties] if args.key?(:properties) end |