Class: Google::Apis::DataprocV1beta2::SparkJob
- Inherits:
-
Object
- Object
- Google::Apis::DataprocV1beta2::SparkJob
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataproc_v1beta2/classes.rb,
generated/google/apis/dataproc_v1beta2/representations.rb,
generated/google/apis/dataproc_v1beta2/representations.rb
Overview
A Cloud Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN.
Instance Attribute Summary collapse
-
#archive_uris ⇒ Array<String>
Optional.
-
#args ⇒ Array<String>
Optional.
-
#file_uris ⇒ Array<String>
Optional.
-
#jar_file_uris ⇒ Array<String>
Optional.
-
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
-
#main_class ⇒ String
The name of the driver's main class.
-
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file that contains the main class.
-
#properties ⇒ Hash<String,String>
Optional.
Instance Method Summary collapse
-
#initialize(**args) ⇒ SparkJob
constructor
A new instance of SparkJob.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ SparkJob
Returns a new instance of SparkJob
2701 2702 2703 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2701 def initialize(**args) update!(**args) end |
Instance Attribute Details
#archive_uris ⇒ Array<String>
Optional. HCFS URIs of archives to be extracted in the working directory of
Spark drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, and .
zip.
Corresponds to the JSON property archiveUris
2656 2657 2658 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2656 def archive_uris @archive_uris end |
#args ⇒ Array<String>
Optional. The arguments to pass to the driver. Do not include arguments, such
as --conf, that can be set as job properties, since a collision may occur that
causes an incorrect job submission.
Corresponds to the JSON property args
2663 2664 2665 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2663 def args @args end |
#file_uris ⇒ Array<String>
Optional. HCFS URIs of files to be copied to the working directory of Spark
drivers and distributed tasks. Useful for naively parallel tasks.
Corresponds to the JSON property fileUris
2669 2670 2671 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2669 def file_uris @file_uris end |
#jar_file_uris ⇒ Array<String>
Optional. HCFS URIs of jar files to add to the CLASSPATHs of the Spark driver
and tasks.
Corresponds to the JSON property jarFileUris
2675 2676 2677 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2675 def jar_file_uris @jar_file_uris end |
#logging_config ⇒ Google::Apis::DataprocV1beta2::LoggingConfig
The runtime logging config of the job.
Corresponds to the JSON property loggingConfig
2680 2681 2682 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2680 def logging_config @logging_config end |
#main_class ⇒ String
The name of the driver's main class. The jar file that contains the class must
be in the default CLASSPATH or specified in jar_file_uris.
Corresponds to the JSON property mainClass
2686 2687 2688 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2686 def main_class @main_class end |
#main_jar_file_uri ⇒ String
The HCFS URI of the jar file that contains the main class.
Corresponds to the JSON property mainJarFileUri
2691 2692 2693 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2691 def main_jar_file_uri @main_jar_file_uri end |
#properties ⇒ Hash<String,String>
Optional. A mapping of property names to values, used to configure Spark.
Properties that conflict with values set by the Cloud Dataproc API may be
overwritten. Can include properties set in /etc/spark/conf/spark-defaults.conf
and classes in user code.
Corresponds to the JSON property properties
2699 2700 2701 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2699 def properties @properties end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 |
# File 'generated/google/apis/dataproc_v1beta2/classes.rb', line 2706 def update!(**args) @archive_uris = args[:archive_uris] if args.key?(:archive_uris) @args = args[:args] if args.key?(:args) @file_uris = args[:file_uris] if args.key?(:file_uris) @jar_file_uris = args[:jar_file_uris] if args.key?(:jar_file_uris) @logging_config = args[:logging_config] if args.key?(:logging_config) @main_class = args[:main_class] if args.key?(:main_class) @main_jar_file_uri = args[:main_jar_file_uri] if args.key?(:main_jar_file_uri) @properties = args[:properties] if args.key?(:properties) end |