Class HadoopJob

A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).

Inheritance

System.Object

HadoopJob

Implements

Google.Apis.Requests.IDirectResponseSchema

Inherited Members

System.Object.Equals(System.Object)

System.Object.Equals(System.Object, System.Object)

System.Object.GetHashCode()

System.Object.GetType()

System.Object.MemberwiseClone()

System.Object.ReferenceEquals(System.Object, System.Object)

System.Object.ToString()

Namespace: Google.Apis.Dataproc.v1beta2.Data

Assembly: Google.Apis.Dataproc.v1beta2.dll

Syntax

public class HadoopJob : IDirectResponseSchema

Properties

ArchiveUris

Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.

Declaration

[JsonProperty("archiveUris")]
public virtual IList<string> ArchiveUris { get; set; }

Property Value

Type	Description
System.Collections.Generic.IList<System.String>

Args

Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.

Declaration

[JsonProperty("args")]
public virtual IList<string> Args { get; set; }

Property Value

Type	Description
System.Collections.Generic.IList<System.String>

ETag

The ETag of the item.

Declaration

public virtual string ETag { get; set; }

Property Value

Type	Description
System.String

FileUris

Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.

Declaration

[JsonProperty("fileUris")]
public virtual IList<string> FileUris { get; set; }

Property Value

Type	Description
System.Collections.Generic.IList<System.String>

JarFileUris

Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.

Declaration

[JsonProperty("jarFileUris")]
public virtual IList<string> JarFileUris { get; set; }

Property Value

Type	Description
System.Collections.Generic.IList<System.String>

LoggingConfig

Optional. The runtime log config for job execution.

Declaration

[JsonProperty("loggingConfig")]
public virtual LoggingConfig LoggingConfig { get; set; }

Property Value

Type	Description
LoggingConfig

MainClass

The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.

Declaration

[JsonProperty("mainClass")]
public virtual string MainClass { get; set; }

Property Value

Type	Description
System.String

MainJarFileUri

The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics-binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:///home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'

Declaration

[JsonProperty("mainJarFileUri")]
public virtual string MainJarFileUri { get; set; }

Property Value

Type	Description
System.String

Properties

Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf/*-site and classes in user code.

Declaration

[JsonProperty("properties")]
public virtual IDictionary<string, string> Properties { get; set; }

Property Value

Type	Description
System.Collections.Generic.IDictionary<System.String, System.String>

Implements

Google.Apis.Requests.IDirectResponseSchema