com.google.api.services.dataproc.v1beta2.model

Class HadoopJob

java.lang.Object

java.util.AbstractMap<String,Object>

com.google.api.client.util.GenericData

com.google.api.client.json.GenericJson

com.google.api.services.dataproc.v1beta2.model.HadoopJob

All Implemented Interfaces:

Cloneable, Map<String,Object>

public final class HadoopJob
extends GenericJson

A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current /hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).

This is the Java data model class that specifies how to parse/serialize into the JSON that is transmitted over HTTP when working with the Cloud Dataproc API. For a detailed explanation see: https://developers.google.com/api-client-library/java/google-http-java-client/json

Author:

Google, Inc.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.api.client.util.GenericData

GenericData.Flags

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K,V>

Constructor Summary

Constructors

Constructor and Description
HadoopJob()

Method Summary

Modifier and Type	Method and Description
HadoopJob	clone()
List<String>	getArchiveUris() Optional.
List<String>	getArgs() Optional.
List<String>	getFileUris() Optional.
List<String>	getJarFileUris() Optional.
LoggingConfig	getLoggingConfig() Optional.
String	getMainClass() The name of the driver's main class.
String	getMainJarFileUri() The HCFS URI of the jar file containing the main class.
Map<String,String>	getProperties() Optional.
HadoopJob	set(String fieldName, Object value)
HadoopJob	setArchiveUris(List<String> archiveUris) Optional.
HadoopJob	setArgs(List<String> args) Optional.
HadoopJob	setFileUris(List<String> fileUris) Optional.
HadoopJob	setJarFileUris(List<String> jarFileUris) Optional.
HadoopJob	setLoggingConfig(LoggingConfig loggingConfig) Optional.
HadoopJob	setMainClass(String mainClass) The name of the driver's main class.
HadoopJob	setMainJarFileUri(String mainJarFileUri) The HCFS URI of the jar file containing the main class.
HadoopJob	setProperties(Map<String,String> properties) Optional.

Methods inherited from class com.google.api.client.json.GenericJson

getFactory, setFactory, toPrettyString, toString

Methods inherited from class com.google.api.client.util.GenericData

entrySet, equals, get, getClassInfo, getUnknownKeys, hashCode, put, putAll, remove, setUnknownKeys

Methods inherited from class java.util.AbstractMap

clear, containsKey, containsValue, isEmpty, keySet, size, values

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

compute, computeIfAbsent, computeIfPresent, forEach, getOrDefault, merge, putIfAbsent, remove, replace, replace, replaceAll

Constructor Detail

HadoopJob

public HadoopJob()

Method Detail

getArchiveUris

public List<String> getArchiveUris()

Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.

Returns:

value or null for none

setArchiveUris

public HadoopJob setArchiveUris(List<String> archiveUris)

Optional. HCFS URIs of archives to be extracted in the working directory of Hadoop drivers and tasks. Supported file types: .jar, .tar, .tar.gz, .tgz, or .zip.

Parameters:

archiveUris - archiveUris or null for none

getArgs

public List<String> getArgs()

Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.

Returns:

value or null for none

setArgs

public HadoopJob setArgs(List<String> args)

Optional. The arguments to pass to the driver. Do not include arguments, such as -libjars or -Dfoo=bar, that can be set as job properties, since a collision may occur that causes an incorrect job submission.

Parameters:

args - args or null for none

getFileUris

public List<String> getFileUris()

Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.

Returns:

value or null for none

setFileUris

public HadoopJob setFileUris(List<String> fileUris)

Optional. HCFS (Hadoop Compatible Filesystem) URIs of files to be copied to the working directory of Hadoop drivers and distributed tasks. Useful for naively parallel tasks.

Parameters:

fileUris - fileUris or null for none

getJarFileUris

public List<String> getJarFileUris()

Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.

Returns:

value or null for none

setJarFileUris

public HadoopJob setJarFileUris(List<String> jarFileUris)

Optional. Jar file URIs to add to the CLASSPATHs of the Hadoop driver and tasks.

Parameters:

jarFileUris - jarFileUris or null for none

getLoggingConfig

public LoggingConfig getLoggingConfig()

Optional. The runtime log config for job execution.

Returns:

value or null for none

setLoggingConfig

public HadoopJob setLoggingConfig(LoggingConfig loggingConfig)

Optional. The runtime log config for job execution.

Parameters:

loggingConfig - loggingConfig or null for none

getMainClass

public String getMainClass()

The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.

Returns:

value or null for none

setMainClass

public HadoopJob setMainClass(String mainClass)

The name of the driver's main class. The jar file containing the class must be in the default CLASSPATH or specified in jar_file_uris.

Parameters:

mainClass - mainClass or null for none

getMainJarFileUri

public String getMainJarFileUri()

The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics- binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'

Returns:

value or null for none

setMainJarFileUri

public HadoopJob setMainJarFileUri(String mainJarFileUri)

The HCFS URI of the jar file containing the main class. Examples: 'gs://foo-bucket/analytics- binaries/extract-useful-metrics-mr.jar' 'hdfs:/tmp/test-samples/custom-wordcount.jar' 'file:home/usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar'

Parameters:

mainJarFileUri - mainJarFileUri or null for none

getProperties

public Map<String,String> getProperties()

Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf-site and classes in user code.

Returns:

value or null for none

setProperties

public HadoopJob setProperties(Map<String,String> properties)

Optional. A mapping of property names to values, used to configure Hadoop. Properties that conflict with values set by the Dataproc API may be overwritten. Can include properties set in /etc/hadoop/conf-site and classes in user code.

Parameters:

properties - properties or null for none

set

public HadoopJob set(String fieldName,
                     Object value)

Overrides:

set in class GenericJson

clone

public HadoopJob clone()

Overrides:

clone in class GenericJson