com.google.api.services.bigquery.model

Class SparkOptions

java.lang.Object

java.util.AbstractMap<String,Object>

com.google.api.client.util.GenericData

com.google.api.client.json.GenericJson

com.google.api.services.bigquery.model.SparkOptions

All Implemented Interfaces:

Cloneable, Map<String,Object>

public final class SparkOptions
extends com.google.api.client.json.GenericJson

Options for a user-defined Spark routine.

This is the Java data model class that specifies how to parse/serialize into the JSON that is transmitted over HTTP when working with the BigQuery API. For a detailed explanation see: https://developers.google.com/api-client-library/java/google-http-java-client/json

Author:

Google, Inc.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.api.client.util.GenericData

com.google.api.client.util.GenericData.Flags

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K,V>

Constructor Summary

Constructors

Constructor and Description
SparkOptions()

Method Summary

Modifier and Type	Method and Description
SparkOptions	clone()
List<String>	getArchiveUris() Archive files to be extracted into the working directory of each executor.
String	getConnection() Fully qualified name of the user-provided Spark connection object.
String	getContainerImage() Custom container image for the runtime environment.
List<String>	getFileUris() Files to be placed in the working directory of each executor.
List<String>	getJarUris() JARs to include on the driver and executor CLASSPATH.
String	getMainClass() The fully qualified name of a class in jar_uris, for example, com.example.wordcount.
String	getMainFileUri() The main file/jar URI of the Spark application.
Map<String,String>	getProperties() Configuration properties as a set of key/value pairs, which will be passed on to the Spark application.
List<String>	getPyFileUris() Python files to be placed on the PYTHONPATH for PySpark application.
String	getRuntimeVersion() Runtime version.
SparkOptions	set(String fieldName, Object value)
SparkOptions	setArchiveUris(List<String> archiveUris) Archive files to be extracted into the working directory of each executor.
SparkOptions	setConnection(String connection) Fully qualified name of the user-provided Spark connection object.
SparkOptions	setContainerImage(String containerImage) Custom container image for the runtime environment.
SparkOptions	setFileUris(List<String> fileUris) Files to be placed in the working directory of each executor.
SparkOptions	setJarUris(List<String> jarUris) JARs to include on the driver and executor CLASSPATH.
SparkOptions	setMainClass(String mainClass) The fully qualified name of a class in jar_uris, for example, com.example.wordcount.
SparkOptions	setMainFileUri(String mainFileUri) The main file/jar URI of the Spark application.
SparkOptions	setProperties(Map<String,String> properties) Configuration properties as a set of key/value pairs, which will be passed on to the Spark application.
SparkOptions	setPyFileUris(List<String> pyFileUris) Python files to be placed on the PYTHONPATH for PySpark application.
SparkOptions	setRuntimeVersion(String runtimeVersion) Runtime version.

Methods inherited from class com.google.api.client.json.GenericJson

getFactory, setFactory, toPrettyString, toString

Methods inherited from class com.google.api.client.util.GenericData

entrySet, equals, get, getClassInfo, getUnknownKeys, hashCode, put, putAll, remove, setUnknownKeys

Methods inherited from class java.util.AbstractMap

clear, containsKey, containsValue, isEmpty, keySet, size, values

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

compute, computeIfAbsent, computeIfPresent, forEach, getOrDefault, merge, putIfAbsent, remove, replace, replace, replaceAll

Constructor Detail

SparkOptions

public SparkOptions()

Method Detail

getArchiveUris

public List<String> getArchiveUris()

Archive files to be extracted into the working directory of each executor. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Returns:

value or null for none

setArchiveUris

public SparkOptions setArchiveUris(List<String> archiveUris)

Archive files to be extracted into the working directory of each executor. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Parameters:

archiveUris - archiveUris or null for none

getConnection

public String getConnection()

Fully qualified name of the user-provided Spark connection object. Format: ```"projects/{project_id}/locations/{location_id}/connections/{connection_id}"```

Returns:

value or null for none

setConnection

public SparkOptions setConnection(String connection)

Fully qualified name of the user-provided Spark connection object. Format: ```"projects/{project_id}/locations/{location_id}/connections/{connection_id}"```

Parameters:

connection - connection or null for none

getContainerImage

public String getContainerImage()

Custom container image for the runtime environment.

Returns:

value or null for none

setContainerImage

public SparkOptions setContainerImage(String containerImage)

Custom container image for the runtime environment.

Parameters:

containerImage - containerImage or null for none

getFileUris

public List<String> getFileUris()

Files to be placed in the working directory of each executor. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Returns:

value or null for none

setFileUris

public SparkOptions setFileUris(List<String> fileUris)

Files to be placed in the working directory of each executor. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Parameters:

fileUris - fileUris or null for none

getJarUris

public List<String> getJarUris()

JARs to include on the driver and executor CLASSPATH. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Returns:

value or null for none

setJarUris

public SparkOptions setJarUris(List<String> jarUris)

JARs to include on the driver and executor CLASSPATH. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Parameters:

jarUris - jarUris or null for none

getMainClass

public String getMainClass()

The fully qualified name of a class in jar_uris, for example, com.example.wordcount. Exactly one of main_class and main_jar_uri field should be set for Java/Scala language type.

Returns:

value or null for none

setMainClass

public SparkOptions setMainClass(String mainClass)

The fully qualified name of a class in jar_uris, for example, com.example.wordcount. Exactly one of main_class and main_jar_uri field should be set for Java/Scala language type.

Parameters:

mainClass - mainClass or null for none

getMainFileUri

public String getMainFileUri()

The main file/jar URI of the Spark application. Exactly one of the definition_body field and the main_file_uri field must be set for Python. Exactly one of main_class and main_file_uri field should be set for Java/Scala language type.

Returns:

value or null for none

setMainFileUri

public SparkOptions setMainFileUri(String mainFileUri)

The main file/jar URI of the Spark application. Exactly one of the definition_body field and the main_file_uri field must be set for Python. Exactly one of main_class and main_file_uri field should be set for Java/Scala language type.

Parameters:

mainFileUri - mainFileUri or null for none

getProperties

public Map<String,String> getProperties()

Configuration properties as a set of key/value pairs, which will be passed on to the Spark application. For more information, see [Apache Spark](https://spark.apache.org/docs/latest/index.html) and the [procedure option list](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition- language#procedure_option_list).

Returns:

value or null for none

setProperties

public SparkOptions setProperties(Map<String,String> properties)

Configuration properties as a set of key/value pairs, which will be passed on to the Spark application. For more information, see [Apache Spark](https://spark.apache.org/docs/latest/index.html) and the [procedure option list](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition- language#procedure_option_list).

Parameters:

properties - properties or null for none

getPyFileUris

public List<String> getPyFileUris()

Python files to be placed on the PYTHONPATH for PySpark application. Supported file types: `.py`, `.egg`, and `.zip`. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Returns:

value or null for none

setPyFileUris

public SparkOptions setPyFileUris(List<String> pyFileUris)

Python files to be placed on the PYTHONPATH for PySpark application. Supported file types: `.py`, `.egg`, and `.zip`. For more information about Apache Spark, see [Apache Spark](https://spark.apache.org/docs/latest/index.html).

Parameters:

pyFileUris - pyFileUris or null for none

getRuntimeVersion

public String getRuntimeVersion()

Runtime version. If not specified, the default runtime version is used.

Returns:

value or null for none

setRuntimeVersion

public SparkOptions setRuntimeVersion(String runtimeVersion)

Runtime version. If not specified, the default runtime version is used.

Parameters:

runtimeVersion - runtimeVersion or null for none

set

public SparkOptions set(String fieldName,
                        Object value)

Overrides:

set in class com.google.api.client.json.GenericJson

clone

public SparkOptions clone()

Overrides:

clone in class com.google.api.client.json.GenericJson