Namespace Google.Apis.Dataproc.v1beta2.Data
Classes
AcceleratorConfig
Specifies the type and number of accelerator cards attached to the instances of an instance group (see GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/)).
AutoscalingConfig
Autoscaling Policy config associated with the cluster.
AutoscalingPolicy
Describes an autoscaling policy for Dataproc cluster autoscaler.
BasicAutoscalingAlgorithm
Basic algorithm for autoscaling.
BasicYarnAutoscalingConfig
Basic autoscaling configurations for YARN.
Binding
Associates members with a role.
CancelJobRequest
A request to cancel a job.
Cluster
Describes the identifying information, config, and status of a cluster of Compute Engine instances.
ClusterConfig
The cluster config.
ClusterMetrics
Contains cluster daemon metrics, such as HDFS and YARN stats.Beta Feature: This report is available for testing purposes only. It may be changed before final release.
ClusterOperation
The cluster operation triggered by a workflow.
ClusterOperationMetadata
Metadata describing the operation.
ClusterOperationStatus
The status of the operation.
ClusterSelector
A selector that chooses target cluster for jobs based on metadata.
ClusterStatus
The status of a cluster and its instances.
DiagnoseClusterRequest
A request to collect cluster diagnostic information.
DiagnoseClusterResults
The location of diagnostic output.
DiskConfig
Specifies the config of disk options for a group of VM instances.
Empty
A generic empty message that you can re-use to avoid defining duplicated empty messages in your APIs. A typical example is to use it as the request or the response type of an API method. For instance: service Foo { rpc Bar(google.protobuf.Empty) returns (google.protobuf.Empty); } The JSON representation for Empty is empty JSON object {}.
EncryptionConfig
Encryption settings for the cluster.
EndpointConfig
Endpoint config for this cluster
Expr
Represents a textual expression in the Common Expression Language (CEL) syntax. CEL is a C-like expression language. The syntax and semantics of CEL are documented at https://github.com/google/cel-spec.Example (Comparison): title: "Summary size limit" description: "Determines if a summary is less than 100 chars" expression: "document.summary.size() < 100" Example (Equality): title: "Requestor is owner" description: "Determines if requestor is the document owner" expression: "document.owner == request.auth.claims.email" Example (Logic): title: "Public documents" description: "Determine whether the document should be publicly visible" expression: "document.type != 'private' && document.type != 'internal'" Example (Data Manipulation): title: "Notification string" description: "Create a notification string with a timestamp." expression: "'New message received at ' + string(document.create_time)" The exact variables and functions that may be referenced within an expression are determined by the service that evaluates it. See the service documentation for additional information.
GceClusterConfig
Common config settings for resources of Compute Engine cluster instances, applicable to all instances in the cluster.
GetIamPolicyRequest
Request message for GetIamPolicy method.
GetPolicyOptions
Encapsulates settings provided to GetIamPolicy.
GkeClusterConfig
The GKE config for this cluster.
HadoopJob
A Dataproc job for running Apache Hadoop MapReduce (https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html) jobs on Apache Hadoop YARN (https://hadoop.apache.org/docs/r2.7.1/hadoop-yarn/hadoop-yarn-site/YARN.html).
HiveJob
A Dataproc job for running Apache Hive (https://hive.apache.org/) queries on YARN.
InjectCredentialsRequest
A request to inject credentials into a cluster.
InstanceGroupAutoscalingPolicyConfig
Configuration for the size bounds of an instance group, including its proportional size to other groups.
InstanceGroupConfig
The config settings for Compute Engine resources in an instance group, such as a master or worker group.
InstanceReference
A reference to a Compute Engine instance.
InstantiateWorkflowTemplateRequest
A request to instantiate a workflow template.
Job
A Dataproc job resource.
JobMetadata
Job Operation metadata.
JobPlacement
Dataproc job config.
JobReference
Encapsulates the full scoping used to reference a job.
JobScheduling
Job scheduling options.
JobStatus
Dataproc job status.
KerberosConfig
Specifies Kerberos related configuration.
LifecycleConfig
Specifies the cluster auto-delete schedule configuration.
ListAutoscalingPoliciesResponse
A response to a request to list autoscaling policies in a project.
ListClustersResponse
The list of all clusters in a project.
ListJobsResponse
A list of jobs in a project.
ListOperationsResponse
The response message for Operations.ListOperations.
ListWorkflowTemplatesResponse
A response to a request to list workflow templates in a project.
LoggingConfig
The runtime logging config of the job.
ManagedCluster
Cluster that is managed by the workflow.
ManagedGroupConfig
Specifies the resources used to actively manage an instance group.
MetastoreConfig
Specifies a Metastore configuration.
NamespacedGkeDeploymentTarget
A full, namespace-isolated deployment target for an existing GKE cluster.
NodeGroupAffinity
Node Group Affinity for clusters using sole-tenant node groups.
NodeInitializationAction
Specifies an executable to run on a fully configured node and a timeout period for executable completion.
Operation
This resource represents a long-running operation that is the result of a network API call.
OrderedJob
A job executed by the workflow.
ParameterValidation
Configuration for parameter validation.
PigJob
A Dataproc job for running Apache Pig (https://pig.apache.org/) queries on YARN.
Policy
An Identity and Access Management (IAM) policy, which specifies access controls for Google Cloud resources.A Policy is a collection of bindings. A binding binds one or more members to a single role. Members can be user accounts, service accounts, Google groups, and domains (such as G Suite). A role is a named list of permissions; each role can be an IAM predefined role or a user-created custom role.For some types of Google Cloud resources, a binding can also specify a condition, which is a logical expression that allows access to a resource only if the expression evaluates to true. A condition can add constraints based on attributes of the request, the resource, or both. To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).JSON example: { "bindings": [ { "role": "roles/resourcemanager.organizationAdmin", "members": [ "user:mike@example.com", "group:admins@example.com", "domain:google.com", "serviceAccount:my-project-id@appspot.gserviceaccount.com" ] }, { "role": "roles/resourcemanager.organizationViewer", "members": [ "user:eve@example.com" ], "condition": { "title": "expirable access", "description": "Does not grant access after Sep 2020", "expression": "request.time < timestamp('2020-10-01T00:00:00.000Z')", } } ], "etag": "BwWWja0YfJA=", "version": 3 } YAML example: bindings: - members: - user:mike@example.com - group:admins@example.com - domain:google.com - serviceAccount:my-project-id@appspot.gserviceaccount.com role: roles/resourcemanager.organizationAdmin - members: - user:eve@example.com role: roles/resourcemanager.organizationViewer condition: title: expirable access description: Does not grant access after Sep 2020 expression: request.time < timestamp('2020-10-01T00:00:00.000Z') etag: BwWWja0YfJA= version: 3 For a description of IAM and its features, see the IAM documentation (https://cloud.google.com/iam/docs/).
PrestoJob
A Dataproc job for running Presto (https://prestosql.io/) queries. IMPORTANT: The Dataproc Presto Optional Component (https://cloud.google.com/dataproc/docs/concepts/components/presto) must be enabled when the cluster is created to submit a Presto job to the cluster.
PySparkJob
A Dataproc job for running Apache PySpark (https://spark.apache.org/docs/0.9.0/python-programming-guide.html) applications on YARN.
QueryList
A list of queries to run on a cluster.
RegexValidation
Validation based on regular expressions.
ReservationAffinity
Reservation Affinity for consuming Zonal reservation.
SecurityConfig
Security related configuration, including encryption, Kerberos, etc.
SetIamPolicyRequest
Request message for SetIamPolicy method.
ShieldedInstanceConfig
Shielded Instance Config for clusters using Compute Engine Shielded VMs (https://cloud.google.com/security/shielded-cloud/shielded-vm).
SoftwareConfig
Specifies the selection and config of software inside the cluster.
SparkJob
A Dataproc job for running Apache Spark (http://spark.apache.org/) applications on YARN. The specification of the main method to call to drive the job. Specify either the jar file that contains the main class or the main class name. To pass both a main jar and a main class in that jar, add the jar to CommonJob.jar_file_uris, and then specify the main class name in main_class.
SparkRJob
A Dataproc job for running Apache SparkR (https://spark.apache.org/docs/latest/sparkr.html) applications on YARN.
SparkSqlJob
A Dataproc job for running Apache Spark SQL (http://spark.apache.org/sql/) queries.
StartClusterRequest
A request to start a cluster.
Status
The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC (https://github.com/grpc). Each Status message contains three pieces of data: error code, error message, and error details.You can find out more about this error model and how to work with it in the API Design Guide (https://cloud.google.com/apis/design/errors).
StopClusterRequest
A request to stop a cluster.
SubmitJobRequest
A request to submit a job.
TemplateParameter
A configurable parameter that replaces one or more fields in the template. Parameterizable fields: - Labels - File uris - Job properties - Job arguments - Script variables - Main class (in HadoopJob and SparkJob) - Zone (in ClusterSelector)
TestIamPermissionsRequest
Request message for TestIamPermissions method.
TestIamPermissionsResponse
Response message for TestIamPermissions method.
ValueValidation
Validation based on a list of allowed values.
WorkflowGraph
The workflow graph.
WorkflowMetadata
A Dataproc workflow template resource.
WorkflowNode
The workflow node.
WorkflowTemplate
A Dataproc workflow template resource.
WorkflowTemplatePlacement
Specifies workflow execution target.Either managed_cluster or cluster_selector is required.
YarnApplication
A YARN application created by a job. Application information is a subset of org.apache.hadoop.yarn.proto.YarnProtos.ApplicationReportProto.Beta Feature: This report is available for testing purposes only. It may be changed before final release.