Class: Google::Apis::DatapipelinesV1::DatapipelinesService

Inherits:
Core::BaseService
  • Object
show all
Defined in:
lib/google/apis/datapipelines_v1/service.rb

Overview

Data pipelines API

Data Pipelines provides an interface for creating, updating, and managing recurring Data Analytics jobs.

Examples:

require 'google/apis/datapipelines_v1'

Datapipelines = Google::Apis::DatapipelinesV1 # Alias the module
service = Datapipelines::DatapipelinesService.new

See Also:

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ DatapipelinesService

Returns a new instance of DatapipelinesService.



46
47
48
49
50
51
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 46

def initialize
  super('https://datapipelines.googleapis.com/', '',
        client_name: 'google-apis-datapipelines_v1',
        client_version: Google::Apis::DatapipelinesV1::GEM_VERSION)
  @batch_path = 'batch'
end

Instance Attribute Details

#key ⇒ String

Returns API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.

Returns:

  • (String) —

    API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.



39
40
41
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 39

def key
  @key
end

#quota_user ⇒ String

Returns Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

Returns:

  • (String) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.



44
45
46
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 44

def quota_user
  @quota_user
end

Instance Method Details

#create_project_location_pipeline(parent, google_cloud_datapipelines_v1_pipeline_object = nil, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline

Creates a pipeline. For a batch pipeline, you can pass scheduler information. Data Pipelines uses the scheduler information to create an internal scheduler that runs jobs periodically. If the internal scheduler is not configured, you can use RunPipeline to run jobs.

Parameters:

  • parent (String) —

    Required. The location name. For example: projects/PROJECT_ID/locations/ LOCATION_ID.

  • google_cloud_datapipelines_v1_pipeline_object (Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline) (defaults to: nil)
  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



131
132
133
134
135
136
137
138
139
140
141
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 131

def create_project_location_pipeline(parent, google_cloud_datapipelines_v1_pipeline_object = nil, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:post, 'v1/{+parent}/pipelines', options)
  command.request_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline::Representation
  command.request_object = google_cloud_datapipelines_v1_pipeline_object
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline
  command.params['parent'] = parent unless parent.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end

#delete_project_location_pipeline(name, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleProtobufEmpty

Deletes a pipeline. If a scheduler job is attached to the pipeline, it will be deleted.

Parameters:

  • name (String) —

    Required. The pipeline name. For example: projects/PROJECT_ID/locations/ LOCATION_ID/pipelines/PIPELINE_ID.

  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



165
166
167
168
169
170
171
172
173
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 165

def delete_project_location_pipeline(name, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:delete, 'v1/{+name}', options)
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleProtobufEmpty::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleProtobufEmpty
  command.params['name'] = name unless name.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end

#get_project_location_pipeline(name, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline

Looks up a single pipeline. Returns a "NOT_FOUND" error if no such pipeline exists. Returns a "FORBIDDEN" error if the caller doesn't have permission to access it.

Parameters:

  • name (String) —

    Required. The pipeeline name. For example: projects/PROJECT_ID/locations/ LOCATION_ID/pipelines/PIPELINE_ID.

  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



198
199
200
201
202
203
204
205
206
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 198

def get_project_location_pipeline(name, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:get, 'v1/{+name}', options)
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline
  command.params['name'] = name unless name.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end

#list_project_location_pipelines(parent, filter: nil, page_size: nil, page_token: nil, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1ListPipelinesResponse

Lists pipelines. Returns a "NOT_FOUND" error if the list is empty. Returns a " FORBIDDEN" error if the caller doesn't have permission to access it.

Parameters:

  • parent (String) —

    Required. The location name. For example: projects/PROJECT_ID/locations/ LOCATION_ID.

  • filter (String) (defaults to: nil) —

    An expression for filtering the results of the request. If unspecified, all pipelines will be returned. Multiple filters can be applied and must be comma separated. Fields eligible for filtering are: + type: The type of the pipeline (streaming or batch). Allowed values are ALL, BATCH, and STREAMING. + executor_type: The type of pipeline execution layer. This is always Dataflow for now, but more executors may be added later. Allowed values are ALL and DATAFLOW. + status: The activity status of the pipeline. Allowed values are ALL, ACTIVE, ARCHIVED, and PAUSED. For example, to limit results to active batch processing pipelines: type:BATCH,status:ACTIVE

  • page_size (Fixnum) (defaults to: nil) —

    The maximum number of entities to return. The service may return fewer than this value, even if there are additional pages. If unspecified, the max limit is yet to be determined by the backend implementation.

  • page_token (String) (defaults to: nil) —

    A page token, received from a previous ListPipelines call. Provide this to retrieve the subsequent page. When paginating, all other parameters provided to ListPipelines must match the call that provided the page token.

  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



93
94
95
96
97
98
99
100
101
102
103
104
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 93

def list_project_location_pipelines(parent, filter: nil, page_size: nil, page_token: nil, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:get, 'v1/{+parent}', options)
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1ListPipelinesResponse::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1ListPipelinesResponse
  command.params['parent'] = parent unless parent.nil?
  command.query['filter'] = filter unless filter.nil?
  command.query['pageSize'] = page_size unless page_size.nil?
  command.query['pageToken'] = page_token unless page_token.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end

#patch_project_location_pipeline(name, google_cloud_datapipelines_v1_pipeline_object = nil, update_mask: nil, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline

Updates a pipeline. If successful, the updated [Pipeline] is returned. Returns NOT_FOUND if the pipeline doesn't exist. If UpdatePipeline does not return successfully, you can retry the UpdatePipeline request until you receive a successful response.

Parameters:

  • name (String) —

    The pipeline name. For example: projects/PROJECT_ID/locations/LOCATION_ID/ pipelines/PIPELINE_ID. * PROJECT_ID can contain letters ([A-Za-z]), numbers ([0-9]), hyphens (-), colons (:), and periods (.). For more information, see Identifying projects * LOCATION_ID is the canonical ID for the pipeline's location. The list of available locations can be obtained by calling ListLocations. Note that the Data Pipelines service is not available in all regions. It depends on Cloud Scheduler, an App Engine application, so it's only available in App Engine regions. * PIPELINE_ID is the ID of the pipeline. Must be unique for the selected project and location.

  • google_cloud_datapipelines_v1_pipeline_object (Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline) (defaults to: nil)
  • update_mask (String) (defaults to: nil) —

    The list of fields to be updated.

  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



244
245
246
247
248
249
250
251
252
253
254
255
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 244

def patch_project_location_pipeline(name, google_cloud_datapipelines_v1_pipeline_object = nil, update_mask: nil, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:patch, 'v1/{+name}', options)
  command.request_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline::Representation
  command.request_object = google_cloud_datapipelines_v1_pipeline_object
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline
  command.params['name'] = name unless name.nil?
  command.query['updateMask'] = update_mask unless update_mask.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end

#run_project_location_pipeline(name, google_cloud_datapipelines_v1_run_pipeline_request_object = nil, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1RunPipelineResponse

Creates a job for the specified pipeline directly. You can use this method when the internal scheduler is not configured and you want to trigger the job directly or through an external system. Returns a "NOT_FOUND" error if the pipeline doesn't exist. Returns a "FOBIDDEN" error if the user doesn't have permission to access the pipeline or run jobs for the pipeline.

Parameters:

  • name (String) —

    Required. The pipeline name. For example: projects/PROJECT_ID/locations/ LOCATION_ID/pipelines/PIPELINE_ID.

  • google_cloud_datapipelines_v1_run_pipeline_request_object (Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1RunPipelineRequest) (defaults to: nil)
  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



283
284
285
286
287
288
289
290
291
292
293
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 283

def run_project_location_pipeline(name, google_cloud_datapipelines_v1_run_pipeline_request_object = nil, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:post, 'v1/{+name}:run', options)
  command.request_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1RunPipelineRequest::Representation
  command.request_object = google_cloud_datapipelines_v1_run_pipeline_request_object
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1RunPipelineResponse::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1RunPipelineResponse
  command.params['name'] = name unless name.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end

#stop_project_location_pipeline(name, google_cloud_datapipelines_v1_stop_pipeline_request_object = nil, fields: nil, quota_user: nil, options: nil) {|result, err| ... } ⇒ Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline

Freezes pipeline execution permanently. If there's a corresponding scheduler entry, it's deleted, and the pipeline state is changed to "ARCHIVED". However, pipeline metadata is retained. Upon success, the pipeline state is updated to ARCHIVED.

Parameters:

  • name (String) —

    Required. The pipeline name. For example: projects/PROJECT_ID/locations/ LOCATION_ID/pipelines/PIPELINE_ID.

  • google_cloud_datapipelines_v1_stop_pipeline_request_object (Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1StopPipelineRequest) (defaults to: nil)
  • fields (String) (defaults to: nil) —

    Selector specifying which fields to include in a partial response.

  • quota_user (String) (defaults to: nil) —

    Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.

  • options (Google::Apis::RequestOptions) (defaults to: nil) —

    Request-specific options

Yields:

  • (result, err) —

    Result & error if block supplied

Yield Parameters:

Returns:

Raises:

  • (Google::Apis::ServerError) —

    An error occurred on the server and the request can be retried

  • (Google::Apis::ClientError) —

    The request is invalid and should not be retried without modification

  • (Google::Apis::AuthorizationError) —

    Authorization is required



320
321
322
323
324
325
326
327
328
329
330
 
# File 'lib/google/apis/datapipelines_v1/service.rb', line 320

def stop_project_location_pipeline(name, google_cloud_datapipelines_v1_stop_pipeline_request_object = nil, fields: nil, quota_user: nil, options: nil, &block)
  command = make_simple_command(:post, 'v1/{+name}:stop', options)
  command.request_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1StopPipelineRequest::Representation
  command.request_object = google_cloud_datapipelines_v1_stop_pipeline_request_object
  command.response_representation = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline::Representation
  command.response_class = Google::Apis::DatapipelinesV1::GoogleCloudDatapipelinesV1Pipeline
  command.params['name'] = name unless name.nil?
  command.query['fields'] = fields unless fields.nil?
  command.query['quotaUser'] = quota_user unless quota_user.nil?
  execute_or_queue_command(command, &block)
end