Class: Google::Apis::DataflowV1b3::Source
- Inherits:
-
Object
- Object
- Google::Apis::DataflowV1b3::Source
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataflow_v1b3/classes.rb,
generated/google/apis/dataflow_v1b3/representations.rb,
generated/google/apis/dataflow_v1b3/representations.rb
Overview
A source that records can be read and decoded from.
Instance Attribute Summary collapse
-
#base_specs ⇒ Array<Hash<String,Object>>
While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs.
-
#codec ⇒ Hash<String,Object>
The codec to use to decode data read from the source.
-
#does_not_need_splitting ⇒ Boolean
(also: #does_not_need_splitting?)
Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
-
#metadata ⇒ Google::Apis::DataflowV1b3::SourceMetadata
Metadata about a Source useful for automatically optimizing and tuning the pipeline, etc.
-
#spec ⇒ Hash<String,Object>
The source to read from, plus its parameters.
Instance Method Summary collapse
-
#initialize(**args) ⇒ Source
constructor
A new instance of Source.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ Source
Returns a new instance of Source
3350 3351 3352 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3350 def initialize(**args) update!(**args) end |
Instance Attribute Details
#base_specs ⇒ Array<Hash<String,Object>>
While splitting, sources may specify the produced bundles
as differences against another source, in order to save backend-side
memory and allow bigger jobs. For details, see SourceSplitRequest.
To support this use case, the full set of parameters of the source
is logically obtained by taking the latest explicitly specified value
of each parameter in the order:
base_specs (later items win), spec (overrides anything in base_specs).
Corresponds to the JSON property baseSpecs
3313 3314 3315 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3313 def base_specs @base_specs end |
#codec ⇒ Hash<String,Object>
The codec to use to decode data read from the source.
Corresponds to the JSON property codec
3318 3319 3320 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3318 def codec @codec end |
#does_not_need_splitting ⇒ Boolean Also known as: does_not_need_splitting?
Setting this value to true hints to the framework that the source
doesn't need splitting, and using SourceSplitRequest on it would
yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
E.g. a file splitter may set this to true when splitting a single file
into a set of byte ranges of appropriate size, and set this
to false when splitting a filepattern into individual files.
However, for efficiency, a file splitter may decide to produce
file subranges directly from the filepattern to avoid a splitting
round-trip.
See SourceSplitRequest for an overview of the splitting process.
This field is meaningful only in the Source objects populated
by the user (e.g. when filling in a DerivedSource).
Source objects supplied by the framework to the user don't have
this field populated.
Corresponds to the JSON property doesNotNeedSplitting
3336 3337 3338 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3336 def does_not_need_splitting @does_not_need_splitting end |
#metadata ⇒ Google::Apis::DataflowV1b3::SourceMetadata
Metadata about a Source useful for automatically optimizing
and tuning the pipeline, etc.
Corresponds to the JSON property metadata
3343 3344 3345 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3343 def @metadata end |
#spec ⇒ Hash<String,Object>
The source to read from, plus its parameters.
Corresponds to the JSON property spec
3348 3349 3350 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3348 def spec @spec end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
3355 3356 3357 3358 3359 3360 3361 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3355 def update!(**args) @base_specs = args[:base_specs] if args.key?(:base_specs) @codec = args[:codec] if args.key?(:codec) @does_not_need_splitting = args[:does_not_need_splitting] if args.key?(:does_not_need_splitting) @metadata = args[:metadata] if args.key?(:metadata) @spec = args[:spec] if args.key?(:spec) end |