Class: Google::Apis::DataflowV1b3::Source
- Inherits:
-
Object
- Object
- Google::Apis::DataflowV1b3::Source
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataflow_v1b3/classes.rb,
generated/google/apis/dataflow_v1b3/representations.rb,
generated/google/apis/dataflow_v1b3/representations.rb
Overview
A source that records can be read and decoded from.
Instance Attribute Summary collapse
-
#base_specs ⇒ Array<Hash<String,Object>>
While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs.
-
#codec ⇒ Hash<String,Object>
The codec to use to decode data read from the source.
-
#does_not_need_splitting ⇒ Boolean
(also: #does_not_need_splitting?)
Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
-
#metadata ⇒ Google::Apis::DataflowV1b3::SourceMetadata
Metadata about a Source useful for automatically optimizing and tuning the pipeline, etc.
-
#spec ⇒ Hash<String,Object>
The source to read from, plus its parameters.
Instance Method Summary collapse
-
#initialize(**args) ⇒ Source
constructor
A new instance of Source.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ Source
Returns a new instance of Source
3596 3597 3598 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3596 def initialize(**args) update!(**args) end |
Instance Attribute Details
#base_specs ⇒ Array<Hash<String,Object>>
While splitting, sources may specify the produced bundles
as differences against another source, in order to save backend-side
memory and allow bigger jobs. For details, see SourceSplitRequest.
To support this use case, the full set of parameters of the source
is logically obtained by taking the latest explicitly specified value
of each parameter in the order:
base_specs (later items win), spec (overrides anything in base_specs).
Corresponds to the JSON property baseSpecs
3559 3560 3561 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3559 def base_specs @base_specs end |
#codec ⇒ Hash<String,Object>
The codec to use to decode data read from the source.
Corresponds to the JSON property codec
3564 3565 3566 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3564 def codec @codec end |
#does_not_need_splitting ⇒ Boolean Also known as: does_not_need_splitting?
Setting this value to true hints to the framework that the source
doesn't need splitting, and using SourceSplitRequest on it would
yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
E.g. a file splitter may set this to true when splitting a single file
into a set of byte ranges of appropriate size, and set this
to false when splitting a filepattern into individual files.
However, for efficiency, a file splitter may decide to produce
file subranges directly from the filepattern to avoid a splitting
round-trip.
See SourceSplitRequest for an overview of the splitting process.
This field is meaningful only in the Source objects populated
by the user (e.g. when filling in a DerivedSource).
Source objects supplied by the framework to the user don't have
this field populated.
Corresponds to the JSON property doesNotNeedSplitting
3582 3583 3584 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3582 def does_not_need_splitting @does_not_need_splitting end |
#metadata ⇒ Google::Apis::DataflowV1b3::SourceMetadata
Metadata about a Source useful for automatically optimizing
and tuning the pipeline, etc.
Corresponds to the JSON property metadata
3589 3590 3591 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3589 def @metadata end |
#spec ⇒ Hash<String,Object>
The source to read from, plus its parameters.
Corresponds to the JSON property spec
3594 3595 3596 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3594 def spec @spec end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
3601 3602 3603 3604 3605 3606 3607 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3601 def update!(**args) @base_specs = args[:base_specs] if args.key?(:base_specs) @codec = args[:codec] if args.key?(:codec) @does_not_need_splitting = args[:does_not_need_splitting] if args.key?(:does_not_need_splitting) @metadata = args[:metadata] if args.key?(:metadata) @spec = args[:spec] if args.key?(:spec) end |