Class: Google::Apis::DataflowV1b3::Source
- Inherits:
-
Object
- Object
- Google::Apis::DataflowV1b3::Source
- Includes:
- Core::Hashable, Core::JsonObjectSupport
- Defined in:
- generated/google/apis/dataflow_v1b3/classes.rb,
generated/google/apis/dataflow_v1b3/representations.rb,
generated/google/apis/dataflow_v1b3/representations.rb
Overview
A source that records can be read and decoded from.
Instance Attribute Summary collapse
-
#base_specs ⇒ Array<Hash<String,Object>>
While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs.
-
#codec ⇒ Hash<String,Object>
The codec to use to decode data read from the source.
-
#does_not_need_splitting ⇒ Boolean
(also: #does_not_need_splitting?)
Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
-
#metadata ⇒ Google::Apis::DataflowV1b3::SourceMetadata
Metadata about a Source useful for automatically optimizing and tuning the pipeline, etc.
-
#spec ⇒ Hash<String,Object>
The source to read from, plus its parameters.
Instance Method Summary collapse
-
#initialize(**args) ⇒ Source
constructor
A new instance of Source.
-
#update!(**args) ⇒ Object
Update properties of this object.
Methods included from Core::JsonObjectSupport
Methods included from Core::Hashable
Constructor Details
#initialize(**args) ⇒ Source
Returns a new instance of Source
3838 3839 3840 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3838 def initialize(**args) update!(**args) end |
Instance Attribute Details
#base_specs ⇒ Array<Hash<String,Object>>
While splitting, sources may specify the produced bundles
as differences against another source, in order to save backend-side
memory and allow bigger jobs. For details, see SourceSplitRequest.
To support this use case, the full set of parameters of the source
is logically obtained by taking the latest explicitly specified value
of each parameter in the order:
base_specs (later items win), spec (overrides anything in base_specs).
Corresponds to the JSON property baseSpecs
3801 3802 3803 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3801 def base_specs @base_specs end |
#codec ⇒ Hash<String,Object>
The codec to use to decode data read from the source.
Corresponds to the JSON property codec
3806 3807 3808 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3806 def codec @codec end |
#does_not_need_splitting ⇒ Boolean Also known as: does_not_need_splitting?
Setting this value to true hints to the framework that the source
doesn't need splitting, and using SourceSplitRequest on it would
yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
E.g. a file splitter may set this to true when splitting a single file
into a set of byte ranges of appropriate size, and set this
to false when splitting a filepattern into individual files.
However, for efficiency, a file splitter may decide to produce
file subranges directly from the filepattern to avoid a splitting
round-trip.
See SourceSplitRequest for an overview of the splitting process.
This field is meaningful only in the Source objects populated
by the user (e.g. when filling in a DerivedSource).
Source objects supplied by the framework to the user don't have
this field populated.
Corresponds to the JSON property doesNotNeedSplitting
3824 3825 3826 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3824 def does_not_need_splitting @does_not_need_splitting end |
#metadata ⇒ Google::Apis::DataflowV1b3::SourceMetadata
Metadata about a Source useful for automatically optimizing
and tuning the pipeline, etc.
Corresponds to the JSON property metadata
3831 3832 3833 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3831 def @metadata end |
#spec ⇒ Hash<String,Object>
The source to read from, plus its parameters.
Corresponds to the JSON property spec
3836 3837 3838 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3836 def spec @spec end |
Instance Method Details
#update!(**args) ⇒ Object
Update properties of this object
3843 3844 3845 3846 3847 3848 3849 |
# File 'generated/google/apis/dataflow_v1b3/classes.rb', line 3843 def update!(**args) @base_specs = args[:base_specs] if args.key?(:base_specs) @codec = args[:codec] if args.key?(:codec) @does_not_need_splitting = args[:does_not_need_splitting] if args.key?(:does_not_need_splitting) @metadata = args[:metadata] if args.key?(:metadata) @spec = args[:spec] if args.key?(:spec) end |