On January 1, 2020 this library will no longer support Python 2 on the latest released version. Previously released library versions will continue to be available. For more information please visit Python 2 support on Google Cloud.

Page Iterators¶

Iterators for paging through paged API methods.

These iterators simplify the process of paging through API responses where the request takes a page token and the response is a list of results with a token for the next page. See list pagination in the Google API Style Guide for more details.

API clients that have methods that follow the list pagination pattern can return an Iterator. You can use this iterator to get all of the results across all pages:

>>> results_iterator = client.list_resources()
>>> list(results_iterator)  # Convert to a list (consumes all values).

Or you can walk your way through items and call off the search early if you find what you’re looking for (resulting in possibly fewer requests):

>>> for resource in results_iterator:
...     print(resource.name)
...     if not resource.is_valid:
...         break

At any point, you may check the number of items consumed by referencing the num_results property of the iterator:

>>> for my_item in results_iterator:
...     if results_iterator.num_results >= 10:
...         break

When iterating, not every new item will send a request to the server. To iterate based on each page of items (where a page corresponds to a request):

>>> for page in results_iterator.pages:
...     print('=' * 20)
...     print('    Page number: {:d}'.format(iterator.page_number))
...     print('  Items in page: {:d}'.format(page.num_items))
...     print('     First item: {!r}'.format(next(page)))
...     print('Items remaining: {:d}'.format(page.remaining))
...     print('Next page token: {}'.format(iterator.next_page_token))
====================
    Page number: 1
  Items in page: 1
     First item: <MyItemClass at 0x7f1d3cccf690>
Items remaining: 0
Next page token: eav1OzQB0OM8rLdGXOEsyQWSG
====================
    Page number: 2
  Items in page: 19
     First item: <MyItemClass at 0x7f1d3cccffd0>
Items remaining: 18
Next page token: None

Then, for each page you can get all the resources on that page by iterating through it or using list():

>>> list(page)
[
    <MyItemClass at 0x7fd64a098ad0>,
    <MyItemClass at 0x7fd64a098ed0>,
    <MyItemClass at 0x7fd64a098e90>,
]

class google.api_core.page_iterator.GRPCIterator(client, method, request, items_field, item_to_value=<function _item_to_value_identity>, request_token_field='page_token', response_token_field='next_page_token', max_results=None)[source]¶

Bases: google.api_core.page_iterator.Iterator

A generic class for iterating through gRPC list responses.

Note

The class does not take a page_token argument because it can just be specified in the request.

Parameters

client (google.cloud.client.Client) – The API client. This unused by this class, but kept to satisfy the Iterator interface.
method (Callable[protobuf.Message]) – A bound gRPC method that should take a single message for the request.
request (protobuf.Message) – The request message.
items_field (str) – The field in the response message that has the items for the page.
item_to_value (Callable[GRPCIterator, Any]) – Callable to convert an item from the type in the JSON response into a native object. Will be called with the iterator and a single item.
request_token_field (str) – The field in the request message used to specify the page token.
response_token_field (str) – The field in the response message that has the token for the next page.
max_results (int) – The maximum number of results to fetch.

pages¶

Iterator of pages in the response.

Returns

A: generator of page instances.

Return type

types.GeneratorType[google.api_core.page_iterator.Page]

Raises

ValueError – If the iterator has already been started.

class google.api_core.page_iterator.HTTPIterator(client, api_request, path, item_to_value, items_key='items', page_token=None, max_results=None, extra_params=None, page_start=<function _do_nothing_page_start>, next_token='nextPageToken')[source]¶

Bases: google.api_core.page_iterator.Iterator

A generic class for iterating through HTTP/JSON API list responses.

To make an iterator work, you’ll need to provide a way to convert a JSON item returned from the API into the object of your choice (via item_to_value). You also may need to specify a custom items_key so that a given response (containing a page of results) can be parsed into an iterable page of the actual objects you want.

Parameters

client (google.cloud.client.Client) – The API client.
api_request (Callable) – The function to use to make API requests. Generally, this will be google.cloud._http.JSONConnection.api_request().
path (str) – The method path to query for the list of items.
item_to_value (Callable[google.api_core.page_iterator.Iterator, Any]) – Callable to convert an item from the type in the JSON response into a native object. Will be called with the iterator and a single item.
items_key (str) – The key in the API response where the list of items can be found.
page_token (str) – A token identifying a page in a result set to start fetching results from.
max_results (int) – The maximum number of results to fetch.
extra_params (dict) – Extra query string parameters for the API call.
(Callable[ (page_start) – google.api_core.page_iterator.Iterator, google.api_core.page_iterator.Page, dict]): Callable to provide any special behavior after a new page has been created. Assumed signature takes the Iterator that started the page, the Page that was started and the dictionary containing the page response.
next_token (str) – The name of the field used in the response for page tokens.

pages¶

Iterator of pages in the response.

Returns

A: generator of page instances.

Return type

types.GeneratorType[google.api_core.page_iterator.Page]

Raises

ValueError – If the iterator has already been started.

class google.api_core.page_iterator.Iterator(client, item_to_value=<function _item_to_value_identity>, page_token=None, max_results=None)[source]¶

Bases: object

A generic class for iterating through API list responses.

Parameters

client (google.cloud.client.Client) – The API client.
item_to_value (Callable[google.api_core.page_iterator.Iterator, Any]) – Callable to convert an item from the type in the raw API response into the native object. Will be called with the iterator and a single item.
page_token (str) – A token identifying a page in a result set to start fetching results from.
max_results (int) – The maximum number of results to fetch.

__iter__()[source]¶

Iterator for each item returned.

Returns: A generator of items from the API.
Return type: types.GeneratorType[Any]
Raises: ValueError – If the iterator has already been started.

client = None¶

The client that created this iterator.

Type: Optional[Any]

item_to_value = None¶

Callable to convert an item from the type in the raw API response into the native object. Will be called with the iterator and a single item.

Type: Callable[Iterator, Any]

max_results = None¶

The maximum number of results to fetch.

Type: int

next_page_token = None¶

The token for the next page of results. If this is set before the iterator starts, it effectively offsets the iterator to a specific starting point.

Type: str

num_results = None¶

The total number of results fetched so far.

Type: int

page_number = None¶

The current page of results.

Type: int

property pages¶

Iterator of pages in the response.

Returns

A: generator of page instances.

Return type

types.GeneratorType[google.api_core.page_iterator.Page]

Raises

ValueError – If the iterator has already been started.

class google.api_core.page_iterator.Page(parent, items, item_to_value, raw_page=None)[source]¶

Bases: object

Single page of results in an iterator.

Parameters

parent (google.api_core.page_iterator.Iterator) – The iterator that owns the current page.
items (Sequence[Any]) – An iterable (that also defines __len__) of items from a raw API response.
item_to_value (Callable[google.api_core.page_iterator.Iterator, Any]) – Callable to convert an item from the type in the raw API response into the native object. Will be called with the iterator and a single item.
Optional[google.protobuf.message.Message] (raw_page) – The raw page response.

__iter__()[source]¶: The Page is an iterator of items.

__next__()¶: Get the next value in the page.

next()[source]¶: Get the next value in the page.

property num_items¶

Total items in the page.

Type: int

property raw_page¶: google.protobuf.message.Message

property remaining¶

Remaining items in the page.

Type: int

Page Iterators in AsyncIO¶

AsyncIO iterators for paging through paged API methods.

These iterators simplify the process of paging through API responses where the request takes a page token and the response is a list of results with a token for the next page. See list pagination in the Google API Style Guide for more details.

API clients that have methods that follow the list pagination pattern can return an AsyncIterator:

>>> results_iterator = await client.list_resources()

Or you can walk your way through items and call off the search early if you find what you’re looking for (resulting in possibly fewer requests):

>>> async for resource in results_iterator:
...     print(resource.name)
...     if not resource.is_valid:
...         break

At any point, you may check the number of items consumed by referencing the num_results property of the iterator:

>>> async for my_item in results_iterator:
...     if results_iterator.num_results >= 10:
...         break

When iterating, not every new item will send a request to the server. To iterate based on each page of items (where a page corresponds to a request):

>>> async for page in results_iterator.pages:
...     print('=' * 20)
...     print('    Page number: {:d}'.format(iterator.page_number))
...     print('  Items in page: {:d}'.format(page.num_items))
...     print('     First item: {!r}'.format(next(page)))
...     print('Items remaining: {:d}'.format(page.remaining))
...     print('Next page token: {}'.format(iterator.next_page_token))
====================
    Page number: 1
  Items in page: 1
     First item: <MyItemClass at 0x7f1d3cccf690>
Items remaining: 0
Next page token: eav1OzQB0OM8rLdGXOEsyQWSG
====================
    Page number: 2
  Items in page: 19
     First item: <MyItemClass at 0x7f1d3cccffd0>
Items remaining: 18
Next page token: None

class google.api_core.page_iterator_async.AsyncGRPCIterator(client, method, request, items_field, item_to_value=<function _item_to_value_identity>, request_token_field='page_token', response_token_field='next_page_token', max_results=None)[source]¶

Bases: google.api_core.page_iterator_async.AsyncIterator

A generic class for iterating through gRPC list responses.

Note

The class does not take a page_token argument because it can just be specified in the request.

Parameters

client (google.cloud.client.Client) – The API client. This unused by this class, but kept to satisfy the Iterator interface.
method (Callable[protobuf.Message]) – A bound gRPC method that should take a single message for the request.
request (protobuf.Message) – The request message.
items_field (str) – The field in the response message that has the items for the page.
item_to_value (Callable[GRPCIterator, Any]) – Callable to convert an item from the type in the JSON response into a native object. Will be called with the iterator and a single item.
request_token_field (str) – The field in the request message used to specify the page token.
response_token_field (str) – The field in the response message that has the token for the next page.
max_results (int) – The maximum number of results to fetch.

pages¶

Iterator of pages in the response.

Returns

A: generator of page instances.

Return type

types.GeneratorType[google.api_core.page_iterator.Page]

Raises

ValueError – If the iterator has already been started.

class google.api_core.page_iterator_async.AsyncIterator(client, item_to_value=<function _item_to_value_identity>, page_token=None, max_results=None)[source]¶

Bases: abc.ABC

A generic class for iterating through API list responses.

Parameters

client (google.cloud.client.Client) – The API client.
item_to_value (Callable[google.api_core.page_iterator_async.AsyncIterator, Any]) – Callable to convert an item from the type in the raw API response into the native object. Will be called with the iterator and a single item.
page_token (str) – A token identifying a page in a result set to start fetching results from.
max_results (int) – The maximum number of results to fetch.

__aiter__()[source]¶

Iterator for each item returned.

Returns: A generator of items from the API.
Return type: types.GeneratorType[Any]
Raises: ValueError – If the iterator has already been started.

client = None¶

The client that created this iterator.

Type: Optional[Any]

item_to_value = None¶

Callable to convert an item from the type in the raw API response into the native object. Will be called with the iterator and a single item.

Type: Callable[Iterator, Any]

max_results = None¶

The maximum number of results to fetch.

Type: int

next_page_token = None¶

The token for the next page of results. If this is set before the iterator starts, it effectively offsets the iterator to a specific starting point.

Type: str

num_results = None¶

The total number of results fetched so far.

Type: int

page_number = None¶

The current page of results.

Type: int

property pages¶

Iterator of pages in the response.

Returns

A: generator of page instances.

Return type

types.GeneratorType[google.api_core.page_iterator.Page]

Raises

ValueError – If the iterator has already been started.

Page Iterators¶

Page Iterators in AsyncIO¶

google-api-core

Navigation

Related Topics