Retry¶
Helpers for retrying functions with exponential back-off.
The Retry
decorator can be used to retry functions that raise
exceptions using exponential backoff. Because a exponential sleep algorithm is
used, the retry is limited by a deadline. The deadline is the maxmimum amount
of time a method can block. This is used instead of total number of retries
because it is difficult to ascertain the amount of time a function can block
when using total number of retries and exponential backoff.
By default, this decorator will retry transient
API errors (see if_transient_error()
). For example:
@retry.Retry()
def call_flaky_rpc():
return client.flaky_rpc()
# Will retry flaky_rpc() if it raises transient API errors.
result = call_flaky_rpc()
You can pass a custom predicate to retry on different exceptions, such as waiting for an eventually consistent item to be available:
@retry.Retry(predicate=if_exception_type(exceptions.NotFound))
def check_if_exists():
return client.does_thing_exist()
is_available = check_if_exists()
Some client library methods apply retry automatically. These methods can accept
a retry
parameter that allows you to configure the behavior:
my_retry = retry.Retry(deadline=60)
result = client.some_method(retry=my_retry)
- class google.api_core.retry.Retry(predicate=<function if_exception_type.<locals>.if_exception_type_predicate>, initial=1.0, maximum=60.0, multiplier=2.0, timeout=120.0, on_error=None, **kwargs)[source]¶
Bases:
object
Exponential retry decorator.
This class is a decorator used to add retry or polling behavior to an RPC call.
Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.
There two important concepts that retry/polling behavior may operate on, Deadline and Timeout, which need to be properly defined for the correct usage of this class and the rest of the library.
Deadline: a fixed point in time by which a certain operation must terminate. For example, if a certain operation has a deadline “2022-10-18T23:30:52.123Z” it must terminate (successfully or with an error) by that time, regardless of when it was started or whether it was started at all.
Timeout: the maximum duration of time after which a certain operation must terminate (successfully or with an error). The countdown begins right after an operation was started. For example, if an operation was started at 09:24:00 with timeout of 75 seconds, it must terminate no later than 09:25:15.
Unfortunately, in the past this class (and the api-core library as a whole) has not been properly distinguishing the concepts of “timeout” and “deadline”, and the
deadline
parameter has meanttimeout
. That is whydeadline
has been deprecated andtimeout
should be used instead. If thedeadline
parameter is set, it will override thetimeout
parameter. In other words,retry.deadline
should be treated as just a deprecated alias forretry.timeout
.Said another way, it is safe to assume that this class and the rest of this library operate in terms of timeouts (not deadlines) unless explicitly noted the usage of deadline semantics.
It is also important to understand the three most common applications of the Timeout concept in the context of this library.
Usually the generic Timeout term may stand for one of the following actual timeouts: RPC Timeout, Retry Timeout, or Polling Timeout.
RPC Timeout: a value supplied by the client to the server so that the server side knows the maximum amount of time it is expected to spend handling that specifc RPC. For example, in the case of gRPC transport, RPC Timeout is represented by setting “grpc-timeout” header in the HTTP2 request. The timeout property of this class normally never represents the RPC Timeout as it is handled separately by the
google.api_core.timeout
module of this library.Retry Timeout: this is the most common meaning of the
timeout
property of this class, and defines how long a certain RPC may be retried in case the server returns an error.Polling Timeout: defines how long the client side is allowed to call the polling RPC repeatedly to check a status of a long-running operation. Each polling RPC is expected to succeed (its errors are supposed to be handled by the retry logic). The decision as to whether a new polling attempt needs to be made is based not on the RPC status code but on the status of the returned status of an operation. In other words: we will poll a long-running operation until the operation is done or the polling timeout expires. Each poll will inform us of the status of the operation. The poll consists of an RPC to the server that may itself be retried as per the poll-specific retry settings in case of errors. The operation-level retry settings do NOT apply to polling-RPC retries.
With the actual timeout types being defined above, the client libraries often refer to just Timeout without clarifying which type specifically that is. In that case the actual timeout type (sometimes also refered to as Logical Timeout) can be determined from the context. If it is a unary rpc call (i.e. a regular one) Timeout usually stands for the RPC Timeout (if provided directly as a standalone value) or Retry Timeout (if provided as
retry.timeout
property of the unary RPC’s retry config). ForOperation
orPollingFuture
in general Timeout stands for Polling Timeout.- Parameters
predicate (Callable[Exception]) – A callable that should return
True
if the given exception is retryable.initial (float) – The minimum amount of time to delay in seconds. This must be greater than 0.
maximum (float) – The maximum amount of time to delay in seconds.
multiplier (float) – The multiplier applied to the delay.
timeout (float) – How long to keep retrying, in seconds.
deadline (float) – DEPRECATED: use timeout instead. For backward compatibility, if specified it will override the
timeout
parameter.
- __call__(func, on_error=None)[source]¶
Wrap a callable with retry behavior.
- Parameters
func (Callable) – The callable to add retry behavior to.
on_error (Callable[Exception]) – A function to call while processing a retryable exception. Any error raised by this function will not be caught.
- Returns
- A callable that will invoke
func
with retry behavior.
- A callable that will invoke
- Return type
Callable
- property deadline¶
use
timeout
instead. Refer to theRetry
class documentation for details.- Type
DEPRECATED
- with_deadline(deadline)[source]¶
Return a copy of this retry with the given timeout.
DEPRECATED: use
with_timeout()
instead. Refer to theRetry
class documentation for details.
- with_delay(initial=None, maximum=None, multiplier=None)[source]¶
Return a copy of this retry with the given delay options.
- google.api_core.retry.exponential_sleep_generator(initial, maximum, multiplier=2.0)[source]¶
Generates sleep intervals based on the exponential back-off algorithm.
This implements the Truncated Exponential Back-off algorithm.
- google.api_core.retry.if_exception_type(*exception_types)[source]¶
Creates a predicate to check if the exception is of a given type.
- Parameters
exception_types (Sequence[
type()
]) – The exception types to check for.- Returns
- A predicate that returns True if the provided
exception is of the given type(s).
- Return type
Callable[Exception]
- google.api_core.retry.if_transient_error(exception)¶
A predicate that checks if an exception is a transient API error.
The following server errors are considered transient:
google.api_core.exceptions.InternalServerError
- HTTP 500, gRPCINTERNAL(13)
and its subclasses.
google.api_core.exceptions.TooManyRequests
- HTTP 429requests.exceptions.ConnectionError
requests.exceptions.ChunkedEncodingError
- The server declaredchunked encoding but sent an invalid chunk.
google.auth.exceptions.TransportError
- Used to indicate anerror occurred during an HTTP request.
- google.api_core.retry.retry_target(target, predicate, sleep_generator, timeout=None, on_error=None, **kwargs)[source]¶
Call a function and retry if it fails.
This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper
Retry
.- Parameters
target (Callable) – The function to call and retry. This must be a nullary function - apply arguments with functools.partial.
predicate (Callable[Exception]) – A callable used to determine if an exception raised by the target should be considered retryable. It should return True to retry or False otherwise.
sleep_generator (Iterable[float]) – An infinite iterator that determines how long to sleep between retries.
timeout (float) – How long to keep retrying the target.
on_error (Callable[Exception]) – A function to call while processing a retryable exception. Any error raised by this function will not be caught.
deadline (float) – DEPRECATED: use
timeout
instead. For backward compatibility, if specified it will overridetimeout
parameter.
- Returns
the return value of the target function.
- Return type
Any
- Raises
google.api_core.RetryError – If the deadline is exceeded while retrying.
ValueError – If the sleep generator stops yielding values.
Exception – If the target raises a method that isn’t retryable.
Retry in AsyncIO¶
Helpers for retrying coroutine functions with exponential back-off.
The AsyncRetry
decorator shares most functionality and behavior with
Retry
, but supports coroutine functions. Please refer to description
of Retry
for more details.
By default, this decorator will retry transient
API errors (see if_transient_error()
). For example:
@retry_async.AsyncRetry()
async def call_flaky_rpc():
return await client.flaky_rpc()
# Will retry flaky_rpc() if it raises transient API errors.
result = await call_flaky_rpc()
You can pass a custom predicate to retry on different exceptions, such as waiting for an eventually consistent item to be available:
@retry_async.AsyncRetry(predicate=retry_async.if_exception_type(exceptions.NotFound))
async def check_if_exists():
return await client.does_thing_exist()
is_available = await check_if_exists()
Some client library methods apply retry automatically. These methods can accept
a retry
parameter that allows you to configure the behavior:
my_retry = retry_async.AsyncRetry(deadline=60)
result = await client.some_method(retry=my_retry)
- class google.api_core.retry_async.AsyncRetry(predicate=<function if_exception_type.<locals>.if_exception_type_predicate>, initial=1.0, maximum=60.0, multiplier=2.0, timeout=120.0, on_error=None, **kwargs)[source]¶
Bases:
object
Exponential retry decorator for async functions.
This class is a decorator used to add exponential back-off retry behavior to an RPC call.
Although the default behavior is to retry transient API errors, a different predicate can be provided to retry other exceptions.
- Parameters
predicate (Callable[Exception]) – A callable that should return
True
if the given exception is retryable.initial (float) – The minimum a,out of time to delay in seconds. This must be greater than 0.
maximum (float) – The maximum amout of time to delay in seconds.
multiplier (float) – The multiplier applied to the delay.
timeout (float) – How long to keep retrying in seconds.
on_error (Callable[Exception]) – A function to call while processing a retryable exception. Any error raised by this function will not be caught.
deadline (float) – DEPRECATED use
timeout
instead. If set it willparameter. (override timeout) –
- __call__(func, on_error=None)[source]¶
Wrap a callable with retry behavior.
- Parameters
func (Callable) – The callable to add retry behavior to.
on_error (Callable[Exception]) – A function to call while processing a retryable exception. Any error raised by this function will not be caught.
- Returns
- A callable that will invoke
func
with retry behavior.
- A callable that will invoke
- Return type
Callable
- with_deadline(deadline)[source]¶
Return a copy of this retry with the given deadline. DEPRECATED: use
with_timeout()
instead.- Parameters
deadline (float) – How long to keep retrying.
- Returns
A new retry instance with the given deadline.
- Return type
- with_delay(initial=None, maximum=None, multiplier=None)[source]¶
Return a copy of this retry with the given delay options.
- Parameters
- Returns
A new retry instance with the given predicate.
- Return type
- with_predicate(predicate)[source]¶
Return a copy of this retry with the given predicate.
- Parameters
predicate (Callable[Exception]) – A callable that should return
True
if the given exception is retryable.- Returns
A new retry instance with the given predicate.
- Return type
- async google.api_core.retry_async.retry_target(target, predicate, sleep_generator, timeout=None, on_error=None, **kwargs)[source]¶
Call a function and retry if it fails.
This is the lowest-level retry helper. Generally, you’ll use the higher-level retry helper
Retry
.- Parameters
target (Callable) – The function to call and retry. This must be a nullary function - apply arguments with functools.partial.
predicate (Callable[Exception]) – A callable used to determine if an exception raised by the target should be considered retryable. It should return True to retry or False otherwise.
sleep_generator (Iterable[float]) – An infinite iterator that determines how long to sleep between retries.
timeout (float) – How long to keep retrying the target, in seconds.
on_error (Callable[Exception]) – A function to call while processing a retryable exception. Any error raised by this function will not be caught.
deadline (float) – DEPRECATED use
timeout
instead. For backwardcompatibility –
parameter. (if set it will override the timeout) –
- Returns
the return value of the target function.
- Return type
Any
- Raises
google.api_core.RetryError – If the deadline is exceeded while retrying.
ValueError – If the sleep generator stops yielding values.
Exception – If the target raises a method that isn’t retryable.