As of January 1, 2020 this library no longer supports Python 2 on the latest released version. Library versions released prior to that date will continue to be available. For more information please visit Python 2 support on Google Cloud.

Requests Utilities

requests utilities for Google Media Downloads and Resumable Uploads.

This sub-package assumes callers will use the requests library as transport and google-auth for sending authenticated HTTP traffic with requests.

Authorized Transport

To use google-auth and requests to create an authorized transport that has read-only access to Google Cloud Storage (GCS):

>>> import google.auth
>>> import google.auth.transport.requests as tr_requests
>>>
>>> ro_scope = 'https://www.googleapis.com/auth/devstorage.read_only'
>>> credentials, _ = google.auth.default(scopes=(ro_scope,))
>>> transport = tr_requests.AuthorizedSession(credentials)
>>> transport
<google.auth.transport.requests.AuthorizedSession object at 0x...>

Simple Downloads

To download an object from Google Cloud Storage, construct the media URL for the GCS object and download it with an authorized transport that has access to the resource:

>>> from google.resumable_media.requests import Download
>>>
>>> url_template = (
...     'https://www.googleapis.com/download/storage/v1/b/'
...     '{bucket}/o/{blob_name}?alt=media')
>>> media_url = url_template.format(
...     bucket=bucket, blob_name=blob_name)
>>>
>>> download = Download(media_url)
>>> response = download.consume(transport)
>>> download.finished
True
>>> response
<Response [200]>
>>> response.headers['Content-Length']
'1364156'
>>> len(response.content)
1364156

To download only a portion of the bytes in the object, specify start and end byte positions (both optional):

>>> download = Download(media_url, start=4096, end=8191)
>>> response = download.consume(transport)
>>> download.finished
True
>>> response
<Response [206]>
>>> response.headers['Content-Length']
'4096'
>>> response.headers['Content-Range']
'bytes 4096-8191/1364156'
>>> len(response.content)
4096

Chunked Downloads

For very large objects or objects of unknown size, it may make more sense to download the object in chunks rather than all at once. This can be done to avoid dropped connections with a poor internet connection or can allow multiple chunks to be downloaded in parallel to speed up the total download.

A ChunkedDownload uses the same media URL and authorized transport that a basic Download would use, but also requires a chunk size and a write-able byte stream. The chunk size is used to determine how much of the resouce to consume with each request and the stream is to allow the resource to be written out (e.g. to disk) without having to fit in memory all at once.

>>> from google.resumable_media.requests import ChunkedDownload
>>>
>>> chunk_size = 50 * 1024 * 1024  # 50MB
>>> stream = io.BytesIO()
>>> download = ChunkedDownload(
...     media_url, chunk_size, stream)
>>> # Check the state of the download before starting.
>>> download.bytes_downloaded
0
>>> download.total_bytes is None
True
>>> response = download.consume_next_chunk(transport)
>>> # Check the state of the download after consuming one chunk.
>>> download.finished
False
>>> download.bytes_downloaded  # chunk_size
52428800
>>> download.total_bytes  # 1GB
1073741824
>>> response
<Response [206]>
>>> response.headers['Content-Length']
'52428800'
>>> response.headers['Content-Range']
'bytes 0-52428799/1073741824'
>>> len(response.content) == chunk_size
True
>>> stream.seek(0)
0
>>> stream.read(29)
b'The beginning of the chunk...'

The download will change it’s finished status to True once the final chunk is consumed. In some cases, the final chunk may not be the same size as the other chunks:

>>> # The state of the download in progress.
>>> download.finished
False
>>> download.bytes_downloaded  # 20 chunks at 50MB
1048576000
>>> download.total_bytes  # 1GB
1073741824
>>> response = download.consume_next_chunk(transport)
>>> # The state of the download after consuming the final chunk.
>>> download.finished
True
>>> download.bytes_downloaded == download.total_bytes
True
>>> response
<Response [206]>
>>> response.headers['Content-Length']
'25165824'
>>> response.headers['Content-Range']
'bytes 1048576000-1073741823/1073741824'
>>> len(response.content) < download.chunk_size
True

In addition, a ChunkedDownload can also take optional start and end byte positions.

Usually, no checksum is returned with a chunked download. Even if one is returned, it is not validated. If you need to validate the checksum, you can do so by buffering the chunks and validating the checksum against the completed download.

Simple Uploads

Among the three supported upload classes, the simplest is SimpleUpload. A simple upload should be used when the resource being uploaded is small and when there is no metadata (other than the name) associated with the resource.

>>> from google.resumable_media.requests import SimpleUpload
>>>
>>> url_template = (
...     'https://www.googleapis.com/upload/storage/v1/b/{bucket}/o?'
...     'uploadType=media&'
...     'name={blob_name}')
>>> upload_url = url_template.format(
...     bucket=bucket, blob_name=blob_name)
>>>
>>> upload = SimpleUpload(upload_url)
>>> data = b'Some not too large content.'
>>> content_type = 'text/plain'
>>> response = upload.transmit(transport, data, content_type)
>>> upload.finished
True
>>> response
<Response [200]>
>>> json_response = response.json()
>>> json_response['bucket'] == bucket
True
>>> json_response['name'] == blob_name
True
>>> json_response['contentType'] == content_type
True
>>> json_response['md5Hash']
'M0XLEsX9/sMdiI+4pB4CAQ=='
>>> int(json_response['size']) == len(data)
True

In the rare case that an upload fails, an InvalidResponse will be raised:

>>> upload = SimpleUpload(upload_url)
>>> error = None
>>> try:
...     upload.transmit(transport, data, content_type)
... except resumable_media.InvalidResponse as caught_exc:
...     error = caught_exc
...
>>> error
InvalidResponse('Request failed with status code', 503,
                'Expected one of', <HTTPStatus.OK: 200>)
>>> error.response
<Response [503]>
>>>
>>> upload.finished
True

Even in the case of failure, we see that the upload is finished, i.e. it cannot be re-used.

Multipart Uploads

After the simple upload, the MultipartUpload can be used to achieve essentially the same task. However, a multipart upload allows some metadata about the resource to be sent along as well. (This is the “multi”: we send a first part with the metadata and a second part with the actual bytes in the resource.)

Usage is similar to the simple upload, but transmit() accepts an extra required argument: metadata.

>>> from google.resumable_media.requests import MultipartUpload
>>>
>>> url_template = (
...     'https://www.googleapis.com/upload/storage/v1/b/{bucket}/o?'
...     'uploadType=multipart')
>>> upload_url = url_template.format(bucket=bucket)
>>>
>>> upload = MultipartUpload(upload_url)
>>> metadata = {
...     'name': blob_name,
...     'metadata': {
...         'color': 'grurple',
...     },
... }
>>> response = upload.transmit(transport, data, metadata, content_type)
>>> upload.finished
True
>>> response
<Response [200]>
>>> json_response = response.json()
>>> json_response['bucket'] == bucket
True
>>> json_response['name'] == blob_name
True
>>> json_response['metadata'] == metadata['metadata']
True

As with the simple upload, in the case of failure an InvalidResponse is raised, enclosing the response that caused the failure and the upload object cannot be re-used after a failure.

Resumable Uploads

A ResumableUpload deviates from the other two upload classes: it transmits a resource over the course of multiple requests. This is intended to be used in cases where:

• the size of the resource is not known (i.e. it is generated on the fly)

• requests must be short-lived

• the client has request size limitations

• the resource is too large to fit into memory

In general, a resource should be sent in a single request to avoid latency and reduce QPS. See GCS best practices for more things to consider when using a resumable upload.

After creating a ResumableUpload instance, a resumable upload session must be initiated to let the server know that a series of chunked upload requests will be coming and to obtain an upload_id for the session. In contrast to the other two upload classes, initiate() takes a byte stream as input rather than raw bytes as data. This can be a file object, a BytesIO object or any other stream implementing the same interface.

>>> from google.resumable_media.requests import ResumableUpload
>>>
>>> url_template = (
...     'https://www.googleapis.com/upload/storage/v1/b/{bucket}/o?'
...     'uploadType=resumable')
>>> upload_url = url_template.format(bucket=bucket)
>>>
>>> chunk_size = 1024 * 1024  # 1MB
>>> upload = ResumableUpload(upload_url, chunk_size)
>>> stream = io.BytesIO(data)
>>> # The upload doesn't know how "big" it is until seeing a stream.
>>> upload.total_bytes is None
True
>>> metadata = {'name': blob_name}
>>> response = upload.initiate(transport, stream, metadata, content_type)
>>> response
<Response [200]>
>>> upload.resumable_url == response.headers['Location']
True
>>> upload.total_bytes == len(data)
True
>>> upload_id = response.headers['X-GUploader-UploadID']
>>> upload_id
'ABCdef189XY_super_serious'
>>> upload.resumable_url == upload_url + '&upload_id=' + upload_id
True

Once a ResumableUpload has been initiated, the resource is transmitted in chunks until completion:

>>> response0 = upload.transmit_next_chunk(transport)
>>> response0
<Response [308]>
>>> upload.finished
False
>>> upload.bytes_uploaded == upload.chunk_size
True
>>>
>>> response1 = upload.transmit_next_chunk(transport)
>>> response1
<Response [308]>
>>> upload.finished
False
>>> upload.bytes_uploaded == 2 * upload.chunk_size
True
>>>
>>> response2 = upload.transmit_next_chunk(transport)
>>> response2
<Response [200]>
>>> upload.finished
True
>>> upload.bytes_uploaded == upload.total_bytes
True
>>> json_response = response2.json()
>>> json_response['bucket'] == bucket
True
>>> json_response['name'] == blob_name
True

class google.resumable_media.requests.ChunkedDownload(media_url, chunk_size, stream, start=0, end=None, headers=None)

Download a resource in chunks from a Google API.

Parameters

• media_url (str) – The URL containing the media to be downloaded.

• chunk_size (int) – The number of bytes to be retrieved in each request.

• stream (IO[bytes]) – A write-able stream (i.e. file-like object) that will be used to concatenate chunks of the resource as they are downloaded.

• start (int) – The first byte in a range to be downloaded. If not provided, defaults to 0.

• end (int) – The last byte in a range to be downloaded. If not provided, will download to the end of the media.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with each request, e.g. headers for data encryption key headers.

media_url

The URL containing the media to be downloaded.

Type

str

start

The first byte in a range to be downloaded.

Type

Optional[int]

end

The last byte in a range to be downloaded.

Type

Optional[int]

chunk_size

The number of bytes to be retrieved in each request.

Type

int

Raises

ValueError – If start is negative.

property bytes_downloaded

Number of bytes that have been downloaded.

Type

int

consume_next_chunk(transport, timeout=(61, 60))

Consume the next chunk of the resource to be downloaded.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

Raises

ValueError – If the current download has finished.

property finished

Flag indicating if the download has completed.

Type

bool

property invalid

Indicates if the download is in an invalid state.

This will occur if a call to consume_next_chunk() fails.

Type

bool

property total_bytes

The total number of bytes to be downloaded.

Type

Optional[int]

class google.resumable_media.requests.Download(media_url, stream=None, start=None, end=None, headers=None, checksum='md5')

Helper to manage downloading a resource from a Google API.

“Slices” of the resource can be retrieved by specifying a range with start and / or end. However, in typical usage, neither start nor end is expected to be provided.

Parameters

• media_url (str) – The URL containing the media to be downloaded.

• stream (IO[bytes]) – A write-able stream (i.e. file-like object) that the downloaded resource can be written to.

• start (int) – The first byte in a range to be downloaded. If not provided, but end is provided, will download from the beginning to end of the media.

• end (int) – The last byte in a range to be downloaded. If not provided, but start is provided, will download from the start to the end of the media.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with the request, e.g. headers for encrypted data.

• Optional (checksum) – The type of checksum to compute to verify the integrity of the object. The response headers must contain a checksum of the requested type. If the headers lack an appropriate checksum (for instance in the case of transcoded or ranged downloads where the remote service does not know the correct checksum) an INFO-level log will be emitted. Supported values are “md5”, “crc32c” and None. The default is “md5”.

media_url

The URL containing the media to be downloaded.

Type

str

start

The first byte in a range to be downloaded.

Type

Optional[int]

end

The last byte in a range to be downloaded.

Type

Optional[int]

consume(transport, timeout=(61, 60))

Consume the resource to be downloaded.

If a stream is attached to this download, then the downloaded resource will be written to the stream.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

Raises

• DataCorruption – If the download’s checksum doesn’t agree with server-computed checksum.

• ValueError – If the current Download has already finished.

property finished

Flag indicating if the download has completed.

Type

bool

class google.resumable_media.requests.MultipartUpload(upload_url, headers=None, checksum=None)

Upload a resource with metadata to a Google API.

A multipart upload sends both metadata and the resource in a single (multipart) request.

Parameters

• upload_url (str) – The URL where the content will be uploaded.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with the request, e.g. headers for encrypted data.

• Optional (checksum) – The type of checksum to compute to verify the integrity of the object. The request metadata will be amended to include the computed value. Using this option will override a manually-set checksum value. Supported values are “md5”, “crc32c” and None. The default is None.

upload_url

The URL where the content will be uploaded.

Type

str

property finished

Flag indicating if the upload has completed.

Type

bool

transmit(transport, data, metadata, content_type, timeout=(61, 60))

Transmit the resource to be uploaded.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• data (bytes) – The resource content to be uploaded.

• metadata (Mapping[str, str]) – The resource metadata, such as an ACL list.

• content_type (str) – The content type of the resource, e.g. a JPEG image has content type image/jpeg.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

class google.resumable_media.requests.RawChunkedDownload(media_url, chunk_size, stream, start=0, end=None, headers=None)

Download a raw resource in chunks from a Google API.

Parameters

• media_url (str) – The URL containing the media to be downloaded.

• chunk_size (int) – The number of bytes to be retrieved in each request.

• stream (IO[bytes]) – A write-able stream (i.e. file-like object) that will be used to concatenate chunks of the resource as they are downloaded.

• start (int) – The first byte in a range to be downloaded. If not provided, defaults to 0.

• end (int) – The last byte in a range to be downloaded. If not provided, will download to the end of the media.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with each request, e.g. headers for data encryption key headers.

media_url

The URL containing the media to be downloaded.

Type

str

start

The first byte in a range to be downloaded.

Type

Optional[int]

end

The last byte in a range to be downloaded.

Type

Optional[int]

chunk_size

The number of bytes to be retrieved in each request.

Type

int

Raises

ValueError – If start is negative.

property bytes_downloaded

Number of bytes that have been downloaded.

Type

int

consume_next_chunk(transport, timeout=(61, 60))

Consume the next chunk of the resource to be downloaded.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

Raises

ValueError – If the current download has finished.

property finished

Flag indicating if the download has completed.

Type

bool

property invalid

Indicates if the download is in an invalid state.

This will occur if a call to consume_next_chunk() fails.

Type

bool

property total_bytes

The total number of bytes to be downloaded.

Type

Optional[int]

class google.resumable_media.requests.RawDownload(media_url, stream=None, start=None, end=None, headers=None, checksum='md5')

Helper to manage downloading a raw resource from a Google API.

“Slices” of the resource can be retrieved by specifying a range with start and / or end. However, in typical usage, neither start nor end is expected to be provided.

Parameters

• media_url (str) – The URL containing the media to be downloaded.

• stream (IO[bytes]) – A write-able stream (i.e. file-like object) that the downloaded resource can be written to.

• start (int) – The first byte in a range to be downloaded. If not provided, but end is provided, will download from the beginning to end of the media.

• end (int) – The last byte in a range to be downloaded. If not provided, but start is provided, will download from the start to the end of the media.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with the request, e.g. headers for encrypted data.

• Optional (checksum) – The type of checksum to compute to verify the integrity of the object. The response headers must contain a checksum of the requested type. If the headers lack an appropriate checksum (for instance in the case of transcoded or ranged downloads where the remote service does not know the correct checksum) an INFO-level log will be emitted. Supported values are “md5”, “crc32c” and None. The default is “md5”.

media_url

The URL containing the media to be downloaded.

Type

str

start

The first byte in a range to be downloaded.

Type

Optional[int]

end

The last byte in a range to be downloaded.

Type

Optional[int]

consume(transport, timeout=(61, 60))

Consume the resource to be downloaded.

If a stream is attached to this download, then the downloaded resource will be written to the stream.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

Raises

• DataCorruption – If the download’s checksum doesn’t agree with server-computed checksum.

• ValueError – If the current Download has already finished.

property finished

Flag indicating if the download has completed.

Type

bool

class google.resumable_media.requests.ResumableUpload(upload_url, chunk_size, checksum=None, headers=None)

Initiate and fulfill a resumable upload to a Google API.

A resumable upload sends an initial request with the resource metadata and then gets assigned an upload ID / upload URL to send bytes to. Using the upload URL, the upload is then done in chunks (determined by the user) until all bytes have been uploaded.

When constructing a resumable upload, only the resumable upload URL and the chunk size are required:

>>> from google.resumable_media.requests import ResumableUpload
>>>
>>> url_template = (
...     'https://www.googleapis.com/upload/storage/v1/b/{bucket}/o?'
...     'uploadType=resumable')
>>> upload_url = url_template.format(bucket=bucket)
>>>
>>> chunk_size = 3 * 1024 * 1024  # 3MB
>>> upload = ResumableUpload(upload_url, chunk_size)

When initiating an upload (via initiate()), the caller is expected to pass the resource being uploaded as a file-like stream. If the size of the resource is explicitly known, it can be passed in directly:

>>> import os
>>>
>>> upload.total_bytes is None
True
>>>
>>> stream = open(filename, 'rb')
>>> total_bytes = os.path.getsize(filename)
>>> metadata = {'name': filename}
>>> response = upload.initiate(
...     transport, stream, metadata, 'text/plain',
...     total_bytes=total_bytes)
>>> response
<Response [200]>
>>>
>>> upload.total_bytes == total_bytes
True

If the stream is in a “final” state (i.e. it won’t have any more bytes written to it), the total number of bytes can be determined implicitly from the stream itself:

>>> stream = io.BytesIO(data)
>>> response = upload.initiate(
...     transport, stream, metadata, content_type)
>>>
>>> upload.total_bytes == len(data)
True

If the size of the resource is unknown when the upload is initiated, the stream_final argument can be used. This might occur if the resource is being dynamically created on the client (e.g. application logs). To use this argument:

>>> response = upload.initiate(
...     transport, stream, metadata, content_type,
...     stream_final=False)
>>>
>>> upload.total_bytes is None
True

Parameters

• upload_url (str) – The URL where the resumable upload will be initiated.

• chunk_size (int) – The size of each chunk used to upload the resource.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with the initiate() request, e.g. headers for encrypted data. These will not be sent with transmit_next_chunk() or recover() requests.

• Optional (checksum) – The type of checksum to compute to verify the integrity of the object. After the upload is complete, the server-computed checksum of the resulting object will be checked and google.resumable_media.common.DataCorruption will be raised on a mismatch. The corrupted file will not be deleted from the remote host automatically. Supported values are “md5”, “crc32c” and None. The default is None.

upload_url

The URL where the content will be uploaded.

Type

str

Raises

ValueError – If chunk_size is not a multiple of UPLOAD_CHUNK_SIZE.

property bytes_uploaded

Number of bytes that have been uploaded.

Type

int

property chunk_size

The size of each chunk used to upload the resource.

Type

int

property finished

Flag indicating if the upload has completed.

Type

bool

initiate(transport, stream, metadata, content_type, total_bytes=None, stream_final=True, timeout=(61, 60))

Initiate a resumable upload.

By default, this method assumes your stream is in a “final” state ready to transmit. However, stream_final=False can be used to indicate that the size of the resource is not known. This can happen if bytes are being dynamically fed into stream, e.g. if the stream is attached to application logs.

If stream_final=False is used, chunk_size bytes will be read from the stream every time transmit_next_chunk() is called. If one of those reads produces strictly fewer bites than the chunk size, the upload will be concluded.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• stream (IO[bytes]) – The stream (i.e. file-like object) that will be uploaded. The stream must be at the beginning (i.e. stream.tell() == 0).

• metadata (Mapping[str, str]) – The resource metadata, such as an ACL list.

• content_type (str) – The content type of the resource, e.g. a JPEG image has content type image/jpeg.

• total_bytes (Optional[int]) – The total number of bytes to be uploaded. If specified, the upload size will not be determined from the stream (even if stream_final=True).

• stream_final (Optional[bool]) – Indicates if the stream is “final” (i.e. no more bytes will be added to it). In this case we determine the upload size from the size of the stream. If total_bytes is passed, this argument will be ignored.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

property invalid

Indicates if the upload is in an invalid state.

This will occur if a call to transmit_next_chunk() fails. To recover from such a failure, call recover().

Type

bool

recover(transport)

Recover from a failure and check the status of the current upload.

This will verify the progress with the server and make sure the current upload is in a valid state before transmit_next_chunk() can be used again. See https://cloud.google.com/storage/docs/performing-resumable-uploads#status-check for more information.

This method can be used when a ResumableUpload is in an invalid state due to a request failure.

Parameters

transport (Session) – A requests object which can make authenticated requests.

Returns

The HTTP response returned by transport.

Return type

Response

property resumable_url

The URL of the in-progress resumable upload.

Type

Optional[str]

property total_bytes

The total number of bytes to be uploaded.

If this upload is initiated (via initiate()) with stream_final=True, this value will be populated based on the size of the stream being uploaded. (By default stream_final=True.)

If this upload is initiated with stream_final=False, total_bytes will be None since it cannot be determined from the stream.

Type

Optional[int]

transmit_next_chunk(transport, timeout=(61, 60))

Transmit the next chunk of the resource to be uploaded.

If the current upload was initiated with stream_final=False, this method will dynamically determine if the upload has completed. The upload will be considered complete if the stream produces fewer than chunk_size bytes when a chunk is read from it.

In the case of failure, an exception is thrown that preserves the failed response:

>>> error = None
>>> try:
...     upload.transmit_next_chunk(transport)
... except resumable_media.InvalidResponse as caught_exc:
...     error = caught_exc
...
>>> error
InvalidResponse('Request failed with status code', 400,
                'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PERMANENT_REDIRECT: 308>)
>>> error.response
<Response [400]>

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

Raises

• InvalidResponse – If the status code is not 200 or http.client.PERMANENT_REDIRECT.

• DataCorruption – If this is the final chunk, a checksum validation was requested, and the checksum does not match or is not available.

class google.resumable_media.requests.SimpleUpload(upload_url, headers=None)

Upload a resource to a Google API.

A simple media upload sends no metadata and completes the upload in a single request.

Parameters

• upload_url (str) – The URL where the content will be uploaded.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with the request, e.g. headers for encrypted data.

upload_url

The URL where the content will be uploaded.

Type

str

property finished

Flag indicating if the upload has completed.

Type

bool

transmit(transport, data, content_type, timeout=(61, 60))

Transmit the resource to be uploaded.

Parameters

• transport (Session) – A requests object which can make authenticated requests.

• data (bytes) – The resource content to be uploaded.

• content_type (str) – The content type of the resource, e.g. a JPEG image has content type image/jpeg.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

class google.resumable_media.requests.XMLMPUContainer(upload_url, filename, headers=None, upload_id=None)

Initiate and close an upload using the XML MPU API.

An XML MPU sends an initial request and then receives an upload ID. Using the upload ID, the upload is then done in numbered parts and the parts can be uploaded concurrently.

In order to avoid concurrency issues with this container object, the uploading of individual parts is handled separately, by XMLMPUPart objects spawned from this container class. The XMLMPUPart objects are not necessarily in the same process as the container, so they do not update the container automatically.

MPUs are sometimes referred to as “Multipart Uploads”, which is ambiguous given the JSON multipart upload, so the abbreviation “MPU” will be used throughout.

See: https://cloud.google.com/storage/docs/multipart-uploads

Parameters

• upload_url (str) – The URL of the object (without query parameters). The initiate, PUT, and finalization requests will all use this URL, with varying query parameters.

• headers (Optional[Mapping[str, str]]) – Extra headers that should be sent with the initiate() request, e.g. headers for encrypted data. These headers will be propagated to individual XMLMPUPart objects spawned from this container as well.

upload_url

The URL where the content will be uploaded.

Type

str

upload_id

The ID of the upload from the initialization response.

Type

Optional(int)

cancel(transport, timeout=(61, 60))

Cancel an MPU request and permanently delete any uploaded parts.

This cannot be undone.

Parameters

• transport (object) – An object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

finalize(transport, timeout=(61, 60))

Finalize an MPU request with all the parts.

Parameters

• transport (object) – An object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

property finished

Flag indicating if the upload has completed.

Type

bool

initiate(transport, content_type, timeout=(61, 60))

Initiate an MPU and record the upload ID.

Parameters

• transport (object) – An object which can make authenticated requests.

• content_type (str) – The content type of the resource, e.g. a JPEG image has content type image/jpeg.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response

register_part(part_number, etag)

Register an uploaded part by part number and corresponding etag.

XMLMPUPart objects represent individual parts, and their part number and etag can be registered to the container object with this method and therefore incorporated in the finalize() call to finish the upload.

This method accepts part_number and etag, but not XMLMPUPart objects themselves, to reduce the complexity involved in running XMLMPUPart uploads in separate processes.

Parameters

• part_number (int) – The part number. Parts are assembled into the final uploaded object with finalize() in order of their part numbers.

• etag (str) – The etag included in the server response after upload.

class google.resumable_media.requests.XMLMPUPart(upload_url, upload_id, filename, start, end, part_number, headers=None, checksum=None)

property finished

Flag indicating if the upload has completed.

Type

bool

upload(transport, timeout=(61, 60))

Upload the part.

Parameters

• transport (object) – An object which can make authenticated requests.

• timeout (Optional[Union[float, Tuple[float, float]]]) –

  The number of seconds to wait for the server response. Depending on the retry strategy, a request may be repeated several times using the same timeout each time.

  Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

The HTTP response returned by transport.

Return type

Response