On January 1, 2020 this library will no longer support Python 2 on the latest released version. Previously released library versions will continue to be available. For more information please visit Python 2 support on Google Cloud.

Storage Client

Client for interacting with the Google Cloud Storage API.

class google.cloud.storage.client.Client(project=<object object>, credentials=None, _http=None, client_info=None, client_options=None)[source]

Bases: google.cloud.client.ClientWithProject

Client to bundle configuration needed for API requests.

Parameters
  • project (str or None) – the project which the client acts on behalf of. Will be passed when creating a topic. If not passed, falls back to the default inferred from the environment.

  • credentials (Credentials) – (Optional) The OAuth2 Credentials to use for this client. If not passed (and if no _http object is passed), falls back to the default inferred from the environment.

  • _http (Session) – (Optional) HTTP object to make requests. Can be any object that defines request() with the same interface as requests.Session.request(). If not passed, an _http object is created that is bound to the credentials for the current object. This parameter should be considered private, and could change in the future.

  • client_info (ClientInfo) – The client info used to send a user-agent string along with API requests. If None, then default info will be used. Generally, you only need to set this if you’re developing your own library or partner tool.

  • client_options (ClientOptions or dict) – (Optional) Client options used to set user options on the client. API Endpoint should be set through client_options.

SCOPE = ('https://www.googleapis.com/auth/devstorage.full_control', 'https://www.googleapis.com/auth/devstorage.read_only', 'https://www.googleapis.com/auth/devstorage.read_write')

The scopes required for authenticating as a Cloud Storage consumer.

batch()[source]

Factory constructor for batch object.

Note

This will not make an HTTP request; it simply instantiates a batch object owned by this client.

Return type

google.cloud.storage.batch.Batch

Returns

The batch object created.

bucket(bucket_name, user_project=None)[source]

Factory constructor for bucket object.

Note

This will not make an HTTP request; it simply instantiates a bucket object owned by this client.

Parameters
  • bucket_name (str) – The name of the bucket to be instantiated.

  • user_project (str) – (Optional) The project ID to be billed for API requests made via the bucket.

Return type

google.cloud.storage.bucket.Bucket

Returns

The bucket object created.

classmethod create_anonymous_client()[source]

Factory: return client with anonymous credentials.

Note

Such a client has only limited access to “public” buckets: listing their contents and downloading their blobs.

Return type

google.cloud.storage.client.Client

Returns

Instance w/ anonymous credentials and no project.

create_bucket(bucket_or_name, requester_pays=None, project=None, user_project=None, location=None, predefined_acl=None, predefined_default_object_acl=None, timeout=60)[source]

API call: create a new bucket via a POST request.

See https://cloud.google.com/storage/docs/json_api/v1/buckets/insert

Parameters
  • bucket_or_name (Union[ Bucket, str, ]) – The bucket resource to pass or name to create.

  • requester_pays (bool) – DEPRECATED. Use Bucket().requester_pays instead. (Optional) Whether requester pays for API requests for this bucket and its blobs.

  • project (str) – (Optional) The project under which the bucket is to be created. If not passed, uses the project set on the client.

  • user_project (str) – (Optional) The project ID to be billed for API requests made via created bucket.

  • location (str) – (Optional) The location of the bucket. If not passed, the default location, US, will be used. See https://cloud.google.com/storage/docs/bucket-locations

  • predefined_acl (str) – (Optional) Name of predefined ACL to apply to bucket. See: https://cloud.google.com/storage/docs/access-control/lists#predefined-acl

  • predefined_default_object_acl (str) – (Optional) Name of predefined ACL to apply to bucket’s objects. See: https://cloud.google.com/storage/docs/access-control/lists#predefined-acl

  • timeout (Optional[Union[float, Tuple[float, float]]]) –

    The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

google.cloud.storage.bucket.Bucket

The newly created bucket.

Raises

google.cloud.exceptions.Conflict – If the bucket already exists.

Examples

Create a bucket using a string.

    bucket = client.create_bucket("my-bucket")
    assert isinstance(bucket, Bucket)
    # <Bucket: my-bucket>

Create a bucket using a resource.

>>> from google.cloud import storage
>>> client = storage.Client()
>>> # Set properties on a plain resource object.
>>> bucket = storage.Bucket("my-bucket-name")
>>> bucket.location = "europe-west6"
>>> bucket.storage_class = "COLDLINE"
>>> # Pass that resource object to the client.
>>> bucket = client.create_bucket(bucket)  # API request.
create_hmac_key(service_account_email, project_id=None, user_project=None, timeout=60)[source]

Create an HMAC key for a service account.

Parameters
  • service_account_email (str) – e-mail address of the service account

  • project_id (str) – (Optional) Explicit project ID for the key. Defaults to the client’s project.

  • user_project (str) – (Optional) This parameter is currently ignored.

  • timeout (float or tuple) –

    (Optional) The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Return type

Tuple[HMACKeyMetadata, str]

Returns

metadata for the created key, plus the bytes of the key’s secret, which is an 40-character base64-encoded string.

property current_batch

Currently-active batch.

Return type

google.cloud.storage.batch.Batch or NoneType (if no batch is active).

Returns

The batch at the top of the batch stack.

download_blob_to_file(blob_or_uri, file_obj, start=None, end=None)[source]

Download the contents of a blob object or blob URI into a file-like object.

Parameters
  • blob_or_uri (Union[ Blob, str, ]) – The blob resource to pass or URI to download.

  • file_obj (file) – A file handle to which to write the blob’s data.

  • start (int) – (Optional) The first byte in a range to be downloaded.

  • end (int) – (Optional) The last byte in a range to be downloaded.

Examples

Download a blob using using a blob resource.

>>> from google.cloud import storage
>>> client = storage.Client()
>>> bucket = client.get_bucket('my-bucket-name')
>>> blob = storage.Blob('path/to/blob', bucket)
>>> with open('file-to-download-to') as file_obj:
>>>     client.download_blob_to_file(blob, file_obj)  # API request.

Download a blob using a URI.

>>> from google.cloud import storage
>>> client = storage.Client()
>>> with open('file-to-download-to') as file_obj:
>>>     client.download_blob_to_file(
>>>         'gs://bucket_name/path/to/blob', file_obj)
generate_signed_post_policy_v4(bucket_name, blob_name, expiration, conditions=None, fields=None, credentials=None, virtual_hosted_style=False, bucket_bound_hostname=None, scheme=None, service_account_email=None, access_token=None)[source]

Generate a V4 signed policy object.

Note

Assumes credentials implements the google.auth.credentials.Signing interface. Also assumes credentials has a service_account_email property which identifies the credentials.

Generated policy object allows user to upload objects with a POST request.

Parameters
  • bucket_name (str) – Bucket name.

  • blob_name (str) – Object name.

  • expiration (Union[Integer, datetime.datetime, datetime.timedelta]) – Policy expiration time.

  • conditions (list) – (Optional) List of POST policy conditions, which are used to restrict what is allowed in the request.

  • fields (dict) – (Optional) Additional elements to include into request.

  • credentials (google.auth.credentials.Signing) – (Optional) Credentials object with an associated private key to sign text.

  • virtual_hosted_style (bool) – (Optional) If True, construct the URL relative to the bucket virtual hostname, e.g., ‘<bucket-name>.storage.googleapis.com’.

  • bucket_bound_hostname (str) – (Optional) If passed, construct the URL relative to the bucket-bound hostname. Value can be bare or with a scheme, e.g., ‘example.com’ or ‘http://example.com’. See: https://cloud.google.com/storage/docs/request-endpoints#cname

  • scheme (str) – (Optional) If bucket_bound_hostname is passed as a bare hostname, use this value as a scheme. https will work only when using a CDN. Defaults to "http".

  • service_account_email (str) – (Optional) E-mail address of the service account.

  • access_token (str) – (Optional) Access token for a service account.

Return type

dict

Returns

Signed POST policy.

Example

Generate signed POST policy and upload a file.

>>> from google.cloud import storage
>>> client = storage.Client()
>>> policy = client.generate_signed_post_policy_v4(
    "bucket-name",
    "blob-name",
    expiration=datetime.datetime(2020, 3, 17),
    conditions=[
        ["content-length-range", 0, 255]
    ],
    fields=[
        "x-goog-meta-hello" => "world"
    ],
)
>>> with open("bucket-name", "rb") as f:
    files = {"file": ("bucket-name", f)}
    requests.post(policy["url"], data=policy["fields"], files=files)
get_bucket(bucket_or_name, timeout=60)[source]

API call: retrieve a bucket via a GET request.

See https://cloud.google.com/storage/docs/json_api/v1/buckets/get

Parameters
  • bucket_or_name (Union[ Bucket, str, ]) – The bucket resource to pass or name to create.

  • timeout (Optional[Union[float, Tuple[float, float]]]) –

    The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

google.cloud.storage.bucket.Bucket

The bucket matching the name provided.

Raises

google.cloud.exceptions.NotFound – If the bucket is not found.

Examples

Retrieve a bucket using a string.

    try:
        bucket = client.get_bucket("my-bucket")
    except google.cloud.exceptions.NotFound:
        print("Sorry, that bucket does not exist!")

Get a bucket using a resource.

>>> from google.cloud import storage
>>> client = storage.Client()
>>> # Set properties on a plain resource object.
>>> bucket = client.get_bucket("my-bucket-name")
>>> # Time passes. Another program may have modified the bucket
... # in the meantime, so you want to get the latest state.
>>> bucket = client.get_bucket(bucket)  # API request.
get_hmac_key_metadata(access_id, project_id=None, user_project=None, timeout=60)[source]

Return a metadata instance for the given HMAC key.

Parameters
  • access_id (str) – Unique ID of an existing key.

  • project_id (str) – (Optional) Project ID of an existing key. Defaults to client’s project.

  • timeout (float or tuple) –

    (Optional) The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

  • user_project (str) – (Optional) This parameter is currently ignored.

get_service_account_email(project=None, timeout=60)[source]

Get the email address of the project’s GCS service account

Parameters
  • project (str) – (Optional) Project ID to use for retreiving GCS service account email address. Defaults to the client’s project.

  • timeout (float or tuple) –

    (Optional) The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Return type

str

Returns

service account email address

list_blobs(bucket_or_name, max_results=None, page_token=None, prefix=None, delimiter=None, versions=None, projection='noAcl', fields=None, timeout=60)[source]

Return an iterator used to find blobs in the bucket.

If user_project is set, bills the API request to that project.

Parameters
  • bucket_or_name (Union[ Bucket, str, ]) – The bucket resource to pass or name to create.

  • max_results (int) – (Optional) The maximum number of blobs to return.

  • page_token (str) – (Optional) If present, return the next batch of blobs, using the value, which must correspond to the nextPageToken value returned in the previous response. Deprecated: use the pages property of the returned iterator instead of manually passing the token.

  • prefix (str) – (Optional) Prefix used to filter blobs.

  • delimiter (str) – (Optional) Delimiter, used with prefix to emulate hierarchy.

  • versions (bool) – (Optional) Whether object versions should be returned as separate blobs.

  • projection (str) – (Optional) If used, must be ‘full’ or ‘noAcl’. Defaults to 'noAcl'. Specifies the set of properties to return.

  • fields (str) – (Optional) Selector specifying which fields to include in a partial response. Must be a list of fields. For example to get a partial response with just the next page token and the name and language of each blob returned: 'items(name,contentLanguage),nextPageToken'. See: https://cloud.google.com/storage/docs/json_api/v1/parameters#fields

  • timeout (Optional[Union[float, Tuple[float, float]]]) –

    The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Returns

Iterator of all Blob in this bucket matching the arguments.

list_buckets(max_results=None, page_token=None, prefix=None, projection='noAcl', fields=None, project=None, timeout=60)[source]

Get all buckets in the project associated to the client.

This will not populate the list of blobs available in each bucket.

    for bucket in client.list_buckets():
        print(bucket)

This implements “storage.buckets.list”.

Parameters
  • max_results (int) – (Optional) The maximum number of buckets to return.

  • page_token (str) – (Optional) If present, return the next batch of buckets, using the value, which must correspond to the nextPageToken value returned in the previous response. Deprecated: use the pages property of the returned iterator instead of manually passing the token.

  • prefix (str) – (Optional) Filter results to buckets whose names begin with this prefix.

  • projection (str) – (Optional) Specifies the set of properties to return. If used, must be ‘full’ or ‘noAcl’. Defaults to ‘noAcl’.

  • fields (str) – (Optional) Selector specifying which fields to include in a partial response. Must be a list of fields. For example to get a partial response with just the next page token and the language of each bucket returned: ‘items/id,nextPageToken’

  • project (str) – (Optional) The project whose buckets are to be listed. If not passed, uses the project set on the client.

  • timeout (float or tuple) –

    (Optional) The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Return type

Iterator

Raises

ValueError – if both project is None and the client’s project is also None.

Returns

Iterator of all Bucket belonging to this project.

list_hmac_keys(max_results=None, service_account_email=None, show_deleted_keys=None, project_id=None, user_project=None, timeout=60)[source]

List HMAC keys for a project.

Parameters
  • max_results (int) – (Optional) Max number of keys to return in a given page.

  • service_account_email (str) – (Optional) Limit keys to those created by the given service account.

  • show_deleted_keys (bool) – (Optional) Included deleted keys in the list. Default is to exclude them.

  • project_id (str) – (Optional) Explicit project ID for the key. Defaults to the client’s project.

  • user_project (str) – (Optional) This parameter is currently ignored.

  • timeout (float or tuple) –

    (Optional) The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Return type

Tuple[HMACKeyMetadata, str]

Returns

metadata for the created key, plus the bytes of the key’s secret, which is an 40-character base64-encoded string.

lookup_bucket(bucket_name, timeout=60)[source]

Get a bucket by name, returning None if not found.

You can use this if you would rather check for a None value than catching an exception:

    bucket = client.lookup_bucket("doesnt-exist")
    assert not bucket
    # None
    bucket = client.lookup_bucket("my-bucket")
    assert isinstance(bucket, Bucket)
    # <Bucket: my-bucket>
Parameters
  • bucket_name (str) – The name of the bucket to get.

  • timeout (float or tuple) –

    (Optional) The amount of time, in seconds, to wait for the server response.

    Can also be passed as a tuple (connect_timeout, read_timeout). See requests.Session.request() documentation for details.

Return type

google.cloud.storage.bucket.Bucket

Returns

The bucket matching the name provided or None if not found.