storage: cannot access / delete versioned objects

[pdknsk]

Currently, blob.delete and bucket.delete_blob always delete the live version on buckets with versioning enabled. Other than being a missing feature, this can also cause some very unwanted behaviour, if a new version of the object is uploaded during the process of getting and deleting an object.

--- a/blob.py
+++ b/blob.py
@@ -279,7 +279,8 @@
                  (propagated from
                  :meth:`gcloud.storage.bucket.Bucket.delete_blob`).
         """
-        return self.bucket.delete_blob(self.name, client=client)
+        return self.bucket.delete_blob(self.name, self.generation, 
+            client=client)

     def download_to_file(self, file_obj, encryption_key=None, client=None):
         """Download the contents of this blob into a file-like object.
--- a/bucket.py
+++ b/bucket.py
@@ -357,7 +357,7 @@
         client.connection.api_request(method='DELETE', path=self.path,
                                       _target_object=None)

-    def delete_blob(self, blob_name, client=None):
+    def delete_blob(self, blob_name, generation=None, client=None):
         """Deletes a blob from the current bucket.

         If the blob isn't found (backend 404), raises a
@@ -392,6 +392,8 @@
         """
         client = self._require_client(client)
         blob_path = Blob.path_helper(self.path, blob_name)
+        if generation:
+            blob_path += '?generation=%i' % generation
         # We intentionally pass `_target_object=None` since a DELETE
         # request has no response value (whether in a standard request or
         # in a batch request).

[tseaver]

@pdknsk Thanks for the report! You are correct that we currently have no support for addressing a specific generation of a blob.

ISTM that the generation flag should probably be passed to Blob.__init__ (and through Bucket.blob(), because it should affect other operations as well.

API endpoints taking the generation query string parameter, which should be mapped from the blob's generation property, if set:

• https://cloud.google.com/storage/docs/json_api/v1/objects/get
• https://cloud.google.com/storage/docs/json_api/v1/objects/delete
• https://cloud.google.com/storage/docs/json_api/v1/objects/getIamPolicy
• https://cloud.google.com/storage/docs/json_api/v1/objects/insert (?!)
• https://cloud.google.com/storage/docs/json_api/v1/objects/patch (?!)
• https://cloud.google.com/storage/docs/json_api/v1/objects/setIamPolicy (?!)
• https://cloud.google.com/storage/docs/json_api/v1/objects/testIamPermissions
• https://cloud.google.com/storage/docs/json_api/v1/objects/update (?!)

The (?!) is because it confounds my expectations to be updating a historical generation of the resource: I thought I understood that updating the current version was how a new generation gets created.

API endpoints taking the source_generation query string parameter, which would be mapped from the source blob's generation property, if set:

• https://cloud.google.com/storage/docs/json_api/v1/objects/copy
• https://cloud.google.com/storage/docs/json_api/v1/objects/rewrite

We should probably also plan to support the various "conditional" parameters that these APIs support (ifGenerationMatch, ifGenerationNotMatch, ifMetagenerationMatch, ifMetagenerationNotMatch).

[pdknsk]

The (?!) is for metadata I think, which can be updated on archived objects.

https://cloud.google.com/storage/docs/object-versioning#_UpdatingMetadata

https://cloud.google.com/storage/docs/generations-preconditions#examples_of_race_conditions

[pdknsk]

Implementing preconditions is probably not necessary when generation is always set, as that takes care of every scenario in which preconditions apply, I think.

[evanj]

This is closely related but maybe not the same, let me know if I should create a new issue: I'd love support for preconditions. I personally only need "upload blob only if it does not exist", and not full support. I'm contemplating some gross hacks to do this, although for now I'm going with the "incorrect" version of checking exists() first.

[jdmargulici]

Is there a roadmap that covers this issue and/or can I volunteer some help?

[chemelnucfin]

Related to #4533 and #4490

[chemelnucfin]

Hello, feature requests will now be tracked in the project Feature Requests. I will close this issue now, please feel free to continue to address any issues/concerns here.