This PR implements the Google Cloud Store implementation and closely aligns with the AWS S3 implementation.
This utilizes the google protos. There were three primary options that I considered here (gcloud_sdk_rs, googleapis-tonic-google-storage-control-v2, or building it locally). Out of all these three options, googleapis-tonic-google-storage-control-v2
seemed to have the easiest approach to just use the rust bindings, without importing a number of libraries like reqwest, etc.
Following the same approach I used for Azure Blob Storage, I started my development by first creating a POC: https://gist.github.com/amankrx/d3861756e00e0d5157cd4230477b4024 This POC tests the operations that we will be performing in the real implementation.
It also has auth set up properly including auto-refresh of the tokens. You should ensure that you have GCS_PRIVATE_KEY
set up in the environment variable.
Fixes #659 /claim #659
Please delete options that aren’t relevant.
I did an e2e testing using the gcs_backend.json5 CAS I have provided. Along with that, I have set up a mocking framework for the GCS client and added a few tests for both the GCS store and the GCS client. I will add a few more tests before merging the PR.
bazel test //...
passes locallygit amend
see some docsAman Kumar
@amankrx
TraceMachina
@TraceMachina