TR

Description

This PR implements the Google Cloud Store implementation and closely aligns with the AWS S3 implementation. This utilizes the google protos. There were three primary options that I considered here (gcloud_sdk_rs, googleapis-tonic-google-storage-control-v2, or building it locally). Out of all these three options, googleapis-tonic-google-storage-control-v2 seemed to have the easiest approach to just use the rust bindings, without importing a number of libraries like reqwest, etc.

Following the same approach I used for Azure Blob Storage, I started my development by first creating a POC: https://gist.github.com/amankrx/d3861756e00e0d5157cd4230477b4024 This POC tests the operations that we will be performing in the real implementation.

It also has auth set up properly including auto-refresh of the tokens. You should ensure that you have GCS_PRIVATE_KEY set up in the environment variable.

Fixes #659 /claim #659

Type of change

Please delete options that aren’t relevant.

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

How Has This Been Tested?

I did an e2e testing using the gcs_backend.json5 CAS I have provided. Along with that, I have set up a mocking framework for the GCS client and added a few tests for both the GCS store and the GCS client. I will add a few more tests before merging the PR.

Checklist

  • Updated documentation if needed
  • Tests added/amended
  • Local tests completed
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable

Claim

Total prize pool $2,500
Total paid $0
Status Pending
Submitted January 09, 2025
Last updated January 09, 2025

Contributors

AM

Aman Kumar

@amankrx

100%

Sponsors

TR

TraceMachina

@TraceMachina

$2,500