Description

This PR implements the Google Cloud Store implementation and closely aligns with the AWS S3 implementation. This is an update to my previous PR I raised using the gRPC implementation. There were multiple issues with that PR, and the primary issue was the stability. This PR fixes all the underlying issues we were facing with that PR. We are using the google-cloud-storage and google-cloud-auth crates.

Note: To run the gcs_backend.json5 example that I have provided, you should ensure that the path to the credentials file is specified in the GOOGLE_APPLICATION_CREDENTIALS environment variable. Check https://docs.rs/google-cloud-storage/0.24.0/google_cloud_storage/#automatically for more details.

Fixes https://github.com/TraceMachina/nativelink/issues/659 /claim https://github.com/TraceMachina/nativelink/issues/659

Type of change

Please delete options that aren’t relevant.

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How Has This Been Tested?

I did an E2E testing using the gcs_backend.json5 example that I have provided. Along with that, I have set up a mocking framework for the GCS client and added a few tests for both the GCS store and the GCS client.

Checklist

• Updated documentation if needed
• Tests added/amended
• bazel test //... passes locally
• PR is contained in a single commit, using git amend see some docs

This change is