Description

Adds a reference to the nativelink benchmarks repository, currently https://github.com/skyrpex/nativelink-benchmarks.

The repository contains some GitHub actions and a website that gathers and shows data using plots, pretty much in the same fashion as https://benchmarks.mikemccandless.com/. You can find some documentation in the README.md file, but basically the project is prepared to read any amount of .jsonl files containing the benchmark data and present them using plots, organized by categories. There is a limited amount of customization for every plot, but we could make it more versatile if needed.

The benchmark.yml GitHub workflow that runs the benchmarks will run nightly, but can also be manually triggered. After running the benchmarks and updating the .jsonl files with the new data, it’ll push the changes to its main branch. Now, I’d suggest to just run the nightly benchmarks and call it a day, since it doesn’t require the nativelink to do any additional setup. The benchmarking will keep track of the last nativelink commit that was benchmarked, and will skip the process if there aren’t new commits to benchmark. Once new benchmarking data is commited to the main branch, the website should be built and deployed. This can be done with connecting the repository to Vercel, for example, and it’ll handle the rest as the project is just a simple static website.

Now, I’m ashamed to say that I’m like fish out of the bowl when trying to compile anything with nativelink, yet alone configuring a local version of nativelink and running some benchmarks with it. That’s why I left out the part of actual benchmarking, which anybody can implement in the scripts/benchmark.ts file. The repository comes with a bunch of sample data that can be removed or adapted to the real benchmark data.

I understand my attempt doesn’t completely fulfill the bounty so I’m happy to take a cut, split it, discuss it, or worst case scenario to deliver this partial contribution for free.

/claim #1700

Closes #1700.

This change is