SC
Add review calibration bench
SCIBASE-AI/SCIBASE.AI#145

/claim #15

Summary

  • add a self-contained peer-review calibration bench for the Community & User Reputation System bounty
  • compare reviewer scores against consensus panels to detect leniency/severity drift, reproducibility blind spots, and calibration gaps
  • produce role/status labels, coaching actions, quarantine recommendations, digestable evidence, and a small demo video

Why this is scoped to issue #15

This focuses on the review-quality side of community reputation: calibrated reviewers, trust signals, coaching tasks, and reproducible evidence for reputation changes. It avoids the broader account reputation/appeals work already covered by other PRs.

Demo

  • review-calibration-bench/docs/demo.mp4
  • review-calibration-bench/docs/demo.svg

Verification

  • npm run check
  • npm test
  • npm run demo
  • ffprobe review-calibration-bench/docs/demo.mp4
  • git diff --check

Claim

Total prize pool $525
Total paid $0
Status Pending
Submitted May 15, 2026
Last updated May 15, 2026

Contributors

FE

Feeelix

@FeeeeelixWong

100%

Sponsors

SC

SCIBASE.AI

@SCIBASE.AI

$525