Fixes #760 /claim #760

What kind of change does this PR introduce?

Feature

Summary

This PR introduces the CursorReplayStrategy feature, which allows the AI model to suggest coordinates for a red dot on a screenshot representing the intended action target location. This new strategy provides the ability to visualize and self-correct initial suggestions, improving replay accuracy. It also includes an update to the screenshot capturing functionality to support capturing all monitors.

Checklist

  • My code follows the style guidelines of OpenAdapt
  • I have performed a self-review of my code
  • If applicable, I have added tests to prove my fix is functional/effective
  • I have linted my code locally prior to submission
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
  • New and existing unit tests pass locally with my changes

How can your code be run and tested?

  1. Test the CursorReplayStrategy by running the AI model and verifying the red dot is correctly suggested and visualized on the screenshot.
  2. Use the self-correction mechanism to adjust the red dot position and verify the corrections are applied accurately.

Other information

No additional context needed.


Claim

Total prize pool $1,000
Total paid $0
Status Pending
Submitted July 04, 2024
Last updated July 04, 2024

Contributors

DA

David Anyatonwu

@onyedikachi-david

100%

Sponsors

OP

OpenAdaptAI

@OpenAdaptAI

$1,000