Summary

Implements issue #72 by introducing a DataSyncPlugin that replicates data from an external SQL source into internal DO SQLite with incremental checkpoints and cron scheduling.

What is included

• New plugin: plugins/data-sync
  • POST /data-sync create/update sync task
  • GET /data-sync list tasks
  • DELETE /data-sync/:name delete task
  • POST /data-sync/run/:name run task immediately
• Incremental pull strategy:
  • First run fetches ordered rows without cursor predicate
  • Next runs use cursorColumn > last_cursor_value
• Internal table management:
  • Auto-create target table
  • Auto-add missing target columns
  • Create unique index on cursor column for conflict-safe upserts
• Cron integration:
  • Scheduled sync execution via cron callbacks
  • Removes cron event on task deletion
• Runtime integration:
  • Registers plugin in worker startup
  • Exposes plugin from dist/plugins.ts
• Tests:
  • Added service tests for first sync, checkpoint sync, schema expansion/indexing, and config behavior

Claim

/claim #72

Test evidence

• npx vitest run plugins/data-sync/service.test.ts

Demo video

• Added to this PR branch: docs/demo/pr98-data-sync-demo.mp4 (commit dff89b7)
• Demo covers:
  1. Data sync plugin file layout
  2. Plugin test pass output
  3. Exposed endpoints
  4. Incremental sync core functions
  5. Runtime + cron integration

Notes

• Full test suite still has existing unrelated RLS failures in this repository; plugin-specific tests pass.