Skip to content
This repository has been archived by the owner on Mar 10, 2024. It is now read-only.

feat: sync states for webhook sync.complete #2018

Merged
merged 2 commits into from
Dec 5, 2023
Merged

Conversation

tomkit
Copy link
Contributor

@tomkit tomkit commented Dec 2, 2023

For customers with SyncConfig (and not ConnectionSyncConfig), ObjectSyncs loosely start simultaneously. Using that, we can emit all sync states as part of the sync.complete webhook so they know if other syncs have been finished, which allows them to see if they can process an object that depends on another, e.g., Accounts before Contacts, without storing additional state on their side.

syncedRecordsUpToWatermark is also included for "full then incremental" syncs in the payload so they can process objects up to a specific watermark if needed.

Note: this isn't foolproof, e.g., in situations where the longest sync time runs up against the sync frequency

Test Plan

Tested locally

sync.complete webhook event

...
"related_sync_states": {
    "account": {
      "finished": true,
      "object": "account",
      "object_type": "common",
      "strategy_type": "full then incremental",
      "synced_records_up_to_watermark": 0
    },
    "contact": {
      "finished": true,
      "object": "contact",
      "object_type": "common",
      "strategy_type": "full then incremental",
      "synced_records_up_to_watermark": 1695849081247
    }
  },
  ...

Deployment instructions

[Add any special deployment instructions here]

Copy link

vercel bot commented Dec 2, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
supaglue-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 5, 2023 6:16pm
1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
management-ui ⬜️ Ignored (Inspect) Visit Preview Dec 5, 2023 6:16pm

@lucasmarshall lucasmarshall changed the title feat: sync states for webhook sync.coplete feat: sync states for webhook sync.complete Dec 2, 2023
@lucasmarshall lucasmarshall merged commit 3ac352d into main Dec 5, 2023
11 checks passed
@lucasmarshall lucasmarshall deleted the related_sync_states branch December 5, 2023 19:27
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants