> ## Documentation Index > Fetch the complete documentation index at: https://docs.virtuous.org/llms.txt > Use this file to discover all available pages before exploring further. # Sync External Donations into Virtuous > The canonical end-to-end architecture for pushing donations from your platform into CRM+ — idempotent, reconcilable, and resilient to transient failure. This workflow describes the full architecture for a partner integration that continuously syncs donations from your platform into CRM+. It builds on [Create a Donation](/crm/workflows/create-a-donation) — the single-gift submission workflow — and extends it into a production-grade pipeline: handling the initial backfill of historical gifts, processing the steady stream of new donations as they occur on your platform, confirming outcomes via webhooks, and reconciling anything that falls through the cracks. If you have not read [Create a Donation](/crm/workflows/create-a-donation), [Transactions](/crm/concepts/transactions), and the [Webhooks and Events](/crm/webhooks/overview) section, start with those. This page assumes familiarity with those patterns. ## The architecture A robust donation sync has four moving parts: ```mermaid theme={null} graph LR Source["Your Platform
(donation event source)"] Queue["Outbound Queue
(pending submissions)"] Submitter["Submitter Worker
(POST to Virtuous)"] Virtuous["Virtuous CRM+
(holding state → nightly batch)"] Receiver["Webhook Receiver
(giftCreate handler)"] Records["Your Records
(canonical donation log)"] Source -->|new donation| Queue Queue -->|drain| Submitter Submitter -->|POST /api/v2/Gift/Transaction| Virtuous Virtuous -->|giftCreate webhook| Receiver Receiver -->|confirm| Records Records -.->|reconciliation poll| Virtuous ``` The four parts: 1. **Outbound queue** — durable storage of donations awaiting submission. New donations from your platform enter the queue; submitted donations leave it. 2. **Submitter worker** — drains the queue, calls `POST /api/v2/Gift/Transaction`, and updates each record's state to "submitted" on success. 3. **Webhook receiver** — handles incoming `giftCreate` events from Virtuous, matches them to submitted donations, and records the resulting Virtuous Gift ID. 4. **Reconciliation poller** — a periodic safety net that catches donations stuck in any intermediate state. Each part is independent and runs on its own schedule. The queue decouples them — if any one component is temporarily slow or unavailable, the others continue normally. *** ## Part 1: capture donations into the outbound queue Whenever a donation occurs on your platform — a successful Stripe charge, a completed Eventbrite registration, a confirmed event ticket sale — write it to a durable outbound queue or table. Do not call Virtuous directly from your platform's hot path. ```sql theme={null} -- A reasonable schema for the queue CREATE TABLE virtuous_donation_queue ( -- Your platform's identifiers platform_donation_id TEXT PRIMARY KEY, customer_id TEXT NOT NULL, -- which nonprofit customer this belongs to -- Donation details donor_platform_id TEXT NOT NULL, amount NUMERIC NOT NULL, currency TEXT NOT NULL DEFAULT 'USD', gift_date DATE NOT NULL, gift_type TEXT NOT NULL DEFAULT 'Cash', project_code TEXT NOT NULL, -- Full payload as JSON for replay payload JSONB NOT NULL, -- Sync state status TEXT NOT NULL DEFAULT 'pending', -- pending | submitted | confirmed | failed submitted_at TIMESTAMPTZ, confirmed_at TIMESTAMPTZ, virtuous_gift_id INTEGER, last_error TEXT, retry_count INTEGER NOT NULL DEFAULT 0, created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(), updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW() ); ``` Three things this schema gets right: * **`platform_donation_id` is the primary key.** This guarantees that a donation captured twice (a Stripe webhook retry, your platform's internal retry) is deduplicated at insertion time. Use `INSERT ... ON CONFLICT DO NOTHING` for idempotent capture. * **The full payload is stored as JSONB.** This lets the submitter resubmit without reconstructing the request and lets you replay any donation through the pipeline later. * **State is explicit.** The lifecycle (`pending → submitted → confirmed`, or `failed`) makes monitoring and reconciliation straightforward. Decouple capture from submission. Even if your Virtuous integration is temporarily unhealthy — credentials revoked, API rate-limited, network partition — your platform's donation flow should continue normally. A donation captured into the queue but not yet submitted is in a known, recoverable state. A donation lost because your integration tried to submit synchronously and the API was down is gone. *** ## Part 2: the submitter worker A worker drains the queue, sending `pending` donations to Virtuous. The worker is the single component that calls `POST /api/v2/Gift/Transaction`. ```javascript JavaScript theme={null} async function runSubmitter(customerId) { const token = await loadCustomerApiToken(customerId); const pending = await db.virtuous_donation_queue.find({ where: { customer_id: customerId, status: 'pending' }, orderBy: 'created_at', limit: 100, }); for (const donation of pending) { try { const response = await fetch( 'https://api.virtuoussoftware.com/api/v2/Gift/Transaction', { method: 'POST', headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json', }, body: JSON.stringify(donation.payload), } ); if (response.ok) { // Accepted into the holding state — still async, but no error. await db.virtuous_donation_queue.update( { platform_donation_id: donation.platform_donation_id }, { status: 'submitted', submitted_at: new Date(), last_error: null, } ); } else if (response.status === 429) { // Rate limited. Stop draining; the next run will retry. const retryAfter = parseInt(response.headers.get('Retry-After') || '60', 10); console.warn(`Rate limited for customer ${customerId}; pausing ${retryAfter}s`); return; } else if (response.status >= 500) { // Server error — leave as pending; retry on next run. const body = await response.text(); await recordRetryableError(donation, response.status, body); } else { // 4xx error — request is malformed. Mark failed to require human review. const body = await response.text(); await db.virtuous_donation_queue.update( { platform_donation_id: donation.platform_donation_id }, { status: 'failed', last_error: `${response.status}: ${body}`, } ); } } catch (err) { await recordRetryableError(donation, 0, err.message); } } } async function recordRetryableError(donation, status, message) { await db.virtuous_donation_queue.update( { platform_donation_id: donation.platform_donation_id }, { retry_count: donation.retry_count + 1, last_error: `${status}: ${message}`, // Leave status as 'pending' so the next run picks it up } ); } ``` Three patterns this worker gets right: * **Stable `transactionSource` and `transactionId`.** The payload stored in the queue contains them — they don't change between retries. This is the foundation of the idempotency guarantee with Virtuous. * **State transitions on success only.** A donation only leaves `pending` when Virtuous confirms acceptance (`2xx` response). Transient failures leave the record in `pending` for the next run to retry. * **4xx vs. 5xx differentiation.** 4xx is a client-side problem that won't resolve on retry — mark `failed` and surface to humans. 5xx is server-side and likely transient — keep retrying. ### Cadence and concurrency Run the submitter on a schedule (every few minutes is typical) or trigger it on queue inserts. For multi-tenant integrations, run a worker per customer to isolate rate-limit budgets — see [Rate Limits](/crm/rate-limits#rate-limit-scope). Avoid concurrent submissions of the same customer's queue from multiple worker instances. Use a queue lock or "FOR UPDATE SKIP LOCKED" SQL pattern to ensure at most one worker drains a customer's queue at a time. *** ## Part 3: the webhook receiver When Virtuous's nightly batch processes a submitted Transaction, it fires a `giftCreate` webhook to your endpoint. The receiver matches the event to the pending submission and transitions the record to `confirmed`. ```javascript JavaScript theme={null} async function handleGiftCreated(event) { const gift = event.data; // Only act on gifts that came from our integration if (gift.transactionSource !== 'YourPlatform') { return; } const donation = await db.virtuous_donation_queue.findOne({ platform_donation_id: gift.transactionId, }); if (!donation) { // Webhook for a donation we don't have a record of. // Log for investigation but don't fail — could be a legitimate orphan. console.warn('giftCreate event for unknown donation', { transactionId: gift.transactionId, virtuousGiftId: gift.id, }); return; } if (donation.status === 'confirmed') { // Duplicate webhook delivery — already processed. return; } await db.virtuous_donation_queue.update( { platform_donation_id: donation.platform_donation_id }, { status: 'confirmed', confirmed_at: new Date(), virtuous_gift_id: gift.id, } ); } ``` The handler is idempotent on the `confirmed` status — a duplicate webhook delivery (which will happen, per [Idempotency and Safe Reprocessing](/crm/webhooks/idempotency-and-safe-reprocessing)) is a no-op. See [Webhooks Overview](/crm/webhooks/overview) for the surrounding receiver pattern including signature verification. ### Handling Contact webhooks If your integration submits donations from new donors, you may also receive `contactCreate` events when the nightly batch creates the donor's Contact record. These don't need special handling for the donation flow — the donor's Contact ID is also embedded in the `giftCreate` event's payload (`contactId`). Subscribe to `contactCreate` only if your integration also tracks Contacts independently. *** ## Part 4: the reconciliation poller Webhooks are the primary outcome-detection mechanism, but a small fraction of donations will not produce a webhook: events lost during an outage, deliveries that exhausted retries, or Transactions that landed in the [needs-update](/crm/concepts/transactions#the-needs-update-bucket) bucket because matching failed. A periodic reconciliation poll catches these. ```javascript JavaScript theme={null} async function runReconciliation(customerId) { const token = await loadCustomerApiToken(customerId); // Find submitted-but-not-confirmed donations older than a reasonable window const stale = await db.virtuous_donation_queue.find({ where: { customer_id: customerId, status: 'submitted', submitted_at: { lt: '24 hours ago' }, }, limit: 100, }); for (const donation of stale) { const response = await fetch( `https://api.virtuoussoftware.com/api/Gift/YourPlatform/${donation.platform_donation_id}`, { headers: { Authorization: `Bearer ${token}` } } ); if (response.ok) { // Gift exists — webhook was lost but the Transaction succeeded. const gift = await response.json(); await db.virtuous_donation_queue.update( { platform_donation_id: donation.platform_donation_id }, { status: 'confirmed', confirmed_at: new Date(), virtuous_gift_id: gift.id, } ); } else if (response.status === 404) { // Transaction was accepted but never produced a real Gift. // Likely landed in needs-update for manual resolution. await db.virtuous_donation_queue.update( { platform_donation_id: donation.platform_donation_id }, { status: 'needs_review', last_error: 'Transaction never resolved — likely in needs-update bucket', } ); } } } ``` Run reconciliation on a slower cadence than the submitter (every few hours is reasonable). The window between submission and reconciliation should be longer than the nightly batch's typical processing delay — 24 hours is safe. Reconciliation is the most important safety net in the architecture. Without it, a missed webhook means a donation that exists in Virtuous but is permanently marked "submitted" in your records — a silent inconsistency that compounds over time. Even if webhook delivery is highly reliable, run reconciliation on a slow cadence as defense in depth. See [Reconcile Failed Syncs](/crm/workflows/reconcile-failed-syncs) for the broader reconciliation pattern, including how to handle records stuck in the needs-update bucket. *** ## Initial historical load Most partner integrations need to import some historical donation data on the first connection — either to bring a new customer's existing Stripe history into a new Virtuous instance, or to backfill data that pre-dates the integration. The same architecture handles both ongoing sync and historical load: Iterate your platform's historical donation data and insert each donation into `virtuous_donation_queue` with status `pending`. For very large backfills (tens of thousands of records), insert in batches with `ON CONFLICT DO NOTHING` to handle re-runs. The submitter doesn't distinguish historical from current — it processes all `pending` records. Throttle the worker's cadence if the backfill would exceed the rate limit (1,500 requests/hour per organization — see [Rate Limits](/crm/rate-limits)). Virtuous fires `giftCreate` for every Transaction the batch processes, including backfilled ones. Your normal receiver path handles them. Confirm the count of `confirmed` donations matches the count of donations you intended to load. Investigate anything in `failed` or `needs_review`. For backfills larger than what fits in a single rate-limit window, plan the load to span multiple windows. A 24,000-donation backfill at the maximum submission rate (1,500/hour) takes 16 hours; spread it overnight to avoid affecting daytime sync of new donations. Coordinate historical loads with the nonprofit customer. A 50,000-donation backfill triggers 50,000 `giftCreate` webhooks to your endpoint over the following nights, and produces a wave of changes visible in the Virtuous UI. The customer's team should know to expect this and have time to validate the data before treating the backfilled gifts as part of normal reporting. *** ## Multi-tenant considerations Partner integrations serving multiple nonprofit customers need a few additional patterns: ### Per-customer credentials Each nonprofit customer has their own Virtuous API token. The submitter and reconciliation poller must load the correct token for the customer they are processing. Use a secrets manager keyed by your `customer_id`. ### Per-customer rate-limit isolation The 1,500-per-hour rate limit applies per Virtuous organization (one customer = one bucket). A burst in one customer's sync does not affect others. Run a per-customer worker rather than a global one — this prevents one busy customer from monopolizing a global queue at the expense of others. ### Per-customer webhook subscriptions Each customer's Virtuous organization needs its own webhook subscription pointing at your endpoint. Create the subscription during customer onboarding via `POST /api/Webhook` against that customer's organization. Verify it remains active periodically — see [Webhooks Overview](/crm/webhooks/overview#managing-webhook-subscriptions). ### Per-customer reconciliation Run reconciliation per customer, not globally. The `customer_id` column in your queue scopes the work — the reconciliation poller for customer A doesn't touch customer B's records. *** ## Monitoring Track these metrics on the sync pipeline: | Metric | Why | | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | **Queue depth (pending)** | A growing depth indicates the submitter can't keep up — likely rate-limit or API issue. | | **Time from pending → confirmed** | The normal value is roughly the nightly batch window plus webhook delivery time. A growing value indicates batch delays or webhook outages. | | **Count in failed status** | Each `failed` record needs human investigation. Alert if this grows. | | **Count in needs\_review status** | Records stuck because Virtuous's matching couldn't auto-resolve them. Alert at a non-zero threshold. | | **giftCreate webhook arrival rate** | Should track the submission rate at roughly a nightly-batch delay. A divergence indicates webhook delivery issues. | Set up alerts for sustained anomalies in each. Most pipeline problems show up first as a divergence in one of these metrics before producing user-visible incidents. *** ## End-to-end checklist Before considering a donation sync integration production-ready, confirm: * Captures donations into a durable queue keyed by your platform's stable identifier. * Submitter uses stable `transactionSource` and `transactionId` across retries — never regenerated. * Submitter differentiates retryable (5xx, network errors) from permanent (4xx) failures. * Webhook receiver verifies signatures (see [Signature Verification](/crm/webhooks/signature-verification)). * Webhook receiver is idempotent — a duplicate delivery does not double-count anything. * Reconciliation poller runs on a schedule and updates status for previously-missed deliveries. * Multi-tenant: per-customer credentials, queues, and webhook subscriptions. * Monitoring covers queue depth, failure rate, and webhook arrival rate. * Failure states (`failed`, `needs_review`) surface to humans for investigation. *** ## Where to go next The deeper treatment of reconciliation — handling needs-update, manual resolution, and recovery patterns. What to do when matching fails and produces duplicate Contacts or Gifts. Extend this one-way donation sync into a continuous two-way sync of Contacts and Gifts. A complete integration recipe that instantiates this architecture for Stripe-sourced donations.