Sync External Donations into Virtuous

This workflow describes the full architecture for a partner integration that continuously syncs donations from your platform into CRM+. It builds on Create a Donation — the single-gift submission workflow — and extends it into a production-grade pipeline: handling the initial backfill of historical gifts, processing the steady stream of new donations as they occur on your platform, confirming outcomes via webhooks, and reconciling anything that falls through the cracks. If you have not read Create a Donation, Transactions, and the Webhooks and Events section, start with those. This page assumes familiarity with those patterns.

The architecture

A robust donation sync has four moving parts: The four parts:

Outbound queue — durable storage of donations awaiting submission. New donations from your platform enter the queue; submitted donations leave it.
Submitter worker — drains the queue, calls POST /api/v2/Gift/Transaction, and updates each record’s state to “submitted” on success.
Webhook receiver — handles incoming giftCreate events from Virtuous, matches them to submitted donations, and records the resulting Virtuous Gift ID.
Reconciliation poller — a periodic safety net that catches donations stuck in any intermediate state.

Each part is independent and runs on its own schedule. The queue decouples them — if any one component is temporarily slow or unavailable, the others continue normally.

Part 1: capture donations into the outbound queue

Whenever a donation occurs on your platform — a successful Stripe charge, a completed Eventbrite registration, a confirmed event ticket sale — write it to a durable outbound queue or table. Do not call Virtuous directly from your platform’s hot path.

-- A reasonable schema for the queue
CREATE TABLE virtuous_donation_queue (
  -- Your platform's identifiers
  platform_donation_id TEXT PRIMARY KEY,
  customer_id TEXT NOT NULL,             -- which nonprofit customer this belongs to

  -- Donation details
  donor_platform_id TEXT NOT NULL,
  amount NUMERIC NOT NULL,
  currency TEXT NOT NULL DEFAULT 'USD',
  gift_date DATE NOT NULL,
  gift_type TEXT NOT NULL DEFAULT 'Cash',
  project_code TEXT NOT NULL,

  -- Full payload as JSON for replay
  payload JSONB NOT NULL,

  -- Sync state
  status TEXT NOT NULL DEFAULT 'pending',   -- pending | submitted | confirmed | failed
  submitted_at TIMESTAMPTZ,
  confirmed_at TIMESTAMPTZ,
  virtuous_gift_id INTEGER,
  last_error TEXT,
  retry_count INTEGER NOT NULL DEFAULT 0,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
  updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

Three things this schema gets right:

platform_donation_id is the primary key. This guarantees that a donation captured twice (a Stripe webhook retry, your platform’s internal retry) is deduplicated at insertion time. Use INSERT ... ON CONFLICT DO NOTHING for idempotent capture.
The full payload is stored as JSONB. This lets the submitter resubmit without reconstructing the request and lets you replay any donation through the pipeline later.
State is explicit. The lifecycle (pending → submitted → confirmed, or failed) makes monitoring and reconciliation straightforward.

Decouple capture from submission. Even if your Virtuous integration is temporarily unhealthy — credentials revoked, API rate-limited, network partition — your platform’s donation flow should continue normally. A donation captured into the queue but not yet submitted is in a known, recoverable state. A donation lost because your integration tried to submit synchronously and the API was down is gone.

Part 2: the submitter worker

A worker drains the queue, sending pending donations to Virtuous. The worker is the single component that calls POST /api/v2/Gift/Transaction.

JavaScript

async function runSubmitter(customerId) {
  const token = await loadCustomerApiToken(customerId);
  const pending = await db.virtuous_donation_queue.find({
    where: { customer_id: customerId, status: 'pending' },
    orderBy: 'created_at',
    limit: 100,
  });

  for (const donation of pending) {
    try {
      const response = await fetch(
        'https://api.virtuoussoftware.com/api/v2/Gift/Transaction',
        {
          method: 'POST',
          headers: {
            Authorization: `Bearer ${token}`,
            'Content-Type': 'application/json',
          },
          body: JSON.stringify(donation.payload),
        }
      );

      if (response.ok) {
        // Accepted into the holding state — still async, but no error.
        await db.virtuous_donation_queue.update(
          { platform_donation_id: donation.platform_donation_id },
          {
            status: 'submitted',
            submitted_at: new Date(),
            last_error: null,
          }
        );
      } else if (response.status === 429) {
        // Rate limited. Stop draining; the next run will retry.
        const retryAfter = parseInt(response.headers.get('Retry-After') || '60', 10);
        console.warn(`Rate limited for customer ${customerId}; pausing ${retryAfter}s`);
        return;
      } else if (response.status >= 500) {
        // Server error — leave as pending; retry on next run.
        const body = await response.text();
        await recordRetryableError(donation, response.status, body);
      } else {
        // 4xx error — request is malformed. Mark failed to require human review.
        const body = await response.text();
        await db.virtuous_donation_queue.update(
          { platform_donation_id: donation.platform_donation_id },
          {
            status: 'failed',
            last_error: `${response.status}: ${body}`,
          }
        );
      }
    } catch (err) {
      await recordRetryableError(donation, 0, err.message);
    }
  }
}

async function recordRetryableError(donation, status, message) {
  await db.virtuous_donation_queue.update(
    { platform_donation_id: donation.platform_donation_id },
    {
      retry_count: donation.retry_count + 1,
      last_error: `${status}: ${message}`,
      // Leave status as 'pending' so the next run picks it up
    }
  );
}

Three patterns this worker gets right:

Stable transactionSource and transactionId. The payload stored in the queue contains them — they don’t change between retries. This is the foundation of the idempotency guarantee with Virtuous.
State transitions on success only. A donation only leaves pending when Virtuous confirms acceptance (2xx response). Transient failures leave the record in pending for the next run to retry.
4xx vs. 5xx differentiation. 4xx is a client-side problem that won’t resolve on retry — mark failed and surface to humans. 5xx is server-side and likely transient — keep retrying.

Cadence and concurrency

Run the submitter on a schedule (every few minutes is typical) or trigger it on queue inserts. For multi-tenant integrations, run a worker per customer to isolate rate-limit budgets — see Rate Limits. Avoid concurrent submissions of the same customer’s queue from multiple worker instances. Use a queue lock or “FOR UPDATE SKIP LOCKED” SQL pattern to ensure at most one worker drains a customer’s queue at a time.

Part 3: the webhook receiver

When Virtuous’s nightly batch processes a submitted Transaction, it fires a giftCreate webhook to your endpoint. The receiver matches the event to the pending submission and transitions the record to confirmed.

JavaScript

async function handleGiftCreated(event) {
  const gift = event.data;

  // Only act on gifts that came from our integration
  if (gift.transactionSource !== 'YourPlatform') {
    return;
  }

  const donation = await db.virtuous_donation_queue.findOne({
    platform_donation_id: gift.transactionId,
  });

  if (!donation) {
    // Webhook for a donation we don't have a record of.
    // Log for investigation but don't fail — could be a legitimate orphan.
    console.warn('giftCreate event for unknown donation', {
      transactionId: gift.transactionId,
      virtuousGiftId: gift.id,
    });
    return;
  }

  if (donation.status === 'confirmed') {
    // Duplicate webhook delivery — already processed.
    return;
  }

  await db.virtuous_donation_queue.update(
    { platform_donation_id: donation.platform_donation_id },
    {
      status: 'confirmed',
      confirmed_at: new Date(),
      virtuous_gift_id: gift.id,
    }
  );
}

The handler is idempotent on the confirmed status — a duplicate webhook delivery (which will happen, per Idempotency and Safe Reprocessing) is a no-op. See Webhooks Overview for the surrounding receiver pattern including signature verification.

Handling Contact webhooks

If your integration submits donations from new donors, you may also receive contactCreate events when the nightly batch creates the donor’s Contact record. These don’t need special handling for the donation flow — the donor’s Contact ID is also embedded in the giftCreate event’s payload (contactId). Subscribe to contactCreate only if your integration also tracks Contacts independently.

Part 4: the reconciliation poller

Webhooks are the primary outcome-detection mechanism, but a small fraction of donations will not produce a webhook: events lost during an outage, deliveries that exhausted retries, or Transactions that landed in the needs-update bucket because matching failed. A periodic reconciliation poll catches these.

JavaScript

async function runReconciliation(customerId) {
  const token = await loadCustomerApiToken(customerId);

  // Find submitted-but-not-confirmed donations older than a reasonable window
  const stale = await db.virtuous_donation_queue.find({
    where: {
      customer_id: customerId,
      status: 'submitted',
      submitted_at: { lt: '24 hours ago' },
    },
    limit: 100,
  });

  for (const donation of stale) {
    const response = await fetch(
      `https://api.virtuoussoftware.com/api/Gift/YourPlatform/${donation.platform_donation_id}`,
      { headers: { Authorization: `Bearer ${token}` } }
    );

    if (response.ok) {
      // Gift exists — webhook was lost but the Transaction succeeded.
      const gift = await response.json();
      await db.virtuous_donation_queue.update(
        { platform_donation_id: donation.platform_donation_id },
        {
          status: 'confirmed',
          confirmed_at: new Date(),
          virtuous_gift_id: gift.id,
        }
      );
    } else if (response.status === 404) {
      // Transaction was accepted but never produced a real Gift.
      // Likely landed in needs-update for manual resolution.
      await db.virtuous_donation_queue.update(
        { platform_donation_id: donation.platform_donation_id },
        {
          status: 'needs_review',
          last_error: 'Transaction never resolved — likely in needs-update bucket',
        }
      );
    }
  }
}

Run reconciliation on a slower cadence than the submitter (every few hours is reasonable). The window between submission and reconciliation should be longer than the nightly batch’s typical processing delay — 24 hours is safe.

Reconciliation is the most important safety net in the architecture. Without it, a missed webhook means a donation that exists in Virtuous but is permanently marked “submitted” in your records — a silent inconsistency that compounds over time. Even if webhook delivery is highly reliable, run reconciliation on a slow cadence as defense in depth.

See Reconcile Failed Syncs for the broader reconciliation pattern, including how to handle records stuck in the needs-update bucket.

Initial historical load

Most partner integrations need to import some historical donation data on the first connection — either to bring a new customer’s existing Stripe history into a new Virtuous instance, or to backfill data that pre-dates the integration. The same architecture handles both ongoing sync and historical load:

Capture all historical donations into the queue

Iterate your platform’s historical donation data and insert each donation into virtuous_donation_queue with status pending. For very large backfills (tens of thousands of records), insert in batches with ON CONFLICT DO NOTHING to handle re-runs.

Let the normal submitter drain the queue

The submitter doesn’t distinguish historical from current — it processes all pending records. Throttle the worker’s cadence if the backfill would exceed the rate limit (1,500 requests/hour per organization — see Rate Limits).

Let the webhook receiver confirm outcomes

Virtuous fires giftCreate for every Transaction the batch processes, including backfilled ones. Your normal receiver path handles them.

Run reconciliation after the backfill completes

Confirm the count of confirmed donations matches the count of donations you intended to load. Investigate anything in failed or needs_review.

For backfills larger than what fits in a single rate-limit window, plan the load to span multiple windows. A 24,000-donation backfill at the maximum submission rate (1,500/hour) takes 16 hours; spread it overnight to avoid affecting daytime sync of new donations.

Coordinate historical loads with the nonprofit customer. A 50,000-donation backfill triggers 50,000 giftCreate webhooks to your endpoint over the following nights, and produces a wave of changes visible in the Virtuous UI. The customer’s team should know to expect this and have time to validate the data before treating the backfilled gifts as part of normal reporting.

Multi-tenant considerations

Partner integrations serving multiple nonprofit customers need a few additional patterns:

Per-customer credentials

Each nonprofit customer has their own Virtuous API token. The submitter and reconciliation poller must load the correct token for the customer they are processing. Use a secrets manager keyed by your customer_id.

Per-customer rate-limit isolation

The 1,500-per-hour rate limit applies per Virtuous organization (one customer = one bucket). A burst in one customer’s sync does not affect others. Run a per-customer worker rather than a global one — this prevents one busy customer from monopolizing a global queue at the expense of others.

Per-customer webhook subscriptions

Each customer’s Virtuous organization needs its own webhook subscription pointing at your endpoint. Create the subscription during customer onboarding via POST /api/Webhook against that customer’s organization. Verify it remains active periodically — see Webhooks Overview.

Per-customer reconciliation

Run reconciliation per customer, not globally. The customer_id column in your queue scopes the work — the reconciliation poller for customer A doesn’t touch customer B’s records.

Monitoring

Track these metrics on the sync pipeline:

Metric	Why
Queue depth (pending)	A growing depth indicates the submitter can’t keep up — likely rate-limit or API issue.
Time from pending → confirmed	The normal value is roughly the nightly batch window plus webhook delivery time. A growing value indicates batch delays or webhook outages.
Count in failed status	Each `failed` record needs human investigation. Alert if this grows.
Count in needs_review status	Records stuck because Virtuous’s matching couldn’t auto-resolve them. Alert at a non-zero threshold.
giftCreate webhook arrival rate	Should track the submission rate at roughly a nightly-batch delay. A divergence indicates webhook delivery issues.

Set up alerts for sustained anomalies in each. Most pipeline problems show up first as a divergence in one of these metrics before producing user-visible incidents.

End-to-end checklist

Before considering a donation sync integration production-ready, confirm:

Captures donations into a durable queue keyed by your platform’s stable identifier.
Submitter uses stable transactionSource and transactionId across retries — never regenerated.
Submitter differentiates retryable (5xx, network errors) from permanent (4xx) failures.
Webhook receiver verifies signatures (see Signature Verification).
Webhook receiver is idempotent — a duplicate delivery does not double-count anything.
Reconciliation poller runs on a schedule and updates status for previously-missed deliveries.
Multi-tenant: per-customer credentials, queues, and webhook subscriptions.
Monitoring covers queue depth, failure rate, and webhook arrival rate.
Failure states (failed, needs_review) surface to humans for investigation.

Where to go next

Reconcile Failed Syncs

The deeper treatment of reconciliation — handling needs-update, manual resolution, and recovery patterns.

Handle Duplicate Records

What to do when matching fails and produces duplicate Contacts or Gifts.

Build a Two-Way Sync

Extend this one-way donation sync into a continuous two-way sync of Contacts and Gifts.

Stripe to Virtuous CRM

A complete integration recipe that instantiates this architecture for Stripe-sourced donations.

​The architecture

​Part 1: capture donations into the outbound queue

​Part 2: the submitter worker

​Cadence and concurrency

​Part 3: the webhook receiver

​Handling Contact webhooks

​Part 4: the reconciliation poller

​Initial historical load

​Multi-tenant considerations

​Per-customer credentials

​Per-customer rate-limit isolation

​Per-customer webhook subscriptions

​Per-customer reconciliation

​Monitoring

​End-to-end checklist

​Where to go next

Reconcile Failed Syncs

Handle Duplicate Records

Build a Two-Way Sync

Stripe to Virtuous CRM

The architecture

Part 1: capture donations into the outbound queue

Part 2: the submitter worker

Cadence and concurrency

Part 3: the webhook receiver

Handling Contact webhooks

Part 4: the reconciliation poller

Initial historical load

Multi-tenant considerations

Per-customer credentials

Per-customer rate-limit isolation

Per-customer webhook subscriptions

Per-customer reconciliation

Monitoring

End-to-end checklist

Where to go next