API Performance Tips - Virtuous API Docs

A slow integration is a bad integration. Reports that take ten minutes to load, syncs that lag hours behind, dashboards that fall over under modest customer growth — all are symptoms of integration code that doesn’t respect the API’s grain. This page covers the practical techniques that keep Raise integrations fast, efficient with rate-limit budget, and pleasant to operate. The audience is integration engineers building or optimizing Raise-backed integrations. The techniques apply equally to small integrations and large ones; the larger the integration, the more important they become.

The performance principles

Six principles, in rough order of impact:

Principle	Description
Use webhooks for change detection, not polling	Webhooks deliver changes within seconds; polling at any reasonable cadence both misses changes and wastes budget
Use the largest page size the endpoint supports	Each request consumes one rate-limit slot regardless of records returned
Cache reference data aggressively	Campaigns, Projects, MotivationCodes, CustomFields change rarely — caching them eliminates most reads
Push filters into the API rather than client-side	Filtering 500 records server-side is far cheaper than reading 50,000 and discarding 49,500
Use `IncludeDetails=false` for bulk reads	Default summary payloads are dramatically smaller than detail payloads
Choose the right endpoint for the job	`/list` is cheaper than `/query`; donor-scoped endpoints are cheaper than filtered queries

The next sections drill into each.

Principle 1: webhooks over polling

For any change-detection workflow — syncing new gifts, reacting to donor updates, processing recurring failures — webhooks deliver the change within seconds and consume zero rate-limit budget for the detection itself. Polling at any reasonable cadence is strictly worse.

The polling trap

A typical naive polling pattern:

JavaScript

// ❌ Anti-pattern: poll every 5 minutes for new gifts
setInterval(async () => {
  const response = await fetch(
    'https://prod-api.raisedonors.com/api/Gift/list?SortBy=createddatetime&Descending=true&Take=100',
    { headers: { Authorization: `Bearer ${token}` } }
  );
  const recent = await response.json();
  // Filter for gifts since last poll and process
}, 5 * 60 * 1000);

Problems with this:

288 requests per day per customer just for change detection. For a partner with 100 customers, that’s 28,800 daily requests producing no user-facing value.
5-minute lag between gift creation and detection.
Edge cases at the boundaries — gifts created exactly between polls can be missed or double-counted.
No insight into deletions or updates unless you also poll modification timestamps.

The webhook alternative

JavaScript

// ✅ Subscribe once at customer onboarding
await fetch('https://prod-api.raisedonors.com/api/Webhook', {
  method: 'POST',
  headers: { /* ... */ },
  body: JSON.stringify({
    name: 'Real-time gift sync',
    notificationUrl: 'https://partner.example.com/raise-webhooks',
    eventTypesList: [10, 11, 12],   // Gift events
    format: 1,
    status: 1,
    securityToken: generatedSecret,
  }),
});

// Receive events as they happen
app.post('/raise-webhooks', async (req, res) => {
  // Verify signature, acknowledge, queue for processing
});

Now change detection is push-based:

One subscription per customer, zero ongoing rate-limit cost for the detection.
Seconds-level latency instead of minutes.
All event types covered — creates, updates, deletes — with no extra work.
Edge cases handled by Raise — retries, ordering, etc.

When polling is still appropriate

Three legitimate cases for polling:

Case	Description
Periodic reconciliation backstop	Daily or hourly query to catch any events the webhook may have missed
Initial backfill	One-time pull of historical data when an integration first connects to a customer
Data that doesn’t produce webhook events	If a particular state change you care about doesn’t fire an event, polling may be the only path

For these cases, throttle aggressively and minimize request count. See Reconcile with CRM+: pattern 3 for the reconciliation pattern.

Principle 2: use the largest page size

For paginated reads, each request consumes one rate-limit slot regardless of how many records it returns. A Take=1000 request returns 1,000 records for the cost of one request; a Take=25 request returns 25.

When this matters most

Workflow	Take size impact
Bulk export	40× fewer requests with `Take=1000` vs. `Take=25`
Reconciliation queries	Same — bulk reads benefit dramatically
Initial backfill	Same — typically the largest workload
Interactive UI list views	Smaller pages (25–50) load faster for users

For partner integrations doing bulk reads, default to Take=1000 and only reduce if a specific endpoint enforces a lower maximum. The pagination loop is otherwise identical.

A bulk read pattern

JavaScript

async function readAllGifts(filterGroups) {
  const all = [];
  let skip = 0;
  const take = 1000;
  let total = null;

  do {
    const response = await fetch(
      'https://prod-api.raisedonors.com/api/Gift/query',
      {
        method: 'POST',
        headers: { /* ... */ },
        body: JSON.stringify({
          skip,
          take,
          sortBy: 'id',
          descending: false,
          groups: filterGroups,
        }),
      }
    );

    const page = await response.json();
    if (total === null) total = page.total;

    all.push(...page.items);
    skip += take;
  } while (skip < total);

  return all;
}

100,000 gifts read with Take=1000 produces 100 requests. The same read with Take=100 produces 1,000 requests. The same data, 10× the rate-limit cost.

Principle 3: cache reference data

Some Raise resources change rarely — Campaigns, Projects, MotivationCodes, Premiums, CustomFields. Caching these eliminates the majority of read traffic for analytics and reporting integrations.

What to cache

Resource	Typical change frequency	Recommended cache TTL
Projects	Rare (a few times per year)	1 hour
Campaigns	Occasional (a few times per quarter)	1 hour
MotivationCodes / MotivationCodeGroups	Rare	1 hour
CustomFields	Rare	1 hour
Premiums	Occasional	1 hour
EmailLists	Rare	1 hour
Gateways and similar metadata	Very rare	24 hours
Query options (integer-to-label mapping)	Effectively never (until v2)	24 hours

A reference-data cache pattern

JavaScript

class ReferenceDataCache {
  constructor(token) {
    this.token = token;
    this.cache = new Map();
    this.fetchedAt = new Map();
    this.ttls = {
      projects: 60 * 60 * 1000,         // 1 hour
      campaigns: 60 * 60 * 1000,
      motivationCodes: 60 * 60 * 1000,
      customFields: 60 * 60 * 1000,
      queryOptions: 24 * 60 * 60 * 1000, // 24 hours
    };
  }

  async getProjects() {
    return this._cachedRead('projects', '/api/Project/list?Take=1000');
  }

  async getCampaigns() {
    return this._cachedRead('campaigns', '/api/Campaign/list?Take=1000');
  }

  async getCustomFields() {
    return this._cachedRead('customFields', '/api/CustomField/list');
  }

  async getQueryOptions(queryType) {
    return this._cachedRead(`queryOptions:${queryType}`, `/api/Query/options/${queryType}`);
  }

  async _cachedRead(key, path) {
    const ttl = this.ttls[key.split(':')[0]];
    const fetchedAt = this.fetchedAt.get(key);

    if (fetchedAt && Date.now() - fetchedAt < ttl) {
      return this.cache.get(key);
    }

    const response = await fetch(
      `https://prod-api.raisedonors.com${path}`,
      { headers: { Authorization: `Bearer ${this.token}` } }
    );
    const data = await response.json();

    this.cache.set(key, data.items ?? data);
    this.fetchedAt.set(key, Date.now());
    return data.items ?? data;
  }
}

For an integration that processes 1,000 gifts and needs to look up the Campaign name for each, this is the difference between 1,000 Campaign fetches and 1 (cached for the rest of the hour).

Cache invalidation strategies

Strategy	When to use
Time-based (TTL)	The simplest pattern; works for resources that change infrequently
On-demand refresh	If a recent fetch returns a record the cache doesn’t have, refresh the cache
Manual invalidation	When the partner integration triggers a known change (e.g., creates a new Project), invalidate the cache immediately

For most reference data, time-based with a 1-hour TTL is the right default. The occasional stale lookup is acceptable; the rate-limit savings are substantial.

Query options caching

The QueryOptions discovery endpoint (GET /api/Query/options/{queryType}) returns the integer-to-label mapping for query operators and filter parameters. This effectively never changes — cache it at integration startup and refresh once a day at most:

JavaScript

const QUERY_OPTIONS_CACHE = new Map();

async function getQueryOptions(queryType, token) {
  if (QUERY_OPTIONS_CACHE.has(queryType)) {
    return QUERY_OPTIONS_CACHE.get(queryType);
  }

  const response = await fetch(
    `https://prod-api.raisedonors.com/api/Query/options/${queryType}`,
    { headers: { Authorization: `Bearer ${token}` } }
  );
  const options = await response.json();
  QUERY_OPTIONS_CACHE.set(queryType, options);
  return options;
}

See Pagination and Filtering: Discovering query options.

Principle 4: push filters into the API

A query that returns 500 records after filtering is much cheaper than one that returns 50,000 records the integration then filters client-side.

Filter at the source

Bad pattern — pull everything, filter in code:

JavaScript

// ❌ Anti-pattern: read all gifts, filter to date range client-side
const allGifts = await readAllGifts(); // Tens of thousands of records
const recentGifts = allGifts.filter((g) => new Date(g.date) > startOfYear);

Good pattern — filter at the API:

JavaScript

// ✅ Push the date filter into the query
const recentGifts = await readAllGifts([
  {
    conditions: [
      { parameter: 'date', operator: GT_OPERATOR, value: '2025-01-01' },
    ],
  },
]);

The second pattern reads only the records that match. For a customer with 50,000 historical gifts and 5,000 recent ones, the second pattern is 10× cheaper.

Combine multiple filters

Filters compose with AND semantics within a condition group. Combining narrows the result set further:

JavaScript

const targetedGifts = await readAllGifts([
  {
    conditions: [
      { parameter: 'date', operator: GT_OPERATOR, value: '2025-01-01' },
      { parameter: 'amount', operator: GT_OPERATOR, value: '100' },
      { parameter: 'isTestMode', operator: EQUALS, value: 'false' },
    ],
    conjunct: AND_CONJUNCT,
  },
]);

Three filters compose to identify recent ($100+) production gifts — one query that returns only the matches.

Use `/list` + `Filter` for free-text matching

For interactive search where the user types a query string, the simple Filter parameter on /list is cheap and fast:

JavaScript

const matches = await fetch(
  `https://prod-api.raisedonors.com/api/Donor/list?Filter=${encodeURIComponent(searchTerm)}&Take=10`,
  { headers: { Authorization: `Bearer ${token}` } }
);

Don’t fall back to POST /api/Donor/query for what’s essentially a search-as-you-type lookup — /list with Filter handles it with less overhead.

Principle 5: use `IncludeDetails=false` for bulk reads

IncludeDetails=true includes related entities (addresses, contact methods, embedded donor on gifts, etc.) in each response item. For bulk reads, this can multiply the payload size 5–10×. The Raise spec calls this out explicitly on POST /api/Donor/query:

When includeDetails=true, the response includes all related entities (DonorAddresses, DonorContactMethods) similar to the GET by ID endpoint. This may impact performance for large result sets.

Use it sparingly

When to use `IncludeDetails=true`	When to use `IncludeDetails=false`
Exporting full donor records to a downstream system	Iterating donors for aggregation
Displaying full donor cards in a UI	Building a list of donor IDs for batch operations
One-off lookups of a specific record	Steady-state sync workflows
Detail views requiring all sub-resources	Reconciliation queries

Default to IncludeDetails=false. Switch to true only when the workflow demonstrably needs the related entities and the result set is small (typically a few hundred records at most).

When you do need details on many records

For workflows that genuinely need full records across many donors, fetch them individually rather than as a bulk read:

JavaScript

// For 500 donors needing full detail, GET by ID one at a time (concurrent, throttled)
async function getFullDonorDetails(donorIds) {
  const results = await Promise.all(
    donorIds.map(async (id) => {
      await rateLimiter.acquire();
      return fetch(
        `https://prod-api.raisedonors.com/api/Donor/${id}`,
        { headers: { Authorization: `Bearer ${token}` } }
      ).then((r) => r.json());
    })
  );
  return results;
}

This is more requests than one bulk query with IncludeDetails=true, but each is small and the workload is more controlled. For very large workloads, prefer the controlled-concurrency pattern over the bulk-with-details one.

Principle 6: choose the right endpoint

Different endpoints have different costs for the same logical question. Some examples:

Donor’s gifts: scoped endpoint vs. filtered query

Pattern	Endpoint
✅ Donor-scoped	`GET /api/Donor/{donorId}/gifts`
❌ Filtered query	`POST /api/Gift/query` with `donorId` filter

The scoped endpoint is cheaper because the filter happens at the source rather than requiring full-table filter evaluation.

Donor lookup by email: search vs. query

Pattern	Endpoint
✅ Interactive search	`GET /api/Donor/search?filter=<email>`
❌ Structured query for a single lookup	`POST /api/Donor/query` with a single email condition

/search is optimized for free-text lookups and is faster than the structured query for single-record matches.

Reading reference data: list vs. query

Pattern	Endpoint
✅ Bulk list	`GET /api/Project/list?Take=1000`
❌ Query for everything	`POST /api/Project/query` with no filter

For unfiltered bulk reads, /list is cheaper than /query.

Reading a single record: get-by-ID vs. filtered query

Pattern	Endpoint
✅ Direct fetch	`GET /api/Donor/{donorId}`
❌ Single-record query	`POST /api/Donor/query` with an ID condition

Always prefer get-by-ID when you have the ID. The general principle: when multiple endpoints could answer the same question, choose the most specific one. Specificity translates to less work at the API and faster responses.

Avoiding common performance traps

A few specific anti-patterns that cause performance issues:

Sequential fetches when parallel would do

JavaScript

// ❌ Sequential — slow
for (const donorId of donorIds) {
  const details = await fetchDonor(donorId);
  process(details);
}

JavaScript

// ✅ Parallel with controlled concurrency
const pool = new ConcurrencyPool(10); // 10 parallel requests max
await Promise.all(
  donorIds.map((id) =>
    pool.run(async () => {
      const details = await fetchDonor(id);
      process(details);
    })
  )
);

For tens or hundreds of fetches, parallelism reduces total wall-clock time significantly. Cap concurrency to stay within rate limits — see Rate Limits for the broader pattern.

Repeated lookups of the same data

JavaScript

// ❌ Re-fetches the same Project for every gift
for (const gift of gifts) {
  const project = await fetchProject(gift.projects[0].projectId);
  // ...
}

JavaScript

// ✅ Cache once, reuse
const projectsCache = await fetchAllProjects();
const projectsById = new Map(projectsCache.map((p) => [p.id, p]));

for (const gift of gifts) {
  const project = projectsById.get(gift.projects[0].projectId);
  // ...
}

Cache reference data once per workload (or for the integration’s lifetime with TTL) and look up from memory.

Synchronous webhook processing

JavaScript

// ❌ Slow processing inside the webhook handler
app.post('/raise-webhooks', async (req, res) => {
  if (!verifySignature(req)) return res.status(401).send();
  await processGift(req.body); // Takes 30 seconds; Raise times out
  res.status(200).send();
});

JavaScript

// ✅ Acknowledge fast, process async
app.post('/raise-webhooks', async (req, res) => {
  if (!verifySignature(req)) return res.status(401).send();
  res.status(200).send();
  setImmediate(() => processGift(req.body)); // Or queue
});

A slow webhook handler causes Raise timeouts and triggers retries — producing duplicate deliveries the integration then has to deduplicate. Acknowledge first, process after.

Fetching unnecessary fields with `selectedColumns`

When you don’t need every field, request a subset:

JavaScript

// ❌ Returns the full record (50+ fields per gift)
const gifts = await fetch(...).then((r) => r.json());

// ✅ Returns only the fields needed
const giftsResponse = await fetch('https://prod-api.raisedonors.com/api/Gift/query', {
  method: 'POST',
  headers: { /* ... */ },
  body: JSON.stringify({
    skip: 0,
    take: 1000,
    selectedColumns: ['id', 'amount', 'date', 'donorId', 'status'],
    groups: filterGroups,
  }),
});

For workflows that only need a handful of fields, selectedColumns reduces response payload by an order of magnitude. The set of valid columns is discovered via QueryOptions.

Measuring performance in production

For partner integrations operating at scale, instrument for visibility:

Track per-endpoint latency

JavaScript

async function timedFetch(url, options) {
  const start = Date.now();
  try {
    const response = await fetch(url, options);
    metrics.timing('raise_request.latency_ms', Date.now() - start, {
      endpoint: simplifyUrl(url),
      status: response.status,
    });
    return response;
  } catch (err) {
    metrics.increment('raise_request.error', { endpoint: simplifyUrl(url) });
    throw err;
  }
}

Per-endpoint latency reveals which endpoints are slow under your customer’s data shape, and helps catch regressions.

Track request count per customer

JavaScript

metrics.increment('raise_request.count', { customer_id, endpoint });

A customer with a spike in request volume may indicate either a new high-traffic workflow or an integration bug producing extra requests. Both are worth investigating.

Track cache hit rate

JavaScript

if (cache.has(key)) {
  metrics.increment('cache.hit', { key_type: keyType });
} else {
  metrics.increment('cache.miss', { key_type: keyType });
}

A reference-data cache with a 99% hit rate is doing its job. A 30% hit rate suggests the cache’s TTL is too short or the cache key isn’t matching what the code requests.

Alert on rate-limit responses

If 429 responses occur (the spec doesn’t formally document them but they happen), alert immediately:

JavaScript

if (response.status === 429) {
  metrics.increment('raise_request.rate_limited');
  await alerter.send({ severity: 'warning', message: 'Hit Raise rate limit' });
}

A sustained rate-limit pattern indicates the integration’s traffic exceeds the customer’s budget — investigate which workflow is the culprit and apply the techniques on this page.

A performance checklist

Run through this when building or auditing an integration:

Change-detection uses webhooks, not polling, wherever possible
Bulk reads use Take=1000
Reference data (Campaigns, Projects, MotivationCodes, CustomFields, QueryOptions) is cached with appropriate TTLs
Filters are applied in the API request, not in client-side filtering of large result sets
Bulk reads use IncludeDetails=false
selectedColumns reduces payload when only a few fields are needed
Donor-scoped endpoints are used for donor-scoped reads (not filtered queries)
Get-by-ID is used when the ID is known (not filtered queries)
Concurrent fetches use a controlled-concurrency pool
Webhook handlers acknowledge fast and process async
Latency, request count, cache hit rate, and rate-limit responses are instrumented

Most of these are small individually; together they make the difference between an integration that scales gracefully and one that hits walls.

Where to go next

Error Recovery Patterns

What to do when the techniques on this page meet the reality of transient failures.

Rate Limits

The throttling patterns that pair with the performance practices.

Pagination and Filtering

The reference for the pagination, filtering, and selectedColumns mechanics used here.

Sync Architecture Patterns

The broader architectural patterns these performance techniques fit into.

​The performance principles

​Principle 1: webhooks over polling

​The polling trap

​The webhook alternative

​When polling is still appropriate

​Principle 2: use the largest page size

​When this matters most

​A bulk read pattern

​Principle 3: cache reference data

​What to cache

​A reference-data cache pattern

​Cache invalidation strategies

​Query options caching

​Principle 4: push filters into the API

​Filter at the source

​Combine multiple filters

​Use /list + Filter for free-text matching

​Principle 5: use IncludeDetails=false for bulk reads

​Use it sparingly

​When you do need details on many records

​Principle 6: choose the right endpoint

​Donor’s gifts: scoped endpoint vs. filtered query

​Donor lookup by email: search vs. query

​Reading reference data: list vs. query

​Reading a single record: get-by-ID vs. filtered query

​Avoiding common performance traps

​Sequential fetches when parallel would do

​Repeated lookups of the same data

​Synchronous webhook processing

​Fetching unnecessary fields with selectedColumns

​Measuring performance in production

​Track per-endpoint latency

​Track request count per customer

​Track cache hit rate

​Alert on rate-limit responses

​A performance checklist

​Where to go next

Error Recovery Patterns

Rate Limits

Pagination and Filtering

Sync Architecture Patterns

The performance principles

Principle 1: webhooks over polling

The polling trap

The webhook alternative

When polling is still appropriate

Principle 2: use the largest page size

When this matters most

A bulk read pattern

Principle 3: cache reference data

What to cache

A reference-data cache pattern

Cache invalidation strategies

Query options caching

Principle 4: push filters into the API

Filter at the source

Combine multiple filters

Use `/list` + `Filter` for free-text matching

Principle 5: use `IncludeDetails=false` for bulk reads

Use it sparingly

When you do need details on many records

Principle 6: choose the right endpoint

Donor’s gifts: scoped endpoint vs. filtered query

Donor lookup by email: search vs. query

Reading reference data: list vs. query

Reading a single record: get-by-ID vs. filtered query

Avoiding common performance traps

Sequential fetches when parallel would do

Repeated lookups of the same data

Synchronous webhook processing

Fetching unnecessary fields with `selectedColumns`

Measuring performance in production

Track per-endpoint latency

Track request count per customer

Track cache hit rate

Alert on rate-limit responses

A performance checklist

Where to go next