> ## Documentation Index > Fetch the complete documentation index at: https://docs.virtuous.org/llms.txt > Use this file to discover all available pages before exploring further. # API Performance Tips > Practical techniques for keeping Raise API integrations fast and within rate-limit budgets — from caching reference data to using webhooks instead of polling, with Raise-specific examples throughout. A slow integration is a bad integration. Reports that take ten minutes to load, syncs that lag hours behind, dashboards that fall over under modest customer growth — all are symptoms of integration code that doesn't respect the API's grain. This page covers the practical techniques that keep Raise integrations fast, efficient with rate-limit budget, and pleasant to operate. The audience is integration engineers building or optimizing Raise-backed integrations. The techniques apply equally to small integrations and large ones; the larger the integration, the more important they become. ## The performance principles Six principles, in rough order of impact: | Principle | Description | | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | **Use webhooks for change detection, not polling** | Webhooks deliver changes within seconds; polling at any reasonable cadence both misses changes and wastes budget | | **Use the largest page size the endpoint supports** | Each request consumes one rate-limit slot regardless of records returned | | **Cache reference data aggressively** | Campaigns, Projects, MotivationCodes, CustomFields change rarely — caching them eliminates most reads | | **Push filters into the API rather than client-side** | Filtering 500 records server-side is far cheaper than reading 50,000 and discarding 49,500 | | **Use `IncludeDetails=false` for bulk reads** | Default summary payloads are dramatically smaller than detail payloads | | **Choose the right endpoint for the job** | `/list` is cheaper than `/query`; donor-scoped endpoints are cheaper than filtered queries | The next sections drill into each. *** ## Principle 1: webhooks over polling For any change-detection workflow — syncing new gifts, reacting to donor updates, processing recurring failures — webhooks deliver the change within seconds and consume zero rate-limit budget for the detection itself. Polling at any reasonable cadence is strictly worse. ### The polling trap A typical naive polling pattern: ```javascript JavaScript theme={null} // ❌ Anti-pattern: poll every 5 minutes for new gifts setInterval(async () => { const response = await fetch( 'https://prod-api.raisedonors.com/api/Gift/list?SortBy=createddatetime&Descending=true&Take=100', { headers: { Authorization: `Bearer ${token}` } } ); const recent = await response.json(); // Filter for gifts since last poll and process }, 5 * 60 * 1000); ``` Problems with this: * **288 requests per day per customer** just for change detection. For a partner with 100 customers, that's 28,800 daily requests producing no user-facing value. * **5-minute lag** between gift creation and detection. * **Edge cases** at the boundaries — gifts created exactly between polls can be missed or double-counted. * **No insight into deletions or updates** unless you also poll modification timestamps. ### The webhook alternative ```javascript JavaScript theme={null} // ✅ Subscribe once at customer onboarding await fetch('https://prod-api.raisedonors.com/api/Webhook', { method: 'POST', headers: { /* ... */ }, body: JSON.stringify({ name: 'Real-time gift sync', notificationUrl: 'https://partner.example.com/raise-webhooks', eventTypesList: [10, 11, 12], // Gift events format: 1, status: 1, securityToken: generatedSecret, }), }); // Receive events as they happen app.post('/raise-webhooks', async (req, res) => { // Verify signature, acknowledge, queue for processing }); ``` Now change detection is push-based: * **One subscription per customer, zero ongoing rate-limit cost** for the detection. * **Seconds-level latency** instead of minutes. * **All event types covered** — creates, updates, deletes — with no extra work. * **Edge cases handled by Raise** — retries, ordering, etc. ### When polling is still appropriate Three legitimate cases for polling: | Case | Description | | -------------------------------------------- | ----------------------------------------------------------------------------------------------- | | **Periodic reconciliation backstop** | Daily or hourly query to catch any events the webhook may have missed | | **Initial backfill** | One-time pull of historical data when an integration first connects to a customer | | **Data that doesn't produce webhook events** | If a particular state change you care about doesn't fire an event, polling may be the only path | For these cases, throttle aggressively and minimize request count. See [Reconcile with CRM+: pattern 3](/raise/workflows/reconcile-with-crm-plus#pattern-3-periodic-reconciliation-backstop) for the reconciliation pattern. *** ## Principle 2: use the largest page size For paginated reads, each request consumes one rate-limit slot regardless of how many records it returns. A `Take=1000` request returns 1,000 records for the cost of one request; a `Take=25` request returns 25. ### When this matters most | Workflow | Take size impact | | ------------------------- | ------------------------------------------------- | | Bulk export | 40× fewer requests with `Take=1000` vs. `Take=25` | | Reconciliation queries | Same — bulk reads benefit dramatically | | Initial backfill | Same — typically the largest workload | | Interactive UI list views | Smaller pages (25–50) load faster for users | For partner integrations doing bulk reads, default to `Take=1000` and only reduce if a specific endpoint enforces a lower maximum. The pagination loop is otherwise identical. ### A bulk read pattern ```javascript JavaScript theme={null} async function readAllGifts(filterGroups) { const all = []; let skip = 0; const take = 1000; let total = null; do { const response = await fetch( 'https://prod-api.raisedonors.com/api/Gift/query', { method: 'POST', headers: { /* ... */ }, body: JSON.stringify({ skip, take, sortBy: 'id', descending: false, groups: filterGroups, }), } ); const page = await response.json(); if (total === null) total = page.total; all.push(...page.items); skip += take; } while (skip < total); return all; } ``` 100,000 gifts read with `Take=1000` produces 100 requests. The same read with `Take=100` produces 1,000 requests. The same data, 10× the rate-limit cost. *** ## Principle 3: cache reference data Some Raise resources change rarely — Campaigns, Projects, MotivationCodes, Premiums, CustomFields. Caching these eliminates the majority of read traffic for analytics and reporting integrations. ### What to cache | Resource | Typical change frequency | Recommended cache TTL | | -------------------------------------------- | ------------------------------------ | --------------------- | | **Projects** | Rare (a few times per year) | 1 hour | | **Campaigns** | Occasional (a few times per quarter) | 1 hour | | **MotivationCodes / MotivationCodeGroups** | Rare | 1 hour | | **CustomFields** | Rare | 1 hour | | **Premiums** | Occasional | 1 hour | | **EmailLists** | Rare | 1 hour | | **Gateways and similar metadata** | Very rare | 24 hours | | **Query options (integer-to-label mapping)** | Effectively never (until v2) | 24 hours | ### A reference-data cache pattern ```javascript JavaScript theme={null} class ReferenceDataCache { constructor(token) { this.token = token; this.cache = new Map(); this.fetchedAt = new Map(); this.ttls = { projects: 60 * 60 * 1000, // 1 hour campaigns: 60 * 60 * 1000, motivationCodes: 60 * 60 * 1000, customFields: 60 * 60 * 1000, queryOptions: 24 * 60 * 60 * 1000, // 24 hours }; } async getProjects() { return this._cachedRead('projects', '/api/Project/list?Take=1000'); } async getCampaigns() { return this._cachedRead('campaigns', '/api/Campaign/list?Take=1000'); } async getCustomFields() { return this._cachedRead('customFields', '/api/CustomField/list'); } async getQueryOptions(queryType) { return this._cachedRead(`queryOptions:${queryType}`, `/api/Query/options/${queryType}`); } async _cachedRead(key, path) { const ttl = this.ttls[key.split(':')[0]]; const fetchedAt = this.fetchedAt.get(key); if (fetchedAt && Date.now() - fetchedAt < ttl) { return this.cache.get(key); } const response = await fetch( `https://prod-api.raisedonors.com${path}`, { headers: { Authorization: `Bearer ${this.token}` } } ); const data = await response.json(); this.cache.set(key, data.items ?? data); this.fetchedAt.set(key, Date.now()); return data.items ?? data; } } ``` For an integration that processes 1,000 gifts and needs to look up the Campaign name for each, this is the difference between 1,000 Campaign fetches and 1 (cached for the rest of the hour). ### Cache invalidation strategies | Strategy | When to use | | ----------------------- | -------------------------------------------------------------------------------------------------------------------- | | **Time-based (TTL)** | The simplest pattern; works for resources that change infrequently | | **On-demand refresh** | If a recent fetch returns a record the cache doesn't have, refresh the cache | | **Manual invalidation** | When the partner integration triggers a known change (e.g., creates a new Project), invalidate the cache immediately | For most reference data, **time-based with a 1-hour TTL** is the right default. The occasional stale lookup is acceptable; the rate-limit savings are substantial. ### Query options caching The QueryOptions discovery endpoint (`GET /api/Query/options/{queryType}`) returns the integer-to-label mapping for query operators and filter parameters. This effectively never changes — cache it at integration startup and refresh once a day at most: ```javascript JavaScript theme={null} const QUERY_OPTIONS_CACHE = new Map(); async function getQueryOptions(queryType, token) { if (QUERY_OPTIONS_CACHE.has(queryType)) { return QUERY_OPTIONS_CACHE.get(queryType); } const response = await fetch( `https://prod-api.raisedonors.com/api/Query/options/${queryType}`, { headers: { Authorization: `Bearer ${token}` } } ); const options = await response.json(); QUERY_OPTIONS_CACHE.set(queryType, options); return options; } ``` See [Pagination and Filtering: Discovering query options](/raise/pagination#discovering-query-options). *** ## Principle 4: push filters into the API A query that returns 500 records after filtering is much cheaper than one that returns 50,000 records the integration then filters client-side. ### Filter at the source Bad pattern — pull everything, filter in code: ```javascript JavaScript theme={null} // ❌ Anti-pattern: read all gifts, filter to date range client-side const allGifts = await readAllGifts(); // Tens of thousands of records const recentGifts = allGifts.filter((g) => new Date(g.date) > startOfYear); ``` Good pattern — filter at the API: ```javascript JavaScript theme={null} // ✅ Push the date filter into the query const recentGifts = await readAllGifts([ { conditions: [ { parameter: 'date', operator: GT_OPERATOR, value: '2025-01-01' }, ], }, ]); ``` The second pattern reads only the records that match. For a customer with 50,000 historical gifts and 5,000 recent ones, the second pattern is 10× cheaper. ### Combine multiple filters Filters compose with AND semantics within a condition group. Combining narrows the result set further: ```javascript JavaScript theme={null} const targetedGifts = await readAllGifts([ { conditions: [ { parameter: 'date', operator: GT_OPERATOR, value: '2025-01-01' }, { parameter: 'amount', operator: GT_OPERATOR, value: '100' }, { parameter: 'isTestMode', operator: EQUALS, value: 'false' }, ], conjunct: AND_CONJUNCT, }, ]); ``` Three filters compose to identify recent (\$100+) production gifts — one query that returns only the matches. ### Use `/list` + `Filter` for free-text matching For interactive search where the user types a query string, the simple `Filter` parameter on `/list` is cheap and fast: ```javascript JavaScript theme={null} const matches = await fetch( `https://prod-api.raisedonors.com/api/Donor/list?Filter=${encodeURIComponent(searchTerm)}&Take=10`, { headers: { Authorization: `Bearer ${token}` } } ); ``` Don't fall back to `POST /api/Donor/query` for what's essentially a search-as-you-type lookup — `/list` with `Filter` handles it with less overhead. *** ## Principle 5: use `IncludeDetails=false` for bulk reads `IncludeDetails=true` includes related entities (addresses, contact methods, embedded donor on gifts, etc.) in each response item. For bulk reads, this can multiply the payload size 5–10×. The Raise spec calls this out explicitly on `POST /api/Donor/query`: > When `includeDetails=true`, the response includes all related entities (DonorAddresses, DonorContactMethods) similar to the GET by ID endpoint. This may impact performance for large result sets. ### Use it sparingly | When to use `IncludeDetails=true` | When to use `IncludeDetails=false` | | --------------------------------------------------- | ------------------------------------------------- | | Exporting full donor records to a downstream system | Iterating donors for aggregation | | Displaying full donor cards in a UI | Building a list of donor IDs for batch operations | | One-off lookups of a specific record | Steady-state sync workflows | | Detail views requiring all sub-resources | Reconciliation queries | Default to `IncludeDetails=false`. Switch to `true` only when the workflow demonstrably needs the related entities and the result set is small (typically a few hundred records at most). ### When you do need details on many records For workflows that genuinely need full records across many donors, fetch them individually rather than as a bulk read: ```javascript JavaScript theme={null} // For 500 donors needing full detail, GET by ID one at a time (concurrent, throttled) async function getFullDonorDetails(donorIds) { const results = await Promise.all( donorIds.map(async (id) => { await rateLimiter.acquire(); return fetch( `https://prod-api.raisedonors.com/api/Donor/${id}`, { headers: { Authorization: `Bearer ${token}` } } ).then((r) => r.json()); }) ); return results; } ``` This is more requests than one bulk query with `IncludeDetails=true`, but each is small and the workload is more controlled. For very large workloads, prefer the controlled-concurrency pattern over the bulk-with-details one. *** ## Principle 6: choose the right endpoint Different endpoints have different costs for the same logical question. Some examples: ### Donor's gifts: scoped endpoint vs. filtered query | Pattern | Endpoint | | ---------------- | -------------------------------------------- | | ✅ Donor-scoped | `GET /api/Donor/{donorId}/gifts` | | ❌ Filtered query | `POST /api/Gift/query` with `donorId` filter | The scoped endpoint is cheaper because the filter happens at the source rather than requiring full-table filter evaluation. ### Donor lookup by email: search vs. query | Pattern | Endpoint | | -------------------------------------- | ----------------------------------------------------- | | ✅ Interactive search | `GET /api/Donor/search?filter=` | | ❌ Structured query for a single lookup | `POST /api/Donor/query` with a single email condition | `/search` is optimized for free-text lookups and is faster than the structured query for single-record matches. ### Reading reference data: list vs. query | Pattern | Endpoint | | ---------------------- | ---------------------------------------- | | ✅ Bulk list | `GET /api/Project/list?Take=1000` | | ❌ Query for everything | `POST /api/Project/query` with no filter | For unfiltered bulk reads, `/list` is cheaper than `/query`. ### Reading a single record: get-by-ID vs. filtered query | Pattern | Endpoint | | --------------------- | -------------------------------------------- | | ✅ Direct fetch | `GET /api/Donor/{donorId}` | | ❌ Single-record query | `POST /api/Donor/query` with an ID condition | Always prefer get-by-ID when you have the ID. The general principle: when multiple endpoints could answer the same question, choose the most specific one. Specificity translates to less work at the API and faster responses. *** ## Avoiding common performance traps A few specific anti-patterns that cause performance issues: ### Sequential fetches when parallel would do ```javascript JavaScript theme={null} // ❌ Sequential — slow for (const donorId of donorIds) { const details = await fetchDonor(donorId); process(details); } ``` ```javascript JavaScript theme={null} // ✅ Parallel with controlled concurrency const pool = new ConcurrencyPool(10); // 10 parallel requests max await Promise.all( donorIds.map((id) => pool.run(async () => { const details = await fetchDonor(id); process(details); }) ) ); ``` For tens or hundreds of fetches, parallelism reduces total wall-clock time significantly. Cap concurrency to stay within rate limits — see [Rate Limits](/raise/rate-limits) for the broader pattern. ### Repeated lookups of the same data ```javascript JavaScript theme={null} // ❌ Re-fetches the same Project for every gift for (const gift of gifts) { const project = await fetchProject(gift.projects[0].projectId); // ... } ``` ```javascript JavaScript theme={null} // ✅ Cache once, reuse const projectsCache = await fetchAllProjects(); const projectsById = new Map(projectsCache.map((p) => [p.id, p])); for (const gift of gifts) { const project = projectsById.get(gift.projects[0].projectId); // ... } ``` Cache reference data once per workload (or for the integration's lifetime with TTL) and look up from memory. ### Synchronous webhook processing ```javascript JavaScript theme={null} // ❌ Slow processing inside the webhook handler app.post('/raise-webhooks', async (req, res) => { if (!verifySignature(req)) return res.status(401).send(); await processGift(req.body); // Takes 30 seconds; Raise times out res.status(200).send(); }); ``` ```javascript JavaScript theme={null} // ✅ Acknowledge fast, process async app.post('/raise-webhooks', async (req, res) => { if (!verifySignature(req)) return res.status(401).send(); res.status(200).send(); setImmediate(() => processGift(req.body)); // Or queue }); ``` A slow webhook handler causes Raise timeouts and triggers retries — producing duplicate deliveries the integration then has to deduplicate. Acknowledge first, process after. ### Fetching unnecessary fields with `selectedColumns` When you don't need every field, request a subset: ```javascript JavaScript theme={null} // ❌ Returns the full record (50+ fields per gift) const gifts = await fetch(...).then((r) => r.json()); // ✅ Returns only the fields needed const giftsResponse = await fetch('https://prod-api.raisedonors.com/api/Gift/query', { method: 'POST', headers: { /* ... */ }, body: JSON.stringify({ skip: 0, take: 1000, selectedColumns: ['id', 'amount', 'date', 'donorId', 'status'], groups: filterGroups, }), }); ``` For workflows that only need a handful of fields, `selectedColumns` reduces response payload by an order of magnitude. The set of valid columns is discovered via QueryOptions. *** ## Measuring performance in production For partner integrations operating at scale, instrument for visibility: ### Track per-endpoint latency ```javascript JavaScript theme={null} async function timedFetch(url, options) { const start = Date.now(); try { const response = await fetch(url, options); metrics.timing('raise_request.latency_ms', Date.now() - start, { endpoint: simplifyUrl(url), status: response.status, }); return response; } catch (err) { metrics.increment('raise_request.error', { endpoint: simplifyUrl(url) }); throw err; } } ``` Per-endpoint latency reveals which endpoints are slow under your customer's data shape, and helps catch regressions. ### Track request count per customer ```javascript JavaScript theme={null} metrics.increment('raise_request.count', { customer_id, endpoint }); ``` A customer with a spike in request volume may indicate either a new high-traffic workflow or an integration bug producing extra requests. Both are worth investigating. ### Track cache hit rate ```javascript JavaScript theme={null} if (cache.has(key)) { metrics.increment('cache.hit', { key_type: keyType }); } else { metrics.increment('cache.miss', { key_type: keyType }); } ``` A reference-data cache with a 99% hit rate is doing its job. A 30% hit rate suggests the cache's TTL is too short or the cache key isn't matching what the code requests. ### Alert on rate-limit responses If `429` responses occur (the spec doesn't formally document them but they happen), alert immediately: ```javascript JavaScript theme={null} if (response.status === 429) { metrics.increment('raise_request.rate_limited'); await alerter.send({ severity: 'warning', message: 'Hit Raise rate limit' }); } ``` A sustained rate-limit pattern indicates the integration's traffic exceeds the customer's budget — investigate which workflow is the culprit and apply the techniques on this page. *** ## A performance checklist Run through this when building or auditing an integration: * Change-detection uses webhooks, not polling, wherever possible * Bulk reads use `Take=1000` * Reference data (Campaigns, Projects, MotivationCodes, CustomFields, QueryOptions) is cached with appropriate TTLs * Filters are applied in the API request, not in client-side filtering of large result sets * Bulk reads use `IncludeDetails=false` * `selectedColumns` reduces payload when only a few fields are needed * Donor-scoped endpoints are used for donor-scoped reads (not filtered queries) * Get-by-ID is used when the ID is known (not filtered queries) * Concurrent fetches use a controlled-concurrency pool * Webhook handlers acknowledge fast and process async * Latency, request count, cache hit rate, and rate-limit responses are instrumented Most of these are small individually; together they make the difference between an integration that scales gracefully and one that hits walls. *** ## Where to go next What to do when the techniques on this page meet the reality of transient failures. The throttling patterns that pair with the performance practices. The reference for the pagination, filtering, and `selectedColumns` mechanics used here. The broader architectural patterns these performance techniques fit into.