Reducing Data Usage and Optimising your OdataLink Feeds

Revision as of 01:09, 8 May 2026 by Ash (talk | contribs) (moved sections to separate wiki pages)

Overview

To keep OdataLink subscriptions affordable and predictable for everyone, we've introduced a Fair Use Policy. This policy sets a monthly download limit for data retrieved from your connected data file(s).

The guidance below applies to all models connecting to Xero, Xero Practice Manager, MYOB and QuickBooks. Following these steps will:

  • Reduce your monthly download volume; and
  • Improve refresh performance

You can monitor your current usage at any time in your OdataLink Admin Portal.

Steps to monitor usage by Data file, Model, and User can be found in the How to track my OdataLink Usage support page.


What is Usage

Usage is defined as the volume of data OdataLink retrieves from your connected data files each month.

This includes data retrieved by Power BI, Excel, or other data pipeline and reporting tools, either via automatic scheduled refreshes or manual report refresh.


For everyone (accountants, bookkeepers, business users)

Please refer to the following support note detailing optimisation techniques that can be applied directly in your OdataLink Admin Portal or in your reporting architecture.

How to Reduce my OdataLink Usage - For everyday users


For Power BI, Fabric, and Excel developers

The following optimisation techniques require working inside Power Query or transformation queries in your reporting software.

5. Push date filters down to the OData feed level (query folding)

Filtering data after it has been downloaded does not reduce egress. The records have already been pulled from the company file.

Filtering must happen at the Navigation step in Power Query so that the filter is sent to OdataLink as part of the OData query (`$filter=...`) and only the matching rows are returned.

  1. Apply date filters as early as possible in the query — directly on the source table, before any merges, joins, or column expansions.
  2. Avoid applying `Table.SelectRows` over a downloaded table when an equivalent filter can be applied at the navigation step.
  3. Verify folding by right-clicking a step in Power Query → **View Native Query**. If the option is greyed out, the filter is no longer folding and is being applied client-side.

Model Date Filters vs Report Date Filters →

6. Use Incremental Refresh for time-series data

For any dataset keyed or based on a date, datetime, or timestamp column, configure Incremental Refresh so that only new or changed partitions are pulled on each refresh and not the entire history.

  • Power BI Premium / Fabric: native incremental refresh on semantic models.
  • Dataflow Gen2 (Fabric): Dataflow Gen2 incremental refresh →
  • Power BI Pro: incremental refresh is supported but requires care to ensure the date filter folds back to the OData source. Test with View Native Query before publishing.

Tip: OData feeds can be staged into a Fabric Dataflow Gen2, then consumed downstream by Power BI, Excel, Power Apps, and Power Automations.

This means OdataLink (and therefore Xero) is queried once per refresh, not once per consumer. This change can significantly cut total egress when multiple reports share the same data.


For SQL, SSIS, and Fabric Pipeline Developers

For users staging OdataLink data into SQL Server, Azure SQL, Fabric Lakehouse / Warehouse, or other persistent stores via SSIS, Azure Data Factory (ADF), or Fabric Data Pipelines.

7. Stage once, pull indefinitely

The single biggest egress saving comes from removing duplicate downloads. Avoid having multiple reports, models, or tools all pulling the same OData entity directly. Instead:

  1. Pull each OData entity into a staging table (SQL Server, Lakehouse, Warehouse) **once per refresh cycle**.
  2. Point all downstream consumers (Power BI semantic models, Excel reports, SSRS, Fabric notebooks) at the staging layer — not at OdataLink.

This collapses N reports × M refreshes into a single retrieval per cycle.

8. Implement true incremental loads with `$filter`

When loading data into a SQL DB or an Azure Lakehouse, do not truncate-and-reload on every run. Use a high-watermark column (e.g. `LastModifiedDateTime`, `UpdatedDateUTC`) and pass it into the OData URL via the `$filter` parameter so OdataLink only returns changed rows.

Example pattern: GET .../Invoices?$filter=LastModifiedDateTime gt 2026-05-01T00:00:00Z

  1. Store the maximum `LastModifiedDateTime` from the previous successful load in a control table.
  2. Use that value as the lower bound for the next pull.
  3. Merge (UPSERT) the result into the target table on the natural key.

This pattern works in SSIS (OData Source component, parameterised URL), Azure Data Factory / Fabric Data Pipelines (Copy Activity with dynamic OData query), and any custom .NET / Python query.

OdataLink OData URL structure and query parameters →

9. Select only the columns you need with `$select`

OData supports projection via `$select=Col1,Col2,...`. By default, retrieving an entity returns every column, many of which are wide free-text fields that you may never report on. Restricting projection at the URL level reduces row size, network transfer, and data file egress.

Example: GET .../Invoices?$select=InvoiceID,InvoiceNumber,Date,Total,Status

10. Use `$top` and pagination deliberately

For initial historical loads, paginate with `$top` and `$skip` (or follow the `@odata.nextLink`) at a controlled rate. Avoid full-table re-pulls in production schedules — full re-loads should be a one-time backfill, not a recurring job.

11. Schedule pipeline runs during off-peak windows and align cycles

  • Stagger pipeline schedules so that a single Xero tenant is not being hammered by overlapping pipelines.
  • Align pipeline frequency with the underlying business cycle. Daily General Ledger snapshots rarely need sub-hourly retrieval.
  • For Fabric Data Pipelines, use **dependency-based triggers** rather than fixed schedules — downstream pipelines run only when the upstream load actually produced new rows.

12. Cache reference data separately

Reference entities (Chart of Accounts, Tax Rates, Tracking Categories, Contacts where stable) change far less frequently than transactional entities (Invoices, Bills, Bank Transactions). Refresh reference tables on a slower cadence (daily or weekly) and transactional tables on a faster cadence — instead of refreshing everything together.


Quick checklist

  • [ ] Refresh schedule matches how often the report is actually used
  • [ ] Weekend / out-of-hours refreshes disabled where not needed
  • [ ] Cut-off Date configured on the model
  • [ ] Default Date Range configured on the model
  • [ ] Polling enabled
  • [ ] Date filters applied at the OData navigation step (not post-download)
  • [ ] Query folding verified via **View Native Query**
  • [ ] Incremental refresh enabled for time-series datasets
  • [ ] Pipelines stage data once into SQL / Lakehouse before downstream consumption
  • [ ] `$filter`-based incremental loads with a high-watermark column
  • [ ] `$select` used to limit columns retrieved
  • [ ] Reference and transactional data refreshed on separate cadences


Need help?

If you have followed the steps above and your usage is still high, contact OdataLink support. We can review your model configuration, refresh patterns, and pipeline design, and recommend further optimisations before any custom pricing applies.


Related Topics